1
00:00:00,120 --> 00:00:07,680
Hi and welcome to actually we we're in less than 24, which is near Listowel transfer, which is a very

2
00:00:07,680 --> 00:00:09,990
cool concept which will illustrate shortly.

3
00:00:10,350 --> 00:00:12,600
So put in this notebook here.

4
00:00:13,440 --> 00:00:15,180
So what is neural cell transfer?

5
00:00:15,210 --> 00:00:19,950
So firstly, this is a picture of my wife in her carnival costume because Carnival is a big part of

6
00:00:20,670 --> 00:00:21,330
my culture.

7
00:00:21,330 --> 00:00:25,500
Back in Trinidad and Tobago, we never miss a carnival for almost 10 years now.

8
00:00:26,190 --> 00:00:28,830
So what neural cell transfer does?

9
00:00:28,830 --> 00:00:35,340
It takes an original image like this here and then to take it combines it with a style image.

10
00:00:35,490 --> 00:00:42,570
Style image is basically like another example of artistic image, and it creates that it takes this

11
00:00:42,570 --> 00:00:49,230
image and puts it into the style of this image of of of the original style image.

12
00:00:49,860 --> 00:00:53,220
So you can probably see it better when they explain it before.

13
00:00:53,790 --> 00:00:56,970
This is an example of it in the shorts and here with this example.

14
00:00:57,540 --> 00:00:59,220
So this is the input image here.

15
00:01:00,130 --> 00:01:01,170
So this is the epidemic here.

16
00:01:01,530 --> 00:01:03,120
This is a style image up here.

17
00:01:03,630 --> 00:01:10,770
And we basically it uses neural networks or deep learning to extract the style from this image here.

18
00:01:11,250 --> 00:01:17,280
And then it applies the style to this image and it does it iteratively using deep learning, and it

19
00:01:17,280 --> 00:01:22,140
produces a really cool effect in the end with this image.

20
00:01:22,170 --> 00:01:29,580
Noah has a style of that starry night painting effect that was an original style image.

21
00:01:30,000 --> 00:01:32,520
So let's take a look at how we do this in Open City.

22
00:01:33,000 --> 00:01:38,310
So already loaded these functions and test images and as well as a molehill molehill.

23
00:01:38,910 --> 00:01:42,390
And it's just download the sample image that we're going to be testing on.

24
00:01:42,960 --> 00:01:50,580
So we have seven artistic styles here, so let's take a look at his artistic style sort of directory

25
00:01:50,590 --> 00:01:50,820
here.

26
00:01:52,590 --> 00:01:56,070
We have these are the paintings.

27
00:01:56,640 --> 00:01:58,170
So there's this one here.

28
00:01:58,170 --> 00:02:01,950
Let me just make the screen a bit bigger so you can see it.

29
00:02:02,090 --> 00:02:05,640
I'm going to go through all but fit is interesting.

30
00:02:06,210 --> 00:02:08,490
You would do a starry night as well.

31
00:02:08,940 --> 00:02:11,760
The screen is quite famous.

32
00:02:13,350 --> 00:02:19,110
That's this one here with this little guy screaming so you can take a look at all the artistic styles

33
00:02:19,110 --> 00:02:19,740
if you want.

34
00:02:19,890 --> 00:02:20,880
Actually, let's go to all.

35
00:02:21,360 --> 00:02:25,530
Let's take a look at the others in the way of.

36
00:02:26,010 --> 00:02:27,800
So we have a screen you saw.

37
00:02:28,500 --> 00:02:31,620
There's this one called with me.

38
00:02:32,190 --> 00:02:35,810
Japanese also believe there's the muse.

39
00:02:36,570 --> 00:02:37,680
It's a pretty cool one here.

40
00:02:38,460 --> 00:02:44,520
Composition and weave, which I actually love this one, too, because it works quite well with the

41
00:02:45,060 --> 00:02:47,910
nasty sounds for neural cell transfer algorithm.

42
00:02:48,690 --> 00:02:50,150
So let's take a look at how we do this.

43
00:02:50,190 --> 00:02:56,070
The first thing we pointed a model here so that the models in this directory here with all of these

44
00:02:56,070 --> 00:02:57,960
artistic styles have their own model.

45
00:02:58,410 --> 00:02:59,670
It's a little T7 model.

46
00:02:59,670 --> 00:03:05,610
Well, which I believe is a caffee file from a mistake and torch available, perhaps because it's learning

47
00:03:05,610 --> 00:03:08,460
from too much that it's not a regular pity towards miles.

48
00:03:09,270 --> 00:03:19,130
So anyway, torture was sort of on coffee anyway, so we get the part to model flowerpots as well and

49
00:03:19,290 --> 00:03:21,720
dressed in a saturation scroll competition.

50
00:03:22,380 --> 00:03:28,950
And then we just go through each model and apply the model to our sample input image, which is a city,

51
00:03:28,950 --> 00:03:30,750
which is this image here.

52
00:03:31,010 --> 00:03:36,720
When we did the tilt shift effects on so we want go through all of these just yet.

53
00:03:37,230 --> 00:03:41,350
See, that fall finale actually isn't really the finale of this lesson.

54
00:03:41,350 --> 00:03:42,960
There's another part of this lesson to love.

55
00:03:43,770 --> 00:03:45,300
But anyway, that's gone.

56
00:03:45,570 --> 00:03:48,810
So let's see how we run the neural style transfer here.

57
00:03:48,990 --> 00:03:52,920
So let's zoom in a bit so you can see this is a style image.

58
00:03:53,400 --> 00:03:59,820
Remember, the style image is the Autobots image in this case that we're looking to apply to our city

59
00:03:59,820 --> 00:04:00,180
image.

60
00:04:00,540 --> 00:04:04,500
So the image being our input image, if you want to call it it.

61
00:04:04,920 --> 00:04:08,130
So we know the model here.

62
00:04:08,370 --> 00:04:14,910
Read from too much using the DNA to the DNA function so we can understand model here.

63
00:04:15,630 --> 00:04:17,280
And then we just get to height and weight.

64
00:04:17,640 --> 00:04:19,110
We resize that image.

65
00:04:20,440 --> 00:04:22,260
It's moving on that and we applied model.

66
00:04:22,260 --> 00:04:26,340
Actually, we know the image model here, 3D image using the bottom image.

67
00:04:26,940 --> 00:04:31,020
That's then that allows it to feed it into the neural style model.

68
00:04:31,020 --> 00:04:38,220
Here we just do a forward forward means we just run the input image through the model and then we get

69
00:04:38,220 --> 00:04:39,180
the outputs here.

70
00:04:39,630 --> 00:04:43,080
This is some standard mean subtraction we need to do for the model.

71
00:04:43,080 --> 00:04:46,200
That adds because of the data so the model was trained on.

72
00:04:46,200 --> 00:04:47,720
You don't need to overthink that.

73
00:04:47,730 --> 00:04:54,510
Those values do this preset values that are hardcoded here that set a specific to this model and the

74
00:04:54,510 --> 00:04:59,400
data was trained on it would be it would these numbers remain the same, but any image?

75
00:04:59,890 --> 00:05:01,200
You want to actually execute on.

76
00:05:01,800 --> 00:05:02,880
Worry about it right now.

77
00:05:03,750 --> 00:05:04,950
And then we display the output.

78
00:05:05,520 --> 00:05:06,390
So let's run.

79
00:05:06,390 --> 00:05:07,290
This could actually run.

80
00:05:07,290 --> 00:05:08,460
This really takes a minute to run.

81
00:05:08,790 --> 00:05:12,360
So I'm going to run it again, but I can see the opposite.

82
00:05:12,810 --> 00:05:14,170
So this is the original image.

83
00:05:14,220 --> 00:05:18,870
This is the artistic image that we're applying this style to the city image and look at.

84
00:05:18,870 --> 00:05:19,290
Look at it.

85
00:05:19,470 --> 00:05:21,660
Well, that looks pretty good.

86
00:05:22,560 --> 00:05:27,000
This looks like a real drawing in this artistic style of this image.

87
00:05:28,410 --> 00:05:29,850
So I'm impressed with that.

88
00:05:29,970 --> 00:05:30,720
Hope you are, too.

89
00:05:31,350 --> 00:05:36,720
Let's take a look at the starry night one little less impressive, but nonetheless, it's very nice.

90
00:05:36,720 --> 00:05:43,410
I like like in windows, a yellow, the mountains of that patchy, but it looks pretty cool as well.

91
00:05:44,250 --> 00:05:47,010
Let's see the Andy Rooney model.

92
00:05:48,510 --> 00:05:49,620
This one is pretty cool, too.

93
00:05:50,310 --> 00:05:51,630
It's not perfect, but pretty cool.

94
00:05:52,440 --> 00:05:53,820
Let's take a look at this one.

95
00:05:55,380 --> 00:06:01,530
This one isn't as impressive because it seems to apply the texture of this art, but everywhere.

96
00:06:01,530 --> 00:06:03,420
So it looks like a bit too hard to see.

97
00:06:03,420 --> 00:06:04,920
But it's nonetheless.

98
00:06:04,920 --> 00:06:05,490
It's pretty cool.

99
00:06:05,520 --> 00:06:08,210
Let's take a look at feathers, for instance.

100
00:06:08,220 --> 00:06:09,600
Generate nice images.

101
00:06:09,600 --> 00:06:11,730
However, in this case, it's not that perfect.

102
00:06:13,410 --> 00:06:17,250
This one actually looks looks quite wealthy with images, and you can see it looked.

103
00:06:17,760 --> 00:06:18,550
It looks quite good.

104
00:06:18,580 --> 00:06:24,810
It's sort of like a polygonal type art effect, and we have we have looks decent as well.

105
00:06:25,950 --> 00:06:28,290
So let's go to the last one candy.

106
00:06:30,060 --> 00:06:30,750
Not too bad.

107
00:06:31,140 --> 00:06:32,310
Actually, that wasn't the last one.

108
00:06:32,910 --> 00:06:36,510
This was the last one here, and it's quite good.

109
00:06:36,780 --> 00:06:40,470
So now we're going to take a look at using an updated NASD algorithm.

110
00:06:41,190 --> 00:06:47,040
So now let's take a look at the updated one of one of the many of the ads I should say to the amnesty

111
00:06:47,040 --> 00:06:47,670
algorithm.

112
00:06:48,090 --> 00:06:54,270
This one was called the FCC b 16 of the amnesty algorithm.

113
00:06:55,230 --> 00:07:00,840
Basically, it did something called instance normalization, which which is basically when swapping

114
00:07:01,100 --> 00:07:06,270
it was and then swapping the batch formalization for the instance, normalization and applying instant

115
00:07:06,270 --> 00:07:12,510
normalization at which reading and testing it led to much faster real time performance and probably

116
00:07:12,510 --> 00:07:16,240
better visual aesthetic performance as well.

117
00:07:16,450 --> 00:07:16,950
Parents.

118
00:07:17,670 --> 00:07:23,330
So the easy CV is the actual people publication where it was published.

119
00:07:23,350 --> 00:07:24,270
That's the conference.

120
00:07:24,750 --> 00:07:33,170
So let's take a look at implementing this so you can see we just point to the 16 folder, which has

121
00:07:33,180 --> 00:07:33,660
a model.

122
00:07:34,170 --> 00:07:38,880
We load the same images we did before, and then we run this model.

123
00:07:39,390 --> 00:07:40,050
So let's run.

124
00:07:40,050 --> 00:07:44,760
This here shouldn't take that long, since it's a fast algorithm.

125
00:07:47,470 --> 00:07:49,420
And it's already starting to generate some images.

126
00:07:50,020 --> 00:07:51,050
So let's take a look.

127
00:07:51,070 --> 00:07:57,070
This is the dark, starry night image, which actually looks much better than the previous one, in

128
00:07:57,070 --> 00:07:57,640
my opinion.

129
00:07:58,810 --> 00:07:59,620
This is one.

130
00:07:59,710 --> 00:08:01,570
This one looks spooky, and that is good.

131
00:08:02,740 --> 00:08:04,210
We'll wait for the others to complete.

132
00:08:04,390 --> 00:08:05,590
It's coming one by one.

133
00:08:08,180 --> 00:08:12,950
It's this one here is the wave, which looks better than the previous one.

134
00:08:13,640 --> 00:08:18,380
This one here looks quite nice and I'm quite pleased with the appearance of that one.

135
00:08:19,340 --> 00:08:20,750
So that's it's finished.

136
00:08:20,750 --> 00:08:22,560
No, we didn't ruin all the images in it.

137
00:08:23,150 --> 00:08:24,170
So you can take a look.

138
00:08:24,560 --> 00:08:26,540
Experiment with this if you want afterward.

139
00:08:27,170 --> 00:08:30,960
Now, let's run this on a video, because that's pretty cool.

140
00:08:30,980 --> 00:08:32,600
Imagine doing this on a video.

141
00:08:33,080 --> 00:08:40,640
We basically create video and artistic style, so have a very short clip of a scene just to a couple

142
00:08:40,640 --> 00:08:41,180
of seconds.

143
00:08:41,990 --> 00:08:47,780
So let's do this and it's we're going to use a video writer of Believe you're familiar with this video

144
00:08:47,780 --> 00:08:48,160
writer?

145
00:08:48,230 --> 00:08:54,860
We've done several times before, and we're going to lower the output after this, and I just I didn't

146
00:08:54,860 --> 00:08:56,210
change this yet.

147
00:08:57,500 --> 00:08:57,920
There we go.

148
00:08:58,100 --> 00:09:02,090
So now this is basically the same procedure as before.

149
00:09:02,600 --> 00:09:08,690
And instead, now we're writing each frame now to the out video for the video input and just using a

150
00:09:08,690 --> 00:09:09,950
different model inside.

151
00:09:10,490 --> 00:09:16,400
And we can bring this into a starry night scene, artistic scene, artistic style, applying that to

152
00:09:16,400 --> 00:09:21,860
the video, image or video of it not only video image, but a video.

153
00:09:23,420 --> 00:09:26,450
For all intents and purposes, a video is just just a sequence of images.

154
00:09:26,450 --> 00:09:29,570
Anyway, so let's just wait for this to complete.

155
00:09:29,750 --> 00:09:33,170
Takes a bit some time, but we'll wait it out.

156
00:09:38,410 --> 00:09:40,930
OK, there we go, it's finished than just over a minute.

157
00:09:41,500 --> 00:09:47,110
So, no, that's converted to me before and clearly of it right here in color.

158
00:09:49,570 --> 00:09:55,060
And of course, if you ever want to take a look at training your own artistic style model, take a look

159
00:09:55,060 --> 00:09:56,470
at this GitHub repo here.

160
00:09:57,130 --> 00:09:59,680
Let's open it up while all video converts.

161
00:10:01,650 --> 00:10:08,360
So this implementation here allows you to train your own models on it, if you ever wanted to, you

162
00:10:08,370 --> 00:10:09,150
could take a look at this.

163
00:10:09,450 --> 00:10:13,570
It may be quite interesting for some of you if you want to get deeper into this.

164
00:10:14,340 --> 00:10:16,860
So feel free to check it out.

165
00:10:16,980 --> 00:10:19,530
Got 3.9 stars, so it's quite good.

166
00:10:20,550 --> 00:10:21,820
So now let's go back to this.

167
00:10:21,840 --> 00:10:28,110
Let's play a video here and you can see, well, it's quite short, but you can see it's a pretty cool

168
00:10:28,110 --> 00:10:28,500
video.

169
00:10:30,210 --> 00:10:36,240
Now, let's show you the original one probably should have used better VIDEO But this one was quite

170
00:10:36,240 --> 00:10:36,690
short.

171
00:10:40,980 --> 00:10:44,100
So let me just bring up the video.

172
00:10:48,430 --> 00:10:50,020
So you can see it here.

173
00:10:51,010 --> 00:10:57,430
That was the original video, just a quick little clip of whether it's taken too long to process, then

174
00:10:57,430 --> 00:11:00,490
you can see the elephants here, which looks quite quite nice.

175
00:11:01,690 --> 00:11:03,490
So that's it for neural style.

176
00:11:03,490 --> 00:11:07,240
Transfer will now move on back to object to vector direction.

177
00:11:07,870 --> 00:11:12,550
The reason I have the lesson split like this is because I didn't want you guys to get to board who had

178
00:11:12,550 --> 00:11:18,340
one thing sort of destroying a variety of cool projects coming to the end of the open TV lectures because

179
00:11:18,340 --> 00:11:26,320
we're up to 40 now and this gives us like it should give you a nice experience with how we can implement

180
00:11:26,320 --> 00:11:31,810
things with open TV because obviously view is quite powerful, which is why I wanted to include it in

181
00:11:31,810 --> 00:11:32,500
this course.

182
00:11:32,860 --> 00:11:39,130
So stay tuned for solid state detectors, which is a not a single stage three single stage detectors,

183
00:11:39,130 --> 00:11:42,820
not solid state solicitors to assess the hard drives of the strikes.

184
00:11:43,480 --> 00:11:49,070
So assess these single shot detectors, which is another type of object detector like you.

185
00:11:50,080 --> 00:11:51,370
So I'll see you in that lesson.

186
00:11:51,520 --> 00:11:51,940
Thank you.