1
00:00:00,150 --> 00:00:02,790
Hi and welcome back to the course in this lesson.

2
00:00:03,270 --> 00:00:05,910
It's going to be a quick, good run through.

3
00:00:08,390 --> 00:00:09,750
Hi, welcome back to the course.

4
00:00:09,750 --> 00:00:15,060
In this lesson, we'll do a quick run-through of training your very own cats versus dogs.

5
00:00:15,060 --> 00:00:16,070
Classify it here.

6
00:00:16,080 --> 00:00:17,090
So let's get started.

7
00:00:17,100 --> 00:00:19,680
So open this notebook and let's begin.

8
00:00:19,680 --> 00:00:23,220
So we'll know that all of these functions or libraries that will be using.

9
00:00:23,220 --> 00:00:29,130
So I've already done that and download and unzip the images here, which I've already done and will

10
00:00:29,130 --> 00:00:33,130
just set our training pets here to the parts that the images were downloaded to.

11
00:00:33,630 --> 00:00:42,120
We can inspect them there so you can see what this is a cat image and let's see what a dog image looks

12
00:00:42,120 --> 00:00:42,600
like.

13
00:00:44,850 --> 00:00:46,290
Let's keep going.

14
00:00:47,790 --> 00:00:53,970
Actually, we want we won't be able to see a dog image because Dog Images has dog in a title, but it's

15
00:00:54,120 --> 00:00:57,660
that we don't have the ability to see all of them there, unfortunately.

16
00:00:58,290 --> 00:00:59,130
But that's OK.

17
00:00:59,310 --> 00:01:01,320
So let's set the pets here.

18
00:01:01,740 --> 00:01:02,850
Print the number of images.

19
00:01:02,850 --> 00:01:08,940
This is a quite large datasets, 25000 images and training get a set and twelve thousand five hundred

20
00:01:08,940 --> 00:01:11,970
in the test dataset, which is quite a lot for a test dataset.

21
00:01:12,660 --> 00:01:19,310
And you can see when you just look at a pet, one of the images here, dog images have dog in a title.

22
00:01:19,320 --> 00:01:20,550
That's how we separated out.

23
00:01:20,550 --> 00:01:23,190
There's not always the best way to separate your image classes.

24
00:01:23,640 --> 00:01:25,590
I tend to use directories for it.

25
00:01:25,800 --> 00:01:27,450
It's usually a good standard to use.

26
00:01:27,930 --> 00:01:31,440
So you have one directory called dogs when they're actually called.

27
00:01:31,620 --> 00:01:31,980
Thank you.

28
00:01:32,070 --> 00:01:33,250
Let me show you what what I mean.

29
00:01:34,140 --> 00:01:36,680
It's better to have it in your training dataset.

30
00:01:36,690 --> 00:01:42,690
You have a subdirectory with dogs and a subdirectory with cuts and keras and PyTorch can load directly

31
00:01:42,690 --> 00:01:48,050
from that, that memory structure as well, which I'll probably demonstrate that in a little less than

32
00:01:48,100 --> 00:01:48,600
actually.

33
00:01:49,410 --> 00:01:52,500
So now we can print that part, you can see Dog in the title.

34
00:01:53,520 --> 00:01:58,830
No, it's created transformations, which you're accustomed doing that would pay to it, though this

35
00:01:58,830 --> 00:02:00,540
is a part I want to discuss with you.

36
00:02:01,080 --> 00:02:03,720
This part could seem a bit tricky to beginners.

37
00:02:04,140 --> 00:02:11,940
So what's happening here is we're creating a class basically as a dataset class that Pikachu uses directly.

38
00:02:12,360 --> 00:02:17,550
It's a function that we use in Python, actually, it's a python type style of coding.

39
00:02:20,480 --> 00:02:25,070
This class that we're creating here is actually quite simple, it might seem a bit confusing to people

40
00:02:25,070 --> 00:02:29,330
who are new to Python or actually even if you're not new to Python.

41
00:02:29,330 --> 00:02:34,850
I remember I was coding Python for two years and I wasn't accustomed to building classes like this type

42
00:02:34,850 --> 00:02:35,620
of structure here.

43
00:02:35,630 --> 00:02:42,020
But what this does, it allows us to basically point to what the dataset that we establish here.

44
00:02:42,020 --> 00:02:47,120
So this is how we get a data set of pots right here and the transformation path and allows us to get

45
00:02:47,120 --> 00:02:48,470
the image label out of it.

46
00:02:48,530 --> 00:02:55,370
So if this dog in the image per title like we saw here, this dog in the title, it returns a one if

47
00:02:55,370 --> 00:02:56,450
not written of zero.

48
00:02:57,050 --> 00:03:00,680
And if self transform is known, it will do this.

49
00:03:00,680 --> 00:03:02,150
If not, it wouldn't.

50
00:03:02,600 --> 00:03:05,960
And it returns, the image transformed and the label here.

51
00:03:06,350 --> 00:03:07,580
So it's quite a simple class.

52
00:03:07,580 --> 00:03:08,540
It's more for class.

53
00:03:08,600 --> 00:03:13,500
It's a multiple object that does allows us to access the image, and it's label quite easily.

54
00:03:13,520 --> 00:03:16,580
So PyTorch can utilize this class directly.

55
00:03:17,030 --> 00:03:19,340
So we create that dataset here.

56
00:03:19,340 --> 00:03:22,340
So we create a training class creating object.

57
00:03:22,460 --> 00:03:29,360
I should say I know about this, an object here, which is outlined there, and it can show you an example

58
00:03:29,360 --> 00:03:29,660
of it.

59
00:03:29,810 --> 00:03:30,440
How it works.

60
00:03:30,950 --> 00:03:33,680
So you can use Trin get item zero.

61
00:03:34,580 --> 00:03:36,110
And what is this place here?

62
00:03:36,120 --> 00:03:41,280
This is the image tensor after the transom has been applied, so you can see how it works.

63
00:03:41,300 --> 00:03:47,540
You can immediately see this is a way we can access it and access to data and as well as label here.

64
00:03:48,830 --> 00:03:52,490
So you can print this out and you can actually see get the shape of that.

65
00:03:52,490 --> 00:03:53,570
That was tenses.

66
00:03:53,580 --> 00:03:58,820
You can see its true dimensions, which treatment, but emotional depth, which means is a color image

67
00:03:59,240 --> 00:04:01,160
60 by 60 is the size.

68
00:04:01,160 --> 00:04:02,390
So that's quite cool.

69
00:04:03,050 --> 00:04:04,430
So now what we'll do.

70
00:04:05,150 --> 00:04:09,500
We're going to use our dataset to create a random split here.

71
00:04:13,860 --> 00:04:19,080
So the reason we create a random split here is because during the training process, we're not going

72
00:04:19,080 --> 00:04:24,660
to test our model on the test to separate only going to use it training data set that we will split

73
00:04:24,720 --> 00:04:27,090
the 20000 and 5000 images.

74
00:04:27,720 --> 00:04:34,500
And this allows us to basically remember when we're training models, it's good to have a test dataset

75
00:04:34,500 --> 00:04:40,230
the outside of the of the training process, because we're going to be passing the training validation

76
00:04:40,560 --> 00:04:44,280
is validation is going to be what we evaluate on during training and the training.

77
00:04:44,280 --> 00:04:45,990
That is what the model is going to be trained on.

78
00:04:46,560 --> 00:04:50,550
And then the final test of the said is what the model is evaluated on afterward.

79
00:04:50,640 --> 00:04:53,010
That's the the benchmark.

80
00:04:53,010 --> 00:04:57,780
That's the that's it's the whole Kaggle does a lot of these competitions where they give you a test

81
00:04:57,780 --> 00:05:01,170
dataset that you only get to perform on after you train your model.

82
00:05:01,170 --> 00:05:02,310
So after your training model.

83
00:05:02,760 --> 00:05:09,870
So basically, when training your model like that, you should split that 25000 into a subset of 20

84
00:05:09,870 --> 00:05:12,060
and five and this is trained out here.

85
00:05:12,630 --> 00:05:18,150
So we just use this to utilities function that does that for you automatically by to which has a number

86
00:05:18,150 --> 00:05:19,650
of convenient functions that don't.

87
00:05:19,770 --> 00:05:23,700
It's impossible to get a full 12 pie touch and teach you like that.

88
00:05:24,150 --> 00:05:28,410
There are courses and books that try to avoid doing the device you use them.

89
00:05:28,920 --> 00:05:32,250
My advice for you is, I mean, I don't advise you, don't use them.

90
00:05:32,280 --> 00:05:37,800
What I'm seeing is my advice to you to get the hang of PyTorch is to experiment on your own.

91
00:05:37,800 --> 00:05:43,380
Basically, just looking at a lot of example codes of how other people have used PyTorch and then trying

92
00:05:43,380 --> 00:05:47,010
to implement them on their own, trying to understand them and then applying them.

93
00:05:47,880 --> 00:05:51,720
It's also good to look at the documentation, don't like to see what it's capable of, but there are

94
00:05:51,720 --> 00:05:57,840
hundreds of functions and pie torching cars, so it's not easy to grasp all of them by just reading

95
00:05:57,870 --> 00:05:58,410
at reading them.

96
00:05:58,920 --> 00:06:03,270
That's a way that you can fool yourself, that you're living in something, but you quickly forget it

97
00:06:03,570 --> 00:06:04,290
the next day.

98
00:06:04,800 --> 00:06:06,360
So there's almost no point to doing that.

99
00:06:07,080 --> 00:06:09,060
So learn by doing it is what I see.

100
00:06:09,900 --> 00:06:12,690
So let's just say just to double check that it's worked.

101
00:06:13,350 --> 00:06:18,800
Now let's create an array of old labels using the get item, and we just get the first the second sorry

102
00:06:18,810 --> 00:06:19,810
item of that array.

103
00:06:21,000 --> 00:06:27,090
So this is our validation set labels, and this takes a little while to run a remember correctly.

104
00:06:27,480 --> 00:06:28,740
And then what we'll do next.

105
00:06:28,740 --> 00:06:30,480
We'll split the plot.

106
00:06:30,490 --> 00:06:35,460
Sorry, the classes here so you can see if you're imbalanced or not amounts at all.

107
00:06:36,090 --> 00:06:40,050
You can see in the test data such class it as we're just five dozen images.

108
00:06:40,050 --> 00:06:42,510
It's 25 25, which is good.

109
00:06:43,020 --> 00:06:44,190
Always good to check that.

110
00:06:45,150 --> 00:06:50,550
So this is a random dose and it's sort of worked well, but assume that's assuming the initial classes

111
00:06:50,550 --> 00:06:51,750
were balanced correctly.

112
00:06:52,530 --> 00:06:56,820
So now we can create our data leaders here, which you've seen before.

113
00:06:57,240 --> 00:07:04,690
So we specified to train and objects which were created above specified, but size set roughly equal.

114
00:07:04,730 --> 00:07:07,820
Shrew, truffle and evolution actually should be false.

115
00:07:07,840 --> 00:07:08,490
Not true.

116
00:07:08,730 --> 00:07:09,960
So let's change that.

117
00:07:11,190 --> 00:07:16,110
And now we can just print out some sample images using towards vision make grid that function, which

118
00:07:16,110 --> 00:07:21,600
you've seen a few times before, and we just used MATLAB to and it showed us in the end.

119
00:07:22,320 --> 00:07:24,810
So we transposed to get a nicer shape grid.

120
00:07:25,320 --> 00:07:26,310
And there we go.

121
00:07:26,730 --> 00:07:28,770
There's all cats and dogs images.

122
00:07:29,220 --> 00:07:31,500
This one has some people in it, some children.

123
00:07:32,490 --> 00:07:35,040
This looks like a cat.

124
00:07:36,060 --> 00:07:36,930
Yeah, it is a cat.

125
00:07:37,830 --> 00:07:38,430
Just checking.

126
00:07:39,000 --> 00:07:42,750
So now we've create a model, and this is a quite simple model that we're creating here.

127
00:07:43,290 --> 00:07:46,740
It's a three by three filter with 16 filters.

128
00:07:46,740 --> 00:07:49,320
This one that's 22 filters, 64 filters.

129
00:07:49,920 --> 00:07:55,470
And then we have to we have 256 units here, 128 units here.

130
00:07:56,110 --> 00:07:58,770
And then another 128 units here.

131
00:07:59,430 --> 00:08:06,480
And then we just compile network like this trick on players to FC layers here and applied itself max

132
00:08:06,480 --> 00:08:07,290
at the end.

133
00:08:07,440 --> 00:08:09,540
So let's create or high touch model.

134
00:08:10,740 --> 00:08:16,150
And no, let's display a summary of our model with the input size specified 366.

135
00:08:16,150 --> 00:08:18,540
So that's the image size and the image depth.

136
00:08:23,850 --> 00:08:24,520
So there we go.

137
00:08:24,540 --> 00:08:33,360
So we see we have relatively small model, four hundred and sixty thousand premises, so it's not that

138
00:08:33,360 --> 00:08:33,570
big.

139
00:08:34,410 --> 00:08:41,490
Next, let's take a look at defining our last criterion and optimizing and setting out epochs to 10.

140
00:08:42,480 --> 00:08:43,830
And what we're going to do here.

141
00:08:44,640 --> 00:08:48,600
This is just going to type to make sure it's a data loader, and it is just to be sure.

142
00:08:49,110 --> 00:08:54,690
And what they're going to, what I'm going to do here is we're going to use t2dm, sorry to basically

143
00:08:54,690 --> 00:08:57,690
output a progress bar during our training, which is quite nice.

144
00:08:58,080 --> 00:09:04,690
It helps visualize this training, and it's not a career style way of visualizing the training process.

145
00:09:04,710 --> 00:09:07,230
This is the same training loop you've seen before.

146
00:09:07,740 --> 00:09:14,220
We keep track of things here so we can display them on the output of after after it's finished its epoch

147
00:09:15,600 --> 00:09:18,720
and then we just print sorry after the finish this epoch here.

148
00:09:19,140 --> 00:09:21,240
This is just the training accuracy.

149
00:09:21,780 --> 00:09:25,920
So let's run this, and let's take a look at the nice outputs we get from this.

150
00:09:25,950 --> 00:09:31,000
You can see we get a nice progress bar using TQM here.

151
00:09:31,020 --> 00:09:35,700
All you have to do is just use Tilikum with tell him, I'm sorry, and then you just put in what you

152
00:09:35,700 --> 00:09:39,510
want as as the epochs we call a T epoch.

153
00:09:39,990 --> 00:09:44,070
And we just set the description here and we just set this criteria to display.

154
00:09:44,670 --> 00:09:46,920
And it's usually it's quite simple to use it.

155
00:09:46,950 --> 00:09:52,350
I wouldn't go into detail what it's here because it could distract you from the course because this

156
00:09:52,350 --> 00:09:54,960
is just a nice way to display our training.

157
00:09:54,960 --> 00:09:58,950
So I'll leave this to trim and I'll come back to it afterward.

158
00:09:59,220 --> 00:09:59,670
Thank you.

159
00:10:04,900 --> 00:10:10,090
So while one that were twins, I actually can go through the rest of the book Gratuitous Kendra at life,

160
00:10:10,090 --> 00:10:13,120
so it's not going to be a big deal.

161
00:10:13,240 --> 00:10:17,460
It's just not going to be running the court life because this has been this is running here.

162
00:10:17,470 --> 00:10:23,290
So this we still have to wait for nine more books to train all nine and a half, actually eight million

163
00:10:23,290 --> 00:10:24,160
bucks to train now.

164
00:10:24,610 --> 00:10:29,560
But let's take a look at this so you can see it's printing the training accuracy here at the end, when

165
00:10:29,560 --> 00:10:32,260
it's finished to go into the academic training dataset.

166
00:10:32,740 --> 00:10:36,610
So you can see we're at 58 percent accuracy, which is quite good for the first epoch.

167
00:10:37,060 --> 00:10:40,130
And validation is even better at it's 68 percent accuracy.

168
00:10:40,150 --> 00:10:47,770
So actually, let's wait for the second epoch to display our validation, accuracy and validation loss.

169
00:10:48,430 --> 00:10:55,720
So it should be better because the law scores better here and we can see, yeah, we're getting 73 percent

170
00:10:55,720 --> 00:10:56,360
accuracy.

171
00:10:56,410 --> 00:11:01,450
So for our fairly simple model, we're getting pretty decent accuracy right now.

172
00:11:01,460 --> 00:11:07,000
I mean, I know it's just two classes, so it's basically what you can do with them to 50 50 on this.

173
00:11:07,570 --> 00:11:11,860
But after just such two epochs with such a simple model, that's quite good.

174
00:11:12,910 --> 00:11:18,280
So what I'm going to show you here, the train loss parameter will train or some parameter.

175
00:11:18,820 --> 00:11:23,380
The train loads variable we're using to keep track of triangles as it progresses.

176
00:11:23,980 --> 00:11:28,870
You can actually use that to visualize it at the end here, which is what we've done before.

177
00:11:29,560 --> 00:11:30,730
Rob's actually tried to run.

178
00:11:30,730 --> 00:11:36,280
That could, but anyway, the output is here and you can see this is this here is displaying the trading

179
00:11:36,280 --> 00:11:41,710
accuracy, which is under this blue line I believe is trading after.

180
00:11:41,710 --> 00:11:44,640
See, this is the test set.

181
00:11:44,890 --> 00:11:45,850
Sorry, this is the test.

182
00:11:45,880 --> 00:11:47,080
No, this is the training accuracy.

183
00:11:47,080 --> 00:11:48,370
This is the test accuracy.

184
00:11:48,940 --> 00:11:50,410
This is the training loss.

185
00:11:50,410 --> 00:11:53,350
This is the test loss and you can plot those values.

186
00:11:53,350 --> 00:11:57,190
That would have been very useful if I put a legend in here, so I wouldn't have been guessing.

187
00:11:58,060 --> 00:12:01,630
So and then we save on model here can save it and give it any name you want.

188
00:12:02,410 --> 00:12:07,850
And what we're going to do here, we're going to just plot the results of our model here.

189
00:12:07,870 --> 00:12:13,180
So this is a primitive way of doing it and don't actually do it this way because you can tell what to

190
00:12:13,180 --> 00:12:16,570
predict with what prediction is to his image.

191
00:12:17,350 --> 00:12:18,970
So there's a better way to do that.

192
00:12:18,970 --> 00:12:20,440
So what do we do?

193
00:12:20,890 --> 00:12:24,670
We just use our model again with a little model that we've saved before.

194
00:12:24,670 --> 00:12:30,310
It's always a good habit to just load your saved model just to make sure you can reload it in the future,

195
00:12:30,310 --> 00:12:33,720
because sometimes things do go wrong when saving models.

196
00:12:33,730 --> 00:12:38,410
Not that it would happen randomly, but it happens when you define models and you try to load it back.

197
00:12:38,410 --> 00:12:43,060
It doesn't always load correctly because there was something missing in the class definition, so it's

198
00:12:43,060 --> 00:12:44,530
always good just to make sure it works.

199
00:12:45,730 --> 00:12:50,350
So what we're going to do here, we're going to get some random samples were not random samples, but

200
00:12:50,350 --> 00:12:56,320
samples from the validation dataset using the DOT next iterate to pass them to the GPU.

201
00:12:56,320 --> 00:12:57,670
If it's if you're using AGP, you.

202
00:12:58,270 --> 00:13:04,210
And then what we're going to do here, we're going to get the probability outputs here from the model

203
00:13:04,660 --> 00:13:06,220
and change this.

204
00:13:06,550 --> 00:13:07,750
It is variable.

205
00:13:08,140 --> 00:13:09,340
Is a dictionary here.

206
00:13:09,340 --> 00:13:09,700
Sorry?

207
00:13:10,090 --> 00:13:13,460
So we're just going to use that to put a title in the image here.

208
00:13:13,480 --> 00:13:20,530
So once we took then this allows us to plot this nice plot here where you can see the predictions over

209
00:13:20,530 --> 00:13:22,150
the images, which I quite like.

210
00:13:22,330 --> 00:13:26,920
So you can see that the catalog dog that got this one wrong, look at it.

211
00:13:26,980 --> 00:13:28,780
It says it's a dog, but it's a cat.

212
00:13:29,440 --> 00:13:31,600
This is also a cat, but it thinks it's a dog.

213
00:13:31,630 --> 00:13:32,920
So you can see it's doing.

214
00:13:33,700 --> 00:13:35,860
It's getting about three quarters of these images, right?

215
00:13:36,010 --> 00:13:38,240
So that's pretty decent.

216
00:13:38,320 --> 00:13:43,820
So now you have so you basically seen here how you can set up PyTorch to be trained to create a Python

217
00:13:43,820 --> 00:13:46,450
Watch CNN model on your own dataset.

218
00:13:46,930 --> 00:13:54,370
So later on, we'll take a look at how we do that using more advanced training methodologies such as

219
00:13:54,370 --> 00:13:55,810
fine tuning and translating.

220
00:13:56,200 --> 00:13:57,880
But for now, we'll stop there.

221
00:13:58,480 --> 00:14:00,300
Your model should still be training right now.

222
00:14:00,310 --> 00:14:01,900
Let's take a look and see what it's getting.

223
00:14:02,440 --> 00:14:06,370
It is getting 77 percent accuracy after 40 books.

224
00:14:06,820 --> 00:14:08,620
I believe this model.

225
00:14:08,680 --> 00:14:09,760
Let's check out the graph.

226
00:14:09,760 --> 00:14:17,140
Actually, on the test accuracy got about ninety two percent sort of just under ninety two percent accuracy,

227
00:14:17,710 --> 00:14:19,000
which is quite good, actually.

228
00:14:19,030 --> 00:14:22,660
Let's take a look at what our training accuracy is at this point seventy seven.

229
00:14:23,350 --> 00:14:27,730
So it's actually quite similar to the test accuracy, the validation accuracy, I should say.

230
00:14:28,330 --> 00:14:29,440
So that's pretty good.

231
00:14:29,470 --> 00:14:31,390
Overall, decent results.

232
00:14:31,390 --> 00:14:32,050
We're getting there.

233
00:14:32,620 --> 00:14:34,210
So I'll stop there for now.

234
00:14:34,210 --> 00:14:41,260
And in the next section, we'll take a look at how we do this with Keros and with the Keros module here.

235
00:14:41,830 --> 00:14:45,670
I'll also be teaching you Kovac's how to implement them in class.

236
00:14:46,090 --> 00:14:47,740
We can do those things in PyTorch.

237
00:14:47,740 --> 00:14:48,730
However we'll be using.

238
00:14:49,060 --> 00:14:54,130
We'll be using the PyTorch Lightning Library to implement these features here.

239
00:14:54,340 --> 00:14:56,470
So stay tuned for those lessons.

240
00:14:56,500 --> 00:14:56,950
Thank you.