1
00:00:00,150 --> 00:00:05,490
Hello, everyone, and welcome to this new session in which we are going to go under the hood and understand

2
00:00:05,490 --> 00:00:10,410
exactly what goes on when we call on this fit method right here.

3
00:00:10,440 --> 00:00:15,430
In fact, we're going to be training our own model without making use of this feed method.

4
00:00:15,450 --> 00:00:20,010
So yeah, we're going to have this custom training loop, which we are going to build.

5
00:00:20,490 --> 00:00:26,370
As you could see here, we have this training block, we have this validation block, and then we have

6
00:00:26,370 --> 00:00:32,190
this method which we named neural learn, and this method which replaces the feed method, which we

7
00:00:32,190 --> 00:00:38,040
used to get in TensorFlow is not only a great machine learning library because it permits us build and

8
00:00:38,040 --> 00:00:39,770
deploy machine learning models.

9
00:00:39,780 --> 00:00:46,470
It's greatness also comes from the fact that it permits us build these models and train them without

10
00:00:46,470 --> 00:00:50,200
getting to know everything which is going on under the hood.

11
00:00:50,220 --> 00:00:59,130
So all you need to do to train TensorFlow model is to specify that model and then make use of this feed

12
00:00:59,130 --> 00:01:06,290
method while passing in your dataset, which is made up of the inputs and the output.

13
00:01:06,300 --> 00:01:11,570
That said, in this section, we'll see how to create our own feed method.

14
00:01:11,580 --> 00:01:17,700
That is, we'll go a step further in understanding how TensorFlow works under the hood.

15
00:01:17,730 --> 00:01:25,890
Recall that when you make use of this feed method right here, what goes on essentially is TensorFlow

16
00:01:26,040 --> 00:01:31,440
applies the gradient descent method to update this weight tester right here.

17
00:01:31,440 --> 00:01:37,560
So all the setters are being updated until the model converges.

18
00:01:38,010 --> 00:01:45,390
Recall that for given inputs x1x2 and outputs y one, y2y3.

19
00:01:45,420 --> 00:01:54,570
Our aim is to update this weights such that when we pass in this inputs into the model, we get an output

20
00:01:54,570 --> 00:02:00,300
right here, which looks practically the same as this outputs.

21
00:02:00,660 --> 00:02:10,890
And to know whether this model outputs are the same or similar to this actual outputs, we apply or

22
00:02:10,890 --> 00:02:17,190
make use of a loss function, which computes the difference between these two and generally.

23
00:02:17,190 --> 00:02:22,200
Initially, when you start training the model, that's when you start updating these parameters.

24
00:02:22,200 --> 00:02:24,540
There will be a great difference between these two.

25
00:02:24,540 --> 00:02:30,150
But as you keep on updating these parameters, we notice that the loss starts dropping.

26
00:02:30,360 --> 00:02:37,020
And so once this loss drops, until it starts, it converges, we then stop the training.

27
00:02:37,380 --> 00:02:47,730
But recall also that the way we update these weights is such that we take the initial weight minus the

28
00:02:47,730 --> 00:02:53,790
learning rates, which is a constant with the fine times, the partial derivative of this loss.

29
00:02:53,820 --> 00:03:00,180
Now, the loss which we've seen here, which we understand that is the difference between the model's

30
00:03:00,180 --> 00:03:02,850
predictions and the actual prediction.

31
00:03:02,850 --> 00:03:09,390
So here we have this partial derivative of the loss with respect to the parameter or the weight in question.

32
00:03:09,810 --> 00:03:18,840
And so if we want to update, say, the stellar double prime to one, we'll simply use this same approach.

33
00:03:19,290 --> 00:03:27,750
Now, the way TensorFlow computes this partial derivative right here is by automatic differentiation.

34
00:03:27,930 --> 00:03:36,810
And so when you have this model with a inputs, as these inputs are being passed into the model in this

35
00:03:36,810 --> 00:03:46,260
forward pass right up to the output TensorFlow records, this gradients others partial derivatives,

36
00:03:46,680 --> 00:03:53,100
and this is done for each and every parameter such that once we get the output and we compute the loss,

37
00:03:53,100 --> 00:03:59,730
we could now get this whole partial derivative right here that is a partial activity of the loss with

38
00:03:59,730 --> 00:04:09,300
respect to each end every weight and then each and every weight has been optimized or been updated via

39
00:04:09,300 --> 00:04:10,160
this formula.

40
00:04:10,170 --> 00:04:13,950
So in general we can have tester, whatever.

41
00:04:13,950 --> 00:04:26,490
And then we have, let's say, IG, we could have I comma, J, equal tetter, I comma, g minus learning

42
00:04:26,490 --> 00:04:33,840
rate times, partial derivative of the loss respect to theta IJ And so as we go to this forward pass,

43
00:04:33,990 --> 00:04:41,490
what TensorFlow does is it keeps in mind the taters and the gradients.

44
00:04:41,490 --> 00:04:43,870
Let's call this gradient the tatter.

45
00:04:43,890 --> 00:04:49,020
So we have the tetter and the dictator which have been stored or which have been recorded.

46
00:04:49,020 --> 00:04:59,040
As we go through this model that's in this forward past that process of record in this gradients here.

47
00:04:59,430 --> 00:05:07,050
As we pass in our inputs in the model is similar to what we would have with a tape recorder where you're

48
00:05:07,050 --> 00:05:13,200
you just need to speak in this mic, The information is recorded and then it could be replayed later

49
00:05:13,200 --> 00:05:13,710
on.

50
00:05:13,920 --> 00:05:21,420
And so here, as information has been passed in this model, this gradients are being stored and they

51
00:05:21,420 --> 00:05:26,310
could be used in the gradient descent algorithm later on.

52
00:05:26,520 --> 00:05:34,080
Getting back to the code, we'll now see how to build a feed method from scratch.

53
00:05:34,290 --> 00:05:40,020
So here we have this let's have for epoch in epochs.

54
00:05:40,020 --> 00:05:45,240
You see that now, unlike your where we just needed to pass in this number of epochs and TensorFlow

55
00:05:45,240 --> 00:05:50,940
does the job, you really need to do much work on your own.

56
00:05:51,720 --> 00:05:54,030
So we have for epoch in epochs.

57
00:05:54,030 --> 00:05:59,580
What we're going to do here is go through each and every batch of our data set.

58
00:05:59,580 --> 00:06:02,250
So we also have that for each epoch.

59
00:06:02,250 --> 00:06:04,530
We are now going to go through each and every batch.

60
00:06:04,530 --> 00:06:15,180
So for X batch Y batch entering data set, let's have this tuple.

61
00:06:15,180 --> 00:06:18,410
So we have X batch y batch in training data set.

62
00:06:18,420 --> 00:06:24,750
What we're going to do now is pass in this X batch into our model.

63
00:06:25,170 --> 00:06:33,000
And then after getting the model output, compare that model output with the actual output and obtain

64
00:06:33,000 --> 00:06:33,810
the loss.

65
00:06:34,830 --> 00:06:38,070
So here we have widespread, which is what the model outputs.

66
00:06:38,070 --> 00:06:39,480
We have the model.

67
00:06:39,990 --> 00:06:41,580
Yeah, we've defined the model already.

68
00:06:41,580 --> 00:06:47,250
So we're working with a sequential model we had built previously.

69
00:06:47,250 --> 00:06:53,880
Yeah, we go, we have this sequential model and then in this model we're going to pass our X batch.

70
00:06:54,420 --> 00:06:55,350
There we go.

71
00:06:55,350 --> 00:06:58,470
We specify that we're training, so we're in training mode.

72
00:06:58,470 --> 00:06:59,850
Training equal true.

73
00:06:59,850 --> 00:07:05,900
Okay, So now we have y Pred and what we could do next is compute the loss.

74
00:07:05,910 --> 00:07:13,290
So here we have loss, equal loss function, which we're going to define loss function, which is going

75
00:07:13,290 --> 00:07:22,800
to take in our widespread and the models are and the actual y so we can make use of this custom BCI

76
00:07:22,830 --> 00:07:23,640
right here.

77
00:07:23,670 --> 00:07:30,090
Now this takes in the y true y pred So we should ensure that we have y true before widespread.

78
00:07:30,090 --> 00:07:35,030
And yeah, we have we should have y batch and then widespread.

79
00:07:35,040 --> 00:07:38,310
Okay, so let's take this off and that's it.

80
00:07:38,310 --> 00:07:41,220
So yeah, we're going to use our custom BCI.

81
00:07:41,460 --> 00:07:45,570
We could take this off and then specify custom BCI.

82
00:07:46,050 --> 00:07:47,850
Now we have our custom BCI.

83
00:07:47,880 --> 00:07:55,740
We could go ahead and update this model's weights, but in order to update our model's weights, we

84
00:07:55,740 --> 00:07:58,260
need those gradients.

85
00:07:58,770 --> 00:08:05,880
The way we get those gradients is by recording or taping this gradients as we pass the input into the

86
00:08:05,880 --> 00:08:11,850
model to do this, this part of our code has to be put in a particular scope.

87
00:08:11,850 --> 00:08:22,900
So yeah, we have width gradients or TF that gradient gradient tape as tape, we are going to have this,

88
00:08:22,920 --> 00:08:23,910
let's have that.

89
00:08:23,910 --> 00:08:28,270
We are going to have this two lines of code.

90
00:08:28,290 --> 00:08:35,700
Now the effect of doing this is we are recording the gradients in this tape, in this recorder.

91
00:08:35,700 --> 00:08:43,200
We could say this as recorder and now all the intermediate values of tetter and intermediate gradients

92
00:08:43,200 --> 00:08:49,560
have been recorded and now could be used in computing this partial derivative right here.

93
00:08:50,310 --> 00:08:53,670
That said, we could now obtain the partial derivatives.

94
00:08:53,670 --> 00:08:56,970
So we have partial derivatives equal to tape.

95
00:08:56,970 --> 00:09:03,210
So this tape all rather equal to recorder because we changed that to recorder so equal this recorder

96
00:09:03,210 --> 00:09:11,850
dot gradient of the loss with respect to all the models trainable weights.

97
00:09:11,850 --> 00:09:19,590
And so this line of code here represents this operation that is computing this partial derivative of

98
00:09:19,590 --> 00:09:22,770
the loss with respect to each and every weight.

99
00:09:23,640 --> 00:09:32,280
Then we move on to optimize our model by going through this stochastic gradient descent or some other

100
00:09:32,280 --> 00:09:37,920
gradient descent based optimizer like the Atom Optimizer, which we've used so far.

101
00:09:38,220 --> 00:09:44,250
That said, we get back up to this compiler, we have this optimizer, let's take this from your copy

102
00:09:44,250 --> 00:09:44,820
that.

103
00:09:44,820 --> 00:09:50,910
And then what we do is we are going to add a so let's take this up.

104
00:09:50,910 --> 00:09:56,430
We have our optimizer here, so let's have this optimizer.

105
00:09:56,430 --> 00:09:57,390
That's it.

106
00:09:57,390 --> 00:09:59,190
Our item optimizer defined.

107
00:09:59,300 --> 00:10:00,410
Were on that cell.

108
00:10:01,130 --> 00:10:06,090
And then what we do now is we have Optimizer.

109
00:10:06,100 --> 00:10:09,410
Optimizer that apply gradients.

110
00:10:09,410 --> 00:10:17,780
So we use this apply gradient method in order to update the weights based on the gradient descent algorithm.

111
00:10:18,260 --> 00:10:24,920
In here we have zip of gradients or rather the partial derivatives.

112
00:10:24,920 --> 00:10:30,950
So we are passing in the partial derivatives and the model trainable weights.

113
00:10:31,130 --> 00:10:34,160
So recall we have some unchangeable weights actually.

114
00:10:34,160 --> 00:10:37,070
So we have this model trainable weights right here.

115
00:10:37,070 --> 00:10:38,300
And the derivatives.

116
00:10:38,510 --> 00:10:40,250
Let's scroll.

117
00:10:40,250 --> 00:10:43,340
Okay, let's have this here.

118
00:10:45,630 --> 00:10:46,560
This.

119
00:10:46,980 --> 00:10:48,150
This is linked.

120
00:10:48,150 --> 00:10:49,650
So this part.

121
00:10:50,540 --> 00:10:52,510
It's linked to this.

122
00:10:52,520 --> 00:11:01,070
And then for this, it's the whole algorithm, the whole gradient descent based algorithm, which takes

123
00:11:01,070 --> 00:11:07,010
in this partial derivative and the model trainable weights does the taters.

124
00:11:08,280 --> 00:11:11,970
Obviously, while defining the optimizer, we have already set the learning rate.

125
00:11:11,970 --> 00:11:13,410
So there we go.

126
00:11:13,830 --> 00:11:16,910
Then from here, we could print out our lost values.

127
00:11:16,920 --> 00:11:18,990
Yeah, we could print out the loss.

128
00:11:19,920 --> 00:11:22,300
Print off the loss.

129
00:11:22,320 --> 00:11:27,240
Okay, So now we said we could now run this and see how this works.

130
00:11:27,240 --> 00:11:30,660
Let's not forget to specify the number of epochs.

131
00:11:30,660 --> 00:11:32,550
So let's have a number of epochs equal.

132
00:11:32,610 --> 00:11:34,020
Say three for stat.

133
00:11:34,410 --> 00:11:39,210
Let's have this epochs and then we run the cell.

134
00:11:40,740 --> 00:11:43,110
We are taught here that the model is not defined.

135
00:11:43,110 --> 00:11:50,160
So let's modify this and have Loonette Loonette model run that again.

136
00:11:50,670 --> 00:11:56,650
We get in this error where we told this expects the shape y, what we get is this.

137
00:11:56,670 --> 00:12:01,230
Now the fact that we have this means that we haven't yet resized our training data, so let's get back

138
00:12:01,230 --> 00:12:05,940
up and then run this cells here for resizing.

139
00:12:06,150 --> 00:12:07,500
And there we go.

140
00:12:07,500 --> 00:12:10,500
Let's run the cells.

141
00:12:10,740 --> 00:12:15,120
Okay, we have that and now it looks fine.

142
00:12:15,720 --> 00:12:17,700
So we have the cells run.

143
00:12:17,700 --> 00:12:21,450
Let's get back to our custom training loop.

144
00:12:22,050 --> 00:12:23,790
Okay, So let's run this.

145
00:12:24,270 --> 00:12:27,210
As we can see, the training goes on smoothly.

146
00:12:27,210 --> 00:12:32,250
You have those last values we should drop in and we run this actually for three epochs.

147
00:12:32,250 --> 00:12:33,180
So there we go.

148
00:12:33,180 --> 00:12:35,670
We have our loss, which is dropping.

149
00:12:35,670 --> 00:12:40,230
Let's and then we could decide to print this out after a given number of steps.

150
00:12:40,230 --> 00:12:45,810
So yeah, we could add the step and enumerate enumerate.

151
00:12:45,840 --> 00:12:46,500
There we go.

152
00:12:46,500 --> 00:12:54,450
We could check out our free python course on neural and AI to understand this in case you new to all

153
00:12:54,450 --> 00:12:55,590
those keywords.

154
00:12:55,590 --> 00:12:56,460
So there we go.

155
00:12:56,460 --> 00:12:58,320
We enumerate, we have step.

156
00:12:58,320 --> 00:13:03,930
And then what we're going to do here is we're going to print our last values only after a number of

157
00:13:03,930 --> 00:13:04,440
steps.

158
00:13:04,440 --> 00:13:10,410
So given that we have 689 steps, that's if our batch size is 32.

159
00:13:10,470 --> 00:13:15,150
Or suppose that after, say, 300 steps, we're going to print this loss out.

160
00:13:15,150 --> 00:13:22,350
So, yeah, we're going to have if the step modulo 300 equals zero, then we'll do the printing.

161
00:13:22,350 --> 00:13:23,160
So there we go.

162
00:13:23,160 --> 00:13:33,000
We have this print and then for every epochs that we're going to print out, training starts for epoch

163
00:13:33,000 --> 00:13:35,070
number and then we format that.

164
00:13:35,070 --> 00:13:36,870
So here we have the epoch.

165
00:13:37,080 --> 00:13:39,450
Okay, let's rerun this again.

166
00:13:40,110 --> 00:13:41,400
Here is what we get now.

167
00:13:41,400 --> 00:13:47,220
So we have this training starts, training starts, training starts, and then we have this intermediate

168
00:13:47,220 --> 00:13:49,100
loss values which have been printed out.

169
00:13:49,110 --> 00:13:51,900
Now what if we include the validation?

170
00:13:51,900 --> 00:14:00,000
So here, notice we finish like we trained the model and then after training for one epoch we don't.

171
00:14:00,000 --> 00:14:01,980
We then get into validation.

172
00:14:01,980 --> 00:14:12,120
So for x batch, val y batch val in val data set, we'll simply copy this out.

173
00:14:12,120 --> 00:14:17,870
So here we have this your let's paste it on your OC.

174
00:14:17,880 --> 00:14:22,370
So here we have the Y pride and our loss.

175
00:14:22,380 --> 00:14:26,430
Val The same model that your training is set to false.

176
00:14:26,430 --> 00:14:29,030
We have training false, we have y print.

177
00:14:29,040 --> 00:14:32,370
Val and then we have y batch.

178
00:14:32,370 --> 00:14:33,960
Val Yes.

179
00:14:33,960 --> 00:14:42,810
X batch of all two and then we print out the last Val So yeah, we have validation loss.

180
00:14:43,260 --> 00:14:45,240
Okay, let's run the cell now.

181
00:14:46,550 --> 00:14:49,640
While training is going on, we could include the metric.

182
00:14:49,790 --> 00:14:55,580
Here we have this metric which is binary accuracy.

183
00:14:56,210 --> 00:14:57,770
Binary accuracy.

184
00:14:59,630 --> 00:15:03,500
Now, while training is going on, we could deal with the metric.

185
00:15:03,500 --> 00:15:08,480
So you're for this or for each batch we're working with.

186
00:15:09,260 --> 00:15:11,860
We are going to update the state.

187
00:15:11,870 --> 00:15:16,280
We are going to print out the result and then we are going to reset the states.

188
00:15:16,520 --> 00:15:22,520
So let's get back to this part and then just after this, that's for this particular batch.

189
00:15:22,520 --> 00:15:32,330
What we have here is we have our metric that update state and then we pass in our why or our why batch.

190
00:15:32,420 --> 00:15:33,710
That's all by batch.

191
00:15:33,710 --> 00:15:34,070
True.

192
00:15:34,070 --> 00:15:36,860
Why and then why Pred?

193
00:15:37,040 --> 00:15:44,900
So once we update this metric, we could now print out this metric value out of this for loop.

194
00:15:44,900 --> 00:15:48,500
So after each epoch we're going to print out the metric value.

195
00:15:49,220 --> 00:15:56,660
And so here we have print of metric dot result.

196
00:15:56,990 --> 00:15:59,240
There we go, print metric results.

197
00:15:59,240 --> 00:16:05,390
And then we should have your the accuracy is there we go.

198
00:16:05,390 --> 00:16:11,330
Accuracy is does it once we once we print out the result, we now go ahead and reset state.

199
00:16:11,330 --> 00:16:16,460
So we have metric that reset states and that should be good.

200
00:16:16,460 --> 00:16:18,860
So we reset the states and that's fine.

201
00:16:18,860 --> 00:16:22,040
We could repeat this exact same process for the validation.

202
00:16:22,040 --> 00:16:28,700
With the validation just in here in this batch, we are going to repeat the same step.

203
00:16:28,700 --> 00:16:35,660
So here we have the metric, space it out here and then yeah, we're going to have metric vol metric

204
00:16:35,660 --> 00:16:37,280
while we update the state.

205
00:16:37,280 --> 00:16:42,980
Yeah, we're going to use why batch file and why pred vao.

206
00:16:43,010 --> 00:16:43,520
Okay.

207
00:16:43,520 --> 00:16:48,170
Once we update the state, we now do this same process here.

208
00:16:48,170 --> 00:16:50,360
So we print and then we reset the states.

209
00:16:50,540 --> 00:16:53,930
Let's have this here print and then reset states.

210
00:16:53,930 --> 00:16:55,250
Let's take that off.

211
00:16:55,610 --> 00:17:02,320
So we have here metric value and then metric vial and that's it.

212
00:17:02,330 --> 00:17:04,610
Let's define metric and metric.

213
00:17:04,610 --> 00:17:05,150
Wow.

214
00:17:05,180 --> 00:17:05,720
Here we go.

215
00:17:05,720 --> 00:17:08,930
We have metric and then we have metric.

216
00:17:08,930 --> 00:17:09,830
Wow.

217
00:17:10,430 --> 00:17:11,270
So that's it.

218
00:17:11,270 --> 00:17:16,610
So here we see how to recreate this feat method that we have here.

219
00:17:16,790 --> 00:17:20,750
Now that we've seen this, we see the training is going out well.

220
00:17:20,750 --> 00:17:25,880
We see that the training lost, the validation loss and that's it.

221
00:17:25,880 --> 00:17:32,060
But the training loss we're getting here is for each and every three hundredths batch.

222
00:17:32,060 --> 00:17:33,920
So let's take this off from here.

223
00:17:33,920 --> 00:17:39,710
Let's let's say we print this out after that's after each epoch.

224
00:17:39,710 --> 00:17:43,850
So the training loss, we print it out after each epoch and that's it.

225
00:17:44,210 --> 00:17:47,750
This means that we don't really need to have this again.

226
00:17:47,750 --> 00:17:48,680
Let's take that off.

227
00:17:48,680 --> 00:17:51,590
So we print out the training loss and the accuracy.

228
00:17:51,590 --> 00:17:52,400
There should be good.

229
00:17:52,400 --> 00:17:57,230
We now run this and here's where we get You see, we have the train loss.

230
00:17:57,230 --> 00:17:59,810
We have the train accuracy, validation, accuracy.

231
00:17:59,810 --> 00:18:02,090
We've changed this to our accuracy.

232
00:18:02,090 --> 00:18:05,660
And then we have the validation loss for Epoch two.

233
00:18:05,690 --> 00:18:07,550
We have the same for Epoch three.

234
00:18:07,550 --> 00:18:09,470
We have this for now.

235
00:18:09,470 --> 00:18:10,940
We're in the ego mode.

236
00:18:10,940 --> 00:18:16,340
So obviously this training process is going to be slower as compared to if we're working in the graph

237
00:18:16,340 --> 00:18:16,930
mode.

238
00:18:16,940 --> 00:18:25,130
Now what we're going to do is we are going to pick out those computation, expensive units of this cell

239
00:18:25,130 --> 00:18:25,950
right here.

240
00:18:25,970 --> 00:18:32,540
Now we know that this is computationally expensive because, yeah, we have to pass in our input into

241
00:18:32,540 --> 00:18:39,170
the model compared to the output, compute the loss, get the gradients, optimize the model of the

242
00:18:39,170 --> 00:18:39,770
the metrics.

243
00:18:39,770 --> 00:18:46,100
So we look at this as one of those blocks and then we have this other block.

244
00:18:46,100 --> 00:18:50,990
Let's scroll up, we have this, let's take this off.

245
00:18:50,990 --> 00:18:55,900
So yeah, we have this block and then we also have this other block.

246
00:18:55,910 --> 00:19:01,160
Now you'll notice that this is what we call the training step, and then this is the validation step.

247
00:19:01,160 --> 00:19:06,230
So what we could simply do here is create this to methods.

248
00:19:06,230 --> 00:19:14,360
So we have training, let's call this training block, we have the training block.

249
00:19:14,360 --> 00:19:22,100
And then what goes on this training block is we take in an X, so you have X batch and then Y batch.

250
00:19:22,430 --> 00:19:24,770
Basically we just copied all this.

251
00:19:24,770 --> 00:19:30,410
So here we copy this code that and then we paste it right here.

252
00:19:30,950 --> 00:19:37,850
So yeah, we have this training block and then your instead of having this, we just say for each step

253
00:19:37,850 --> 00:19:45,670
we're going to go pass to the training block, pass for the training block we pass in X batch and Y.

254
00:19:46,420 --> 00:19:49,100
Then we repeat the same process for the validation.

255
00:19:49,120 --> 00:19:52,240
Here we have this copied out.

256
00:19:52,450 --> 00:19:54,610
And let's copy this.

257
00:19:54,970 --> 00:19:57,820
So yeah, we have the validation block.

258
00:19:57,940 --> 00:19:59,050
That's a file block.

259
00:19:59,080 --> 00:20:06,520
Here we have the block, which takes L batch, val and Y batch Val.

260
00:20:06,730 --> 00:20:07,560
There we go.

261
00:20:07,570 --> 00:20:10,450
We have this string block validation block.

262
00:20:10,540 --> 00:20:13,000
Now, just here we could paste out.

263
00:20:13,000 --> 00:20:18,250
Let's create our validation block and then take X batch.

264
00:20:18,250 --> 00:20:20,100
Val x batch.

265
00:20:20,170 --> 00:20:23,320
Val y batch, Val.

266
00:20:23,770 --> 00:20:24,400
There we go.

267
00:20:24,400 --> 00:20:25,550
We pick this out.

268
00:20:25,570 --> 00:20:28,900
So for this, we have that.

269
00:20:29,650 --> 00:20:37,770
Now, since we compute this loss in the training block, we want to return want to return this loss.

270
00:20:37,780 --> 00:20:39,370
So we return this loss.

271
00:20:40,090 --> 00:20:42,190
Okay, We have the last returned.

272
00:20:42,190 --> 00:20:47,650
And then just right in here, we are going to see loss equals this.

273
00:20:47,650 --> 00:20:48,820
So we call this loss.

274
00:20:48,820 --> 00:20:52,210
And now we could make use of it right here and then we do.

275
00:20:52,210 --> 00:20:53,650
Same for the validation block.

276
00:20:53,650 --> 00:20:58,540
With the validation block, we take this here and then we return our loss.

277
00:20:58,540 --> 00:20:59,290
Val.

278
00:20:59,860 --> 00:21:00,850
Now, that sounds great.

279
00:21:00,850 --> 00:21:06,760
We should now be able to change this in our graph mode, so we have to function.

280
00:21:06,760 --> 00:21:11,500
That's all it takes to convert this to graph mode to F function.

281
00:21:12,400 --> 00:21:13,000
That's it.

282
00:21:13,000 --> 00:21:14,290
So there we go.

283
00:21:14,290 --> 00:21:15,610
We've converted this to graph mode.

284
00:21:15,610 --> 00:21:16,990
We can now run this.

285
00:21:16,990 --> 00:21:18,190
We get in this error.

286
00:21:18,220 --> 00:21:20,140
Let's check on what's going on.

287
00:21:20,500 --> 00:21:22,210
Okay, let's run this now.

288
00:21:22,240 --> 00:21:26,170
We told on and then doesn't match any outer indentation level.

289
00:21:26,470 --> 00:21:26,780
Okay.

290
00:21:26,800 --> 00:21:26,980
Yeah.

291
00:21:26,980 --> 00:21:32,110
We should put this to match up with this width and that should be good.

292
00:21:32,260 --> 00:21:34,600
Let's make sure we don't have the same error here.

293
00:21:34,600 --> 00:21:37,120
So we have this four and that's it.

294
00:21:37,120 --> 00:21:38,890
Okay, so let's run again.

295
00:21:38,890 --> 00:21:42,160
And obviously we should have no problem now.

296
00:21:42,160 --> 00:21:49,120
We could retrain our model and we should get faster training this time around, but still we get this

297
00:21:49,120 --> 00:21:52,240
error loss value must be defined before the loop.

298
00:21:52,240 --> 00:21:53,560
So we get back to the code.

299
00:21:53,560 --> 00:21:59,080
And what we could do is command this section, which is for the training, because this works already.

300
00:21:59,080 --> 00:22:02,860
So let's command the section and then focus on the validation block.

301
00:22:02,860 --> 00:22:10,960
The first remark we have here is we've made an error of putting this for loop in this block.

302
00:22:10,960 --> 00:22:14,050
It's not actually a syntax error, but more of a design error.

303
00:22:14,050 --> 00:22:19,300
So it's better for us to have this var block in this far loop instead.

304
00:22:19,300 --> 00:22:26,410
So that said, we are going to cut out this from this var block, and then what we'll have here now

305
00:22:26,410 --> 00:22:28,060
is for this.

306
00:22:28,060 --> 00:22:31,780
So we have for x y in the data set.

307
00:22:31,780 --> 00:22:35,620
You see we're going to pass this in here and then have this block.

308
00:22:35,620 --> 00:22:37,030
So here we have this var block.

309
00:22:37,030 --> 00:22:39,100
Now everything looks fine.

310
00:22:39,430 --> 00:22:40,240
We should have that.

311
00:22:40,240 --> 00:22:42,550
And then let's now run the cell again.

312
00:22:42,550 --> 00:22:47,830
You now see how just correcting that error automatically solved that issue.

313
00:22:47,830 --> 00:22:53,680
And then we could go ahead and uncomment the section and rerun our training.

314
00:22:53,680 --> 00:22:57,640
While the training is going on, we are going to create this method.

315
00:22:57,640 --> 00:23:01,120
So this method, basically we're going to copy from this.

316
00:23:01,120 --> 00:23:09,340
So here we should add the cell, we add the cell, and then we'll define this method, which we'll call

317
00:23:09,340 --> 00:23:12,460
train, or let's call it neural learn.

318
00:23:12,460 --> 00:23:17,980
So we'll call this method neural learn, and our neural and method we are going to take in a training

319
00:23:17,980 --> 00:23:18,670
data set.

320
00:23:18,670 --> 00:23:23,440
So we have trained data set and then our validation data set, we could also get in the model.

321
00:23:23,440 --> 00:23:27,790
So we have the model we could pass in the number of epochs.

322
00:23:28,060 --> 00:23:28,900
Let's have that here.

323
00:23:28,900 --> 00:23:32,230
So it looks like the fit, the model fit, which comes for TensorFlow.

324
00:23:32,230 --> 00:23:39,310
So here we have the number of epochs and then after the model we have the loss function loss function

325
00:23:39,790 --> 00:23:41,710
metric, Val metric.

326
00:23:41,710 --> 00:23:42,880
And that seems okay.

327
00:23:42,880 --> 00:23:44,230
So yeah, we have that.

328
00:23:45,190 --> 00:23:47,020
We didn't have this call.

329
00:23:47,020 --> 00:23:51,050
So yeah, we just have neural learn and then we pass the model.

330
00:23:51,070 --> 00:24:00,580
In that model, we pass the last function loss function, we pass the metric, we pass the vital metric

331
00:24:01,570 --> 00:24:09,280
metric, we pass the train data set, we pass the data set, and then we pass a number of epochs.

332
00:24:09,280 --> 00:24:11,620
So all we need to do now is just run this.

333
00:24:11,620 --> 00:24:15,040
And then our training process is going to start getting back to your training.

334
00:24:15,040 --> 00:24:15,280
Here.

335
00:24:15,280 --> 00:24:17,530
You see how this training loss.

336
00:24:17,530 --> 00:24:22,480
You see we have the training loss, we have the accuracy, we have the validation loss and we have the

337
00:24:22,480 --> 00:24:24,010
validation accuracy.

338
00:24:25,600 --> 00:24:26,830
Let's stop this.

339
00:24:26,830 --> 00:24:28,960
And then we could even take off the cell.

340
00:24:29,590 --> 00:24:31,180
Let's take off the cell.

341
00:24:31,280 --> 00:24:31,930
Okay.

342
00:24:32,200 --> 00:24:35,100
And then we have that cell off.

343
00:24:35,110 --> 00:24:38,080
Let's delete that cell and then focus on this.

344
00:24:38,080 --> 00:24:41,220
So now we have our neural learning, which takes in the model.

345
00:24:41,230 --> 00:24:46,270
So here, as you could see, we had this optimizer, We had the metric metric.

346
00:24:46,270 --> 00:24:47,760
Val epochs.

347
00:24:47,770 --> 00:24:49,620
Let's get back to the optimizer.

348
00:24:49,630 --> 00:24:51,010
Let's add the optimizer here.

349
00:24:51,010 --> 00:24:59,010
So we should have after this last function metric and then let's have Optimizer Optimizer.

350
00:24:59,020 --> 00:24:59,930
There we go.

351
00:24:59,950 --> 00:25:08,820
Now, after the metric here we have our optimizer looks fine, let's run this and this last function

352
00:25:08,830 --> 00:25:09,460
are defined.

353
00:25:09,460 --> 00:25:12,790
So let's scroll up and we have this custom.

354
00:25:13,390 --> 00:25:16,570
So let's just put out the custom BCI right there.

355
00:25:16,570 --> 00:25:24,940
Custom custom BCI, we run that training is now complete and you are the results we get you now know

356
00:25:24,940 --> 00:25:30,420
how to create your own custom training loops and make them around in graph mode.

357
00:25:30,430 --> 00:25:33,430
Thank you for getting around to this point and see you next time.