Hello, everyone, and welcome to the session in which we are going to implement the code for Resonant
three four in TensorFlow two.
Now yours is a resonant three four model right here we have all the variants like the 5101 and 152.
After going through this section, you'll be able to implement this other variants right here.
And also you'll be able to get results like this where we can see clearly an improvement in the accuracy
of our model.
We are going to construct our resonant 34 model while making use of models of class.
And so you could check on the previous sections.
So you better understand how models obsolescence is implemented in TensorFlow too.
Now that said, here we have this resonant 34 model right here, and then the first layer we have is
our convolutional layer, which has seven by seven filters and they are 64 in number.
Also, we know that the Strider number of strides is equal to.
So you could get all this from the paper.
We have the information from the paper seven by 764 Stride two and then followed by three by three max
pull with a stride of two.
So we get back here.
We have this three by three stride of two.
Now from this we then get into this residual blocks, so we'll get back to the paper.
You see, we have two re of this residual blocks.
Now this is actually this because this is a34 layer we're implementing right here, meaning that if
you are implementing a 50 layer resonate, then you would have one by one, three by three and one by
one.
But for the 34 version, we have three by three and three by three.
Now this repeated twice, so that's why you will notice here we have this repeated three times and each
one of them is our residual block.
Now, our residual block here our is the block is this block right here.
So we are going to later on implement implement this residual block here.
Let's get back to the code.
We have simply our residual block.
You see the parameters here, number of filters, 64, 64 and year 64, just exactly as we have it in
the paper.
You see for each block we have a number of filters to be 64.
Now, if we want to get into into this residual block, we could or we'll see exactly how it's implemented
and we'll see that we're going to have this two Conv layers one three by three and another three by
three.
But for now, let's do it this way.
Let's just consider that that has been implemented.
Now, another reason why I want to implement this this way is because now if we want to convert this
to a at 50, all we need to do now is just to update the code for the residual block since, well,
what makes this different here is just this residual blocks right here.
So that said, let's get back again here.
We have now this for resonant blocks.
So we have this four residual blocks actually here you see you have this is four and it's similar.
Like here you have the three by three, three by three.
And then here the number of channels equal 128.
So you notice here that we have 128 and we have four of them now because here we are living from or
we modifying the number of channels we need to take into consideration.
This number of strides we have right here as this permits us to downsample our features.
Now getting back to the code you'll have this year where we have this DOWNSAMPLING and then we move
on.
128 128 128 Again here we have DOWNSAMPLING, now we go to 256 exactly as it's in the paper.
You can look at this directly from here.
Yeah.
We have 256 and we have a certain number of them which are aligned.
We could see that in the summary.
Here's six of them.
Check this out here.
You see we have six of this aligned and then again we have 512, three of them as is in the paper here,
512 and that's it.
Now from here we have the global average pulling to the global average pulling, and then we have this
final fully connected layer, which with an activation which is soft max, as we've seen previously.
Now, that said, we just in this are called matter.
We're just simply going to call all those different layers which we just created by passing the input.
So here we have our input X, which goes through each and every layer and we get the output right here.
Now, that said, we're going to move on.
To looking at this residual block right here.
So we're going to implement this residual block.
And now this is basically our residual block.
Let's increase this so we could see that clearly.
This is basically our visit, our blog right here.
So we could take one of this.
Let's let's take this one, for example.
We have this residual block right here, and then we get back to the code, see it here.
Now, this is our residual block layer.
Unlike the full model here, we have this residual layer and then you'll see that we have this dotted
Boolean right here, which is true when the number of stripes is drawing from one.
So let's run this year.
Let's let's have this.
We have dot and then let's specify a number of strides.
Say equal one, we let's print out dotted after this.
So there we go.
We have your dotted, we run this, take this off.
You see, it's false.
Now, when we set this to two turns to true.
So basically that's what dotted your dots.
And if you get back, you will notice that we let's get back to this.
We had this we selected this part.
But since this is this isn't a dotted link.
See, this link is full line.
So your our dotted variable will be false.
And when we get to this, our other variable will be true.
Now, let's get back to the code.
You see, we have let's take this off.
You see here we have this, which we understand already, and I will get back here.
Then after we have our two convolutional layers, now we'll define this custom come to DX, which again
we are going to break up subsequently.
So let's just understand that we have this custom curve to DX, which is represented by this year.
So when we have this, this is it right here and this is the other one right here.
Now you'll notice that the number of channels has been passed here and the number of stripes has also
been passed, and that's exactly what was done here.
So you see we passed in the number of channels and number of strides.
Now, since by default, our number of strides is equal one, it means when we don't pass, we simply
see a number of equal one.
But in cases where we have this transitions, we have number of strides equal to.
And so our value here is going to be changed.
Now, that said, we see we define this conv layer, which has a number of channels, kernel size three
as in the paper number of strides.
And then we have the pattern same.
So we ensure that the height and width of our input features remain unchanged.
Now that said, also notice that we have this number of strides here for the second, which is equal
one.
And getting back to the paper that simply because even when we get in, even when we have this transitions
here where we're getting from 64 to 128 and that we're also doing a max pulling, we have this straight
or rather that we have in the striding, not max pulling.
We have this stride value change for only one of the calf layers and not the two.
So you see your only one, another two.
And so that's why right here it is only this one which actually changes for this other one.
It remains fixed, always one.
Now that said, we have the activation layer and then if it's dotted then we're going to have this link
right here.
So if it's dotted, we're going to have let's draw this here.
We're going to have one by one conv layer one by one is actually here.
So we're going to have our one.
Let's take this one off.
We're going to have our one by one conf layer just right here to ensure that this to number of channels
match.
That's the number of channels we get as input here and as output actually match up.
So that's a role of this.
We've seen this already.
Now we get back to the code we see here.
Let's take this off.
We see here this year and then we see the number of the kernel size here is one unlike here where we
have a kernel size of three.
And then we also specify the number of channels to ensure that it matches up with what we expect.
Now, the number of strides here is gotten from this.
So if it's two, you're going to have two is one, you're going to have one.
From here.
We have this set and then we can go ahead and do the calling.
So again, please check out on the previous sessions where we treat models of classes so you understand
exactly what's going on.
So here we have the input.
So the input it gets into the first conv layer, then gets to the second conv layer, that's the output
from the first gets as input.
Second.
And then we get this output.
And now let's suppose that we have a normal layer.
Let's say we have in this one.
Let's get back to this.
Let's suppose that we are working with this one right here.
In that case, then the input will be added to the output directly.
So you see we have this add layer right here, this add layer TensorFlow, it takes the output, which
is this and as this to the input and we now get x eyed goes to the activation, the blue, and that's
our output.
Now the case where we have this in the case where we have this one by one convolutional layer, specify
this the case where we are at this position, then you will see that we will take the input and modify
it before passing it to the output or before adding it with the output.
So you see here we have this output.
There we go.
It remains here and then we modify this or we modify this input.
Sorry.
So we take this input.
There is it here.
Let's take all this off and try to redraw it.
So we have this, we have our rest block and then we have our output.
We have the addition, which is going to be right here, addition.
And then we are going to have our one by one conv layer right here, which comes and adds up with this.
Now this one by one convolution is exactly what's going on right here, and that's what we defined here
since kernel size one.
Now that said, you see, we add this up and then we get our output x add.
So if we are having dotted, we have that else we go through the normal path.
And that's basically why you see here we specify this sometimes and we don't in other times.
Now we have understood how this works.
Let's go ahead to look at the custom curve to the layer.
Now the custom come to the layer is basically made up of our usual account to the layer.
And with a batch number, remember the Resnick model?
The rest of that paper makes use of the batch normalization layer.
So basically here, instead of writing batch non batch nom every time our code, what we just want to
do is combine this two and then we have our batch normalization with our come to the layer together.
So that's it.
We now run the cell.
There we go.
We run the cell, we again run the cell and then we can define our net 34, which is our net model,
which we've just seen rest net 34.
There we go.
We have the rest.
Net 34.
Summary Let's run this.
We get this error.
We need to build our model.
So what we're going to do here is quite simple.
We will not take this resin at 34 and then we'll call this resin 34.
So we'll pass in some inputs into this and 34 model.
So yeah, we supposed to have zeros and then we have one by 256 by two, 56 by three.
So we have this kind of input.
We run that and we see that.
Now we have our model.
So our model summary.
So this is our model summary, 21 million parameters and that's it.
Now we go to the training, but this time around we're going to include some checkpoint.
And so we got this from our previous session where we treated checkpoint in the model checkpoint.
And so you're we're going to ensure that as we train, we save our best model weights.
So that said, here we have this checkpoint called back again.
You could check back on our previous sessions where we treat this callbacks.
So here we have this callback which will permit us to all the weights for our best or our best performing
model or best performing weights, actually.
So here we have a monitor.
We're going to monitor the validation accuracy.
So let's take this off one to validation accuracy, self based best only true.
So that's it.
Let's run this our last function.
But before we move on, let's get back to the section where we had this parameter training right here.
Now we have this custom come to the model, the cells.
We have this custom come to the model which we've seen already, and then we have this batch norm layer.
But it should be noted that with the batch norm layer, we have to specify whether we are in training
mode or in inference or testing mode.
Now, the reason why we are doing this is because the parameters of our batch normal.
Layer will react differently or behave differently in these two different modes.
This means that during training, this layer will normalize the inputs with the mean and variance of
the current batch of inputs.
Now, we've seen this already, and then when we're not training, that's when training equals false.
When inference mode, the layer will normalize its input using the mean and variance of it's moving.
Statistics learned during training.
So the simply means that we have this layer right here with some parameters.
Let's call the parameters, say P, let's call the parameter P so you have some parameters right here.
And then during training, our layer updates these parameters.
But then during inference, we do not want to update these parameters as they were learned during training.
And so we have to specify why or we have to pass in the training or set the training to false when we
are not training the model or when we are evaluating or testing the model.
Now what this simply means is that your will have this to training.
So here I see with pass and training and then here we have training.
So by default we could set this training to true.
So by default we're in training mode and that's what we have.
Now.
Let's run the cell and then get into our residual block.
For residual block here we have training again.
So we have training.
And then since we call on this, we'll have training.
There we go.
We have training and that's it.
We have training.
Let's run this.
We get back to our complete network.
And here we're going to have this training.
So we piece that out here and that should be fine.
So now, when we are not in training mode, we could specify this training parameter such that the batch
nom layer or the batch nom layers parameters aren't modified.
So we have that and that's it.
So we have that set.
Now let's run this and this time around we're going to set this training.
So let's set training to be true.
We have that true.
Let's make sure that this was passed in here.
So let's have this training and the default is true.
Okay?
We run that, we run this, and that looks fine.
Now, we could set this to be false or we don't.
When we don't put any value, it means it's true.
So we set this to be false.
It's training to be false, and that's it.
So there we go.
We have this set.
Now let's get back to our last function.
So we were at this point, we have our metrics, we run the metrics, we compile the model, but this
time around we use a higher learning rate.
So one thing you could also do is as describing the paper decrease this learning rate as soon as the
model has plateauing.
So we could start with a learning rate in the paper.
It should be 0.1, although here we're going to start with 0.01.
So yeah, the say you have this learning rate and then when it starts plateauing, you drop to 0.01
and then you go with that and then instead of plateauing, you drop and so on and so forth.
So this is what the purpose in the paper and you could always implement this.
And we've seen this in some previous section where we implemented this kind of callback which permitted
us to schedule our learning rate.
So that's it.
Let's take this off, get back to the code.
And we have we've run this already, so we could start with the training.
Now here we're going to train for 60 epochs and we're going to include the callbacks.
So let's have this callbacks and we have our model checkpoint callback, which we had defined here.
Let's copy this.
There we go, copy this, and we have it here.
So that's it.
We can now run the cell.
Now we've learned this training.
I we've had this error here.
Seven The model to the ADF five format requires the model to be a functional model or sequential model.
It does not work for subclass models like in our case because such models are defined.
We are the body of a python method which isn't safely serializable and consider saving to the TensorFlow
saved format by setting save format ATF or using save weights.
So now we're going to save those models in the TensorFlow format.
All we need to do here is specify this folder and that should be fine.
The mode is max.
Now, since we want to store the weights which have the highest validation accuracy, that's fine.
Let's now run this again, train our complete and we achieve an.
Accuracy or the best accuracy of 83.6%, whereas for accuracy plot right here.
And then we could evaluate the model here we get 82.3% and 94.6% for the top K accuracy.
Now, what if we load our best model?
Because the model we have in here is the latest model, the very last one.
Now let's load our best model and re evaluate this.
Let's add this code.
So then we go ahead and load our best weights.
Here we have resting and teddy for the load weights.
Then here we have our best weights.
So this should be a string.
Since it's our folder, there's a photo where we start the weights rest net.
We run this, that's fine.
And then we evaluate our model.
So we get in this accuracy of 83.6 and then top K accuracy of 95.6%.
Now we go on to test this.
That's fine.
And here are some results we get.
Happy, angry, sad, happy.
Yeah, we miss.
We miss one.
We miss this one.
That's two, three.
And that's it.
So we missed three.
And then we have let's add the cell.
We have 13 out of 16.
And this is correct, 81.2%.
Correct.
Okay, So that's it.
We've gone from 79% to 83% by modifying or changing our model.
Now, let's plot out this confusion matrix and see what we get.
There we go.
Your results, which are much better than what we have had so far.