Hello, everyone, and welcome to this new and exciting session in which we are going to discuss the
VG model.
VG actually stands for Visual Geometric Group and this was presented in the paper by current Simeon
and Andrew Zimmerman, entitled Very Deep Convolutional Networks for Large Kill Image Recognition.
In the session we are going to discuss different methods which the authors of the VG paper used to drop
the top one validation error rate from 38.1 to 23.7, where this 38.1 was achieved by the breakthrough
KOVNER model, which is the Alex Net model.
Now, in the previous session in which we treated the Alex model and we saw the power of working with
cough nets and solving recognition tasks, one thing we could notice already or very clearly from this
model is that it's quite shallow.
And so Siemionow and Andrews's men go even deeper with the VG model.
In this paper, the authors investigated the effect of the conflict of the convolutional network depth
on its accuracy, not just words, depth and accuracy.
So unlike the Alex Net, where the depth is relatively small and is actually a shallow network, here
the authors use a deeper comb convolutional neural network and make use of smaller convolution filters.
Now recall with Alex net from the very first layer, we already had 11 by 11 filters and we argued that
this helped in capturing large spatial dependencies.
Now we'll explain how it's possible for us to make use of this smaller and more economical convolutional
filters while still capturing large spatial dependencies like the bigger five by five and 11 by 11.
Filters will do to better understand why it's better to work with two three by three conv layers as
compared to working with a single five by five conv layer.
Let's consider the following examples.
So here we have these three examples.
Let's start with this first one here.
So let's have this.
We start with this part here and this one, if you notice, it's five by five.
So here you have kernel size of five, input size of ten and no pattern dilation of one and stride of
one.
So here we have this here and you can see we have this output, which is six by six.
So you have the six by six output.
And the way each and every pixel in the output is gotten is quite simple.
You have the kernel right here.
Let's have this kernel.
We have this kernel which has been passed on a particular patch on the input and then this produces
the outputs.
So simply take this kernel values, multiply by these values, add them up and then get this output.
Now this is the case of five by five.
So the receptive field Spaniards quite great.
See, we could get or we could capture this information in this patch in the input right here.
Now for the number of parameters, you could simply count this.
We have five times five, which is 25 parameters.
Here we have 25 parameters.
And then for the learning capabilities, this one is quite limited here.
Here we have 25 and your will say this is quite limited because if we suppose that we have an input,
let's say we have this input which is passed through a single conv layer and then obviously the conf
layer ends with a nonlinearity, in our case the railroad nonlinearity.
So let's say we have the nonlinearity right here and then we get the output.
Now this doesn't capture as much complex information as we would be able to capture if we had to conv
layer stacked.
Now here, when you stack this first one, this is a case of three by three and this is the case of
five by five.
So year after this first three by three, we then have this other three by three with this other non
linearity.
So here we are able to capture much more complex information from our input information of my input
data as compared to.
And we just have one single cough layer.
So that's why you're.
Let's take this off here.
We the learning capabilities isn't as much as this.
Two, three by three.
Now, let's get to the receptive field span for the two three by three.
To understand this, we are going to take this example right here.
So take note that here we have input size of ten.
So we're going to maintain this.
We have input size ten.
Okay.
Now, the kernel size is three, unlike here where we have five pattern dilation and strike number the
same.
Now notice that the output we have here is eight by eight.
All like your world, six by six.
Now since we are having two, three by three conv layers, we're going to get this output just like
the output we get here is this.
So let's draw, let's put this here.
We have this input, the error is it.
And then we get this output which is going to be input of another three by three.
So this is the first three by three and then this is the next three by three.
So this will be the input of another three by three conv layer and then we should produce the output.
So what we're trying to show you is when we have this output year, which is the input of this next
layer.
So this is this two year combined as one, this forms one.
So this year, this by it now is an input.
So you're instead of having ten, we have eight.
So we take a sample size of eight.
There we go.
Same kind of size patterns dilutions try the same and what we'll want you to notice.
Let's take this off.
What I want you to notice here is the fact that this output is six by six.
And so this means that this year, if we follow this, let's take the mouse.
If we follow you see, this year captures the same information as you would as this one year would capture.
And so we could confidently see that the receptive fields paneer is quite great Now for a number of
parameters.
Here we have nine and here we have nine.
Nine plus nine is 18.
So 18 year 18, you see clearly that this model now is cheaper as compared to a model which uses a five
by five or even 11 by 11.
Now, for the learning capabilities, we've seen this already because we stacked two, we're able to
capture much more complex information from the inputs.
And so this is great.
So in all we see that it's better to use this conv layers with smaller can all sizes.
We want to thank Edward Young for providing this convolution visualizer which you can find on as young
dot GitHub dot IO.
So at this point we've understood why the authors of this paper prefer to work with this smaller kernel
size convolutional layers that's three by three.
And with the smaller conf layers they were able to push the depth to 16.
That's between 16 and 19 layers where the 16 layer version is VG 16 and then the 19 layer version is
VG 19.
In this table we have the summary of those models.
Here we'll focus on the 16 and 19 weight layer models.
So here we have 16 weight layers.
You see, we start with two conf layers and then max spool and then two layers and then max spool and
then three layers, max pull, three layers max pull and then three conf layers.
Then from here we have the max spool, we have a flattened layer and then we have this three fully connected
layers which end up with a soft max.
Since we are dealing with a multiclass classification problem, the authors also noted that the usage
of the hour and normalization that is local response normalization which we saw in the Alex net, did
not improve performance, but instead led to increased memory consumption and computation time in the
section to describe the training process.
Then from your testing and we could get some results.
And here we see we have this top one validation error and we also notice that we have this component
layer or this confident model.
Your A now A BCD.
You could get them by checking on this table here you have a L with the R and normalization, B, C,
D, and E right here.
So basically these are the different models and your other results.
So we'll working with the local response normalization we know.
Notice that this seems to have even the highest error.
So that's why the authors did not make use of this normalization technique.
Now we get the best results with a VG 19.
So this year 19 we get 25.5 for top one and then eight for top five.
Now, if you're new to this notion of top one vowel error and top five while error, you could check
out the previous section where we discussed the top one accuracy and top five accuracy.
You're again.
Now we have this comparison with the state of the art solutions at the time you here Alex net over feet
inception RMS are a clarify this this model by clarify I and Ziegler and Ferguson's model.
So here we see that the VG has at the time had the best results.
And with this we can conclude that stacking up those conv layers with smaller kernel size actually helps
in getting better results.
And in subsequent sections we'll see the limit of just stacking up many conv layers, as with the VG.