Hello, everyone, and welcome to this new and amazing section in which we are going to treat the mobile
net architecture.
This was first developed by Google researchers in 2017 and we have the mobile net version one and 2019.
The again, develop the mobile net version two.
And after this, there was a mobile version three.
But yeah, we are just going to focus on this mobile net version two entitled Inverted Residuals and
Linear Bottlenecks.
By just looking at the title, we could guess the type of environments for which this model was built
for.
In fact, the mobile nets have been built for environments with low compute resources like the mobile
and edge devices.
So in this section we are going to focus on what permits this model that is the mobile net v two to
perform quite well in terms of speed while producing high quality results.
There are two major techniques which make the mobile net version two very powerful, all which permits
them all which permits us work at higher speeds while still maintaining reasonable quality results.
Now these two are the dep y separable convolutions and the inverted residual bottleneck which we have
right here.
Yours, those separable convolutions, this irregular convolution, your separable convolution.
And we'll start by explaining what that separable convolution is.
A deep, separable convolution is simply a combination of a depth wise convolution and a point y convolution.
Now this point why convolution is not in different from a normal convolution layer, but with kernel
size of one so one by one convolution.
Here, that's the point where it's convolution and before this we have the depth wise convolution.
Now to understand like this, now the deep separable convolution which is this to put together in sequence
or sequentially.
Now to understand what this depth wise convolution is, actually let's get back to this demo where we
saw how usual convolution operation works.
As you could see here, we have this inputs.
Basically let's let's have this, let's toggle the movement.
So yeah, we have some inputs.
Now this input is three dimensional.
As you see here we have one, two, three, So 012 we have this three dimensions.
Let's change the color so it's clear for you to see.
We have this first dimension, we have the second dimension, and then we have this other third dimension.
Now, what goes on during the convolution operation is that we have this canal, so we have this year
and then we have also some canal.
So in this case, three by three canal.
We have this on a canal here, three by three.
And the reason why it is three by three is because we have three channels here and the input.
So because the inputs are three channel or have three channels, we have a three channel canal.
Now, here we could add this, we could add a foot channel right here at this four channel.
And then we'll also have here four of this canals.
And then what happens is going here during the convolution operation is exactly what we're seeing here.
So we have this one which is placed at a given position.
Let's pick its color is place at this position, for example.
And then we multiply all the values, like let's say we add this point here.
You see at this point we can see how this filter in this case, this filter.
So this filter is actually we match this filter with this one, we match this one with this in red.
We match this one with this in green.
Now, since there are three, obviously we don't have four.
Now let's get back to this operation where we take this one and multiply with all the values we have
here.
So when we multiply all the values we have here, we get, for example, in this case one by zero plus
zero times zero plus one times zero.
You see all this at the top canceled.
And then we have zero times zero zero times one.
On council.
We have one times one year, so we have one and then we have 100 negative, one times one negative one.
So we have negative one and then negative one times two.
We have negative two.
So this gives us a value of negative two and then we move to the next one.
This one, let's change the color.
You see, we have one times.
All this at the top is zero.
So when we multiply all this by because it's just simply matching.
So when we have one times zero, we have zero zero times zero zero, one times zero zero, one times
zero zero, negative one times two, negative two, then negative one times zero.
We have zero negative one times zero zero, negative one times to negative two.
So this gives us negative four.
At this point we have negative four.
We move to the next one.
Here we have negative one.
All this is zero, obviously.
So we just get to this one one times one.
we have one zero or negative one and then zero.
So we have one.
Let's write that we have one and minus one because this is negative one times one minus one, it gives
us zero.
So here we have negative six.
When you add all this up, basically we taken this, we multiply and then we add, yeah, we take this
multiply, we add, take this multiply, we add and we add all these values.
And this gives us your negative six.
Now, because we have this bias of one, we add plus one, it gives us negative five.
That's how we obtain this value here.
So we obtain this one right here.
Now we'll repeat the same process for all these filters and all the different positions.
So basically, let's toggle this so you see what happens.
You see we repeat this process, we move, we move, we go down and that's it.
There we go.
So we repeat this to the end and we have this final value right here.
Then if we want to have you see that we get this.
So what we have is let's take this off, let's take this orange off.
What we have is now we have this output.
So we have the input, which is of the of channel, which has a number of channels to read here.
We also have this number of channels three, and then we have an output which has a number of channels
one.
But if we want the number of channels to be equal, say to like in this case for the output, we have
one channel and this other channel, then we need to increase the number of filters we have here.
So basically here we have how many filters.
We, we, we could create this again.
So we will have now two.
If we want to have output two, then we will need to have two of this of this three dimensional filter.
So we will have this again, we would have this with its own weights, obviously, and we will have
this and that's it.
So because we have this two now, we will no longer have an output with one channel, but now an output
with two channels, as you could see here.
Now notice how as we click this, we move to this next.
So this is the this year is this one year.
Let's pause this.
So this year is this one right here.
And this one is this one right here.
Now, with this, we are able to get this other channel for the output.
And so that's how the convolution operation works for a normal convolution.
Now, if we get to the depth wise convolution, we will find that this be different from this method
for the depth wise convolution.
As the name goes, this computations are done depth wise.
So first of all, now here this output will only be gotten from interactions of this channel and this
channel right here.
So what goes on here is we take this and then we pass it around, as we usually do, and then we obtain
this new output right here.
So if this is three by three, then we obtain some values one, two, three, four, five, six, seven,
eight, nine.
So these values will be different from this one, because the way we compute these values is different
from the way these values were computed.
The way we computed this value was we take this year, pass it here, take this, pass this, take this,
pass this here, and then add all our resulting sums to obtain this value of negative five as we saw.
But in this case, what we'll obtain for this.
Value will be this five.
What we'll obtain here will be one times zero zero times zero plus one times 0001 times one.
We will have one and then you will have zero year one times one.
Negative one, negative two, negative one, negative two.
It gives us negative two.
So what we'll obtain here will be negative two, and then we'll move on to the next, we'll move on
to the next, we'll go to this next position.
We'll get some value, as we've seen here and up to this last value right here.
So the way we got this was we took all this in different four different channels and then we added them
up to get this bigger.
We just get this directly by taking for each and every channel and just producing the output like this.
So this means that with this we are going to have like here, let's take this off with this because
we are having the each channel all for the filters producing its own output.
We're going to have this three producing three different outputs.
So you already we have three outputs, unlike here where we had to output and the two outputs here were
controlled by the number of canals we used, because here we use two canals, we have two outputs.
But here we this doesn't matter.
You're the number of output or the number of input channels we have here will dictate the number of
outputs.
So here we have three channels.
So we just obviously have these three different outputs.
Now since we have the three output and we want to be able to control the number of output channels,
what we'll do now after the depth wise convolution, which is in fact what we've just explained here
is we're going to add now the one by one convolution.
That's the point where it's convolution after adding the one by one convolution, we are going to specify
the number of channels here and this number of channels of this one by one convolution that will permit
us leave from a certain number of channels like in this case three to another number of channel or to
give a number of channels, let's say two.
So after getting this three channels or getting up with three channels, we now get, we pass to this
point where it's convolution and now we're going to get just two channels.
To better understand this.
Let's take this dead wide convolution image from papers with code.
So you're.
Let's try to reduce this anyway.
Let's have this.
You see, we have one, two, three.
You see this?
One, two, three.
This is a five by five canal.
And then we pass this.
Notice how each and every one is now responsible for its own output.
See this?
See, we have this orange with this foil for this particular channel.
It gives us an output.
The red gives us an output.
The yellow gives us an output unlike previously, where we'll take this pass this year and then add
all this together.
But now instead, what we do is we just simply carry out that addition at the level of the channel,
and then we get this output right here.
Now, let's see why that wise or that separable convolution is more efficient.
And we'll do this by calculating the number of filters.
So here we find the number of filters we need to get from this input, which is one, two, three,
four, five, six, seven.
So we have this seven by seven by three inputs which we want to convert to this, three by three by
two output.
And here the number of filters and the more parameters we used can be calculated as here we have nine
times three.
Obviously in this, nine times three comes from the fact that we have for each of this, we have nine
and times three.
The three is from here actually, because we have three input channels, then we'll have three filter
channels.
And so we're going to have the filter size, which is three by three.
I think we should change this color so it makes it clearer.
Let's change this.
So here we have seven by seven by three.
The three is for this number of channels.
Here we have three by three and then this is three by three, four single one, then times three, this
number of weights.
But now, because we want to have let's change this color again here, because we want to have an output
with two channels, then we would multiply this again by the number of output channels two.
So this is like some general formula to get a number of parameters.
We're going to omit the biases.
So here we have three by three by three, two and seven times 254 parameters.
Now this means that if we have or we want to have a number of channels of, say, 16, then this will
give us 432.
Now, let's consider that we are dealing with depth wise convolution and the point where is convolution
which both form the depth separable convolution with a depth wise we will have.
First of all, we have to note that this is no more needed.
We just need this.
So what we have will be three by three by.
Three.
Now the story comes from the number of inputs by now to get the number of outputs.
We wouldn't carry out any multiplication here because basically the output from the depth wise convolution
is a three by three by 3/10 or in this case or more.
Or we could just say it's a three channel output.
Since we have three channel inputs, we'll have three channel output.
So here to obtain the outputs or we just need this, we saw this already, we just take this multiplier,
get the output, get this first one, we take this multiplier, get this next one, take this multiplier,
get this next one and we're good to go.
So once we have this, we now add the the weight for the point.
Why is convolution now for the point twice it's one by one can also see it's quite cheap compared to
the three by three channels.
So one by one here we have one by one.
So let's just put this here.
We have one by one now times.
The inputs, just like with the usual convolution.
It's the same thing actually, because you're for the for the usual convolution.
Yeah.
We have to calculate number of weights, just get the kernel size then like one of the the kernel size
times it cannot size times the number of input channels times number of output channels.
So here we have the kernel size times kernel size times the number of input channels, which in this
case is three and the number of output channels which is two.
So that's what we have Now.
If we multiply this, we have 27 plus six.
That gives us something like 33.
You see that this gives us territory.
Now, one interesting point to note here is if we multiply, if we modify this and take and say we want
to have 16 output channel, then we'll change this two and put 16.
In that case, our answer will be 27.
Here we have 27 plus 48 now 27 plus 48 or we get that quickly, five seven, that gives us 75.
So you see that here we have increased this number of channels from 2 to 16, number of weights, 75.
But when we did that year, number of weights went to 422.
So clearly that that wise set or the depth separable convolution is one that is way cheaper than the
normal convolution.
And in the paper the others argue that this kind of convolution permits us to reduce the computational
costs by 8 to 9 times than that of standard convolution.
While we have only a small reduction in the accuracy.
That said, from this diagram here, you should now understand why when we presenting a regular convolution,
the others have this filter.
You see the filter here which has this depth into the input.
You see this depth right here, See this?
Whereas when we present the separable convolution, we have this filter which doesn't have any depth.
And that's because here we have no inter channel computation or calculations as compared to this one.
And then after we have this point wise, now the point wise is regular convolution.
So we see we have this depth again, but now it's smaller in size because it's just a one by one filter.
The next improvement we should look at is this inverted residual block right here.
So it's first of all called inverted in comparison to the residual block.
Now, the residual block, as you may notice, you have this large, relatively large channel, and
then it becomes small in the middle and then it becomes large in the outputs.
So this means that we have some inputs, we have some inputs, and then we have our residual block and
then we have some outputs.
Then we have obviously this link right here from the input to the output.
Now with this you see this becomes the data or the inputs big.
He goes to small and then big.
But here's what we have is we pass in a relatively small channel, small number of channels or an input
with relatively smaller number of channels.
So we are small and then in our block it becomes the number of channels increased and then as output
number of channels reduced, hence the term inverted residual block.
Now, in addition to the fact that we're using depth wise convolutions instead of the normal convolutions,
the fact that we have relatively lower dimensional data getting into this block and lower dimensional
data, getting out means that we could transport very low dimensional data throughout our mobile network.
So here we have low dimensional data getting in, low dimensional data getting out, and then inside
we have this expansion layer right here as this expansion layer permits us to capture as much information
as possible from our input features.
One thing to also note is the fact that we're using the rate of six.
The rule of six is different from a usual value in the sense that we the value we have for all x less
than zero.
The value is zero for all x greater than zero, the value is x.
So we have this Y equals x line right here.
But with the rate of six as from the value six, we actually clipped this output.
So what we have here is all values or let's get.
Back.
What we have here is we have the values.
It remains X, but once we get to six, it gets clipped.
So for all values of X greater than six, the value remains at six.
So that's a value six.
And then one other important point is the fact that because we are doing or we are carrying out this
projection from high dimensional data to a low dimensional data, this réélu nonlinearity generally
will cause us to lose too much information.
And because of that, there is no real activation in the final layer here.
You can see this in the summary right here.
You see we have the input.
Then we have the expansion factor, which is DT.
This means that now we have this hyper parameter which we can turn.
So if we want better results, we can turn this expansion factor so that it permits us to get this better
results.
So here we have this expansion factor DT here, which expands number of channels.
So we get in with K and then we now move to TC and then we have this TCA which now takes us to K Prime.
Now also note that here we have the realm of six, really six, but here we have no activation.
Now that said, this is the summary of our mobile net version two.
So here you have mobile version two, you have the different bottlenecks.
There we go.
And then we have this come to the average pull and then come to the this figure right here also shows
us how the mobile net version two outperforms the mobile net v one shuffle net and the nice net.
You see that with the mobile net version two right here.
If you pick this, let's pick for example, this two, you see the number of operations or the computation
cost here is almost similar, but we see this great difference in accuracy where the mobile net v two
outperforms the mobile net V one.
Then apart from classification, the mobile net v two has been used in other tasks like object detection,
semantic segmentation and other computer vision tasks where we have low compute resource.