-: Hello, welcome back to the course on Deep Learning.
In the previous tutorial, we found out
what convolution neural networks are all about.
And today we're going to dive into step one convolution.
So this is the convolution function.
I know we tried to survey from mathematics
and keep things intuitive, but I couldn't help
but share this formula for you because it is so simple.
A convolution is basically a combined integration
of two functions
and it shows you how one function modifies the other
or modifies the shape of the other.
And if you've done any signal processing
or electrical engineering
or profession where signal processing is required
you would've inevitably come
across the convolution function.
It is quite popular.
Now, once again, we're going to keep the mathematics light
or keep them separate.
And if you'd like to get
into the math behind the convolution neural networks
a great additional read is
"Introduction to Convolution Neural Networks," by Jianxin Wu
who is a professor at the Nanjing University in China.
This paper was published literally days ago
like five or six days ago, and it is oriented specifically
at people who are starting out at beginners
who are getting to know convolutional neural networks.
So the mathematics there should be accessible.
I actually emailed Professor Jianxin Wu
and yeah he said his whole goal is to make
break the complex things down
so that people who are new to this field can understand.
And also he mentioned
that he's got some materials available on his homepage.
So if you, in the URL
if you just remove the last two parts and you just go
to like /wjx to that part, that's his homepage
and you'll be able to find more additional tutorials
and materials which haven't been published as papers
but he uses them in his tutorials.
So you might find those useful.
So browser around there if you'd like to get an introduction
into the mathematics behind convolutional neural networks
and kind of build a solid base around that area.
But we're going to move on
and we're gonna talk about the convolution.
So what is the convolution in intuitive terms?
Here on the left we've got an input image as we discussed.
That's how we're going to look at images
just ones and zeros to simplify things.
And you can see the smiley face there.
There we've got a feature detector.
So a feature detectors are three by three matrix.
Does it have to be three by three?
No, it doesn't.
Alex Net I think uses seven by seven.
And then some other, one of those other famous ones
uses like five by five feature detectors.
They can be different, but usually you'll see
that they are three by three
and they are reasons to make them three by three.
So we eventually stick to the convention
of a having a three by three feature detector.
Also, the feature detectors called these are important terms
because you might come across them.
They're many different terms for the feature detector
but the most common ones are feature detector.
Or you might hear it being called kernel
or you might hear it being called filter.
So in this course we're gonna be using
either filter or feature detector interchangeably
but just bear in mind that it has those names.
And a convolution operation is signified
by an X in a circle, just as you saw in the formulas before.
And here what happens is on an intuitive level
or just to think of it
in terms of what is actually happening in the background
rather than the mathematics.
Well, you take this feature detector or filter
and you put it on your image like you see on the left.
So you cover the, for instance, in this case
the top left corner, the nine pixels in the top left corner
and you basically multiply each value by each value.
So respective value.
So the top zero by the top left value
by the top left value, then basically
position number one one by position number one, one
position by number or 01 by 01, 02 by 02 and so on.
So just it's element wise, multiplication of these matrices.
And then you add up the results.
So in this case, nothing matches up.
So always it's always either a zero by zero, zero by one.
So the result is zero.
Here you can see that one of them matched up,
the one on the left matched up
and therefore we got a one here.
Nothing matched up, nothing matched up, nothing matched up.
Then we move on to the next row.
So, and the step at which we're moving
this whole filter is called the stride.
So here we have a stride of one pixel.
So here you can see again something matched up,
the bottom right corner matched up.
Again, stride, bottom one in the middle, matched up
here top right one matched up.
Then nothing matched up.
The stride is one.
You can change the stride, you can make it one, two
you can make it three, whatever you like.
The conventionally, the one that works well
is usually at two.
So that's what people stick to.
And we'll talk about what the stride is
towards the end of the tutorial.
So here we've got, we're matching up, so we just keep, oh
here you can see we've got a two
because two of them matched up
and so on and so on and so on.
And there we go.
There's another one that matched up.
There we go.
And there we're done.
So what's have we created, right?
Couple of important things here.
The image on the right is called a feature map
also has several terms.
It also can be called, sometimes it can bold feature.
So when you apply convolution operator to something,
it doesn't become convoluted it becomes convolved.
And yes, sometimes I think to myself
in the wrong way, but it's the correct term it's convolved
it's a convolved feature, or it can also be called
activation map.
But we are gonna be calling it a feature map in this course.
So it can be called any one of those things.
And what have we done here?
Well, as you can see, we've reduced the size of the image.
That's number one.
And that's the important thing I wanted to mention
about your input image
and the featured detect and the stride, right?
If you have a stride of one
you can see the image reduced a bit.
But if you have a stride of two
the image is going to reduce more.
So the feature app is gonna be even smaller.
And that's an very important function
of the feature detector
of this whole convolution step is to make the image smaller
because it'll be easier to process it
and it'll be just faster and it'll be just faster.
Because imagine like here we've got a what
a seven by seven image.
But imagine if you have a proper photo, right?
Or you have like a 256 by 256 pixel image.
That's a huge number of pixel 256 squared.
Or like, let's say you have a 300 by 300 pixels,
so we don't get confused with the RGB 256
let's just say like we have a 300 by 300 image
in terms of size in pixels
then you have 300 squared number of pixels.
That's a huge number.
And therefore feature detectors
will reduce the size of the image,
and therefore stride of two is actually beneficial.
But then the question is, do we lose information?
Are we losing information
when we're applying the feature detector?
Well, some information we are losing, of course
because we have less values in our resulting matrix.
But at the same time, the purpose of the feature detector
is to detect certain features
certain parts of the image that are integral.
And so for instance, if you think about it this way
like the feature detector has a certain pattern on it
the highest number in your feature map
is when that pattern matches up.
In fact, the highest number you can get
is in an now simplified example,
is when the feature matches exactly.
And you can see with that number four,
we have in our feature map, that's exactly,
so if you look over here,
that's exactly where this feature detector,
because there's only four ones in it matched perfectly.
So you can see this part over here.
So the feature was detected here.
And as we discuss at the very start of this section
that is features is how we see things,
is how we recognize things.
We don't look at every single pixel, so to speak
in what we see on an image or in real life.
We don't look at every single pic.
We look at features, we look at the nose, the hats
the feather, the the eyes under or the little black marks
under the cheaters eyes to distinguish it
between a cheater and a leopard, or the shape of the train,
to distinguish it between a bullet train and a normal train
and so on.
So we don't look at everything, we look at features
and that's what we are preserving
and that's what the feature map helps us preserve actually.
That's what it allows us to bring forward
and get rid of all of the unnecessary things
that even as humans, we don't process this.
So much information going into your eyes
at any given time, like gigabytes of information
if you look at every single dot, if not terabytes
of information going into your eyes per second.
And still we're able to process that
because we get rid of what is unnecessary only focus
on the important features.
Features are important to us.
And that is exactly what the feature map does.
So now moving on
this is our input image and you, we create a feature map.
So the front one
let's say the front one is the one we just created
but then how come there is many of them?
But we create multiple feature maps
because we use different filters, right?
And that's another way that we preserve
lots of the information.
So we don't just have one feature map
we look for certain features and then
or basically the network decides through its training.
And this is something we'll discuss
towards the end of this section.
Through its training
it decides which features are important
for certain types or certain categories,
and it looks for them.
And therefore it'll have different filters.
And we'll talk about filters just now
but basically it'll apply these filters.
So to get this feature map
it applied a filter like the one we saw.
But then to get this feature map it
apply a different filter to get this feature map
and apply a different filter and so on.
And so basically it just creates these feature maps.
And actually that's why personally
I think the term feature detector is better than filter.
So remember over here we have this filter,
which we also can call a feature detector.
Well actually the word feature detector
I think is better suited.
And the reason for that is that's what the purpose is.
We don't wanna just filter out our image,
but even though that's the whole
that's the same just question of terminology.
But basically we want to detect features.
In this layer, we're going to,
or in this feature map we've detected
where certain features are in the image.
In this feature map we've detected
where certain other features are,
where a certain specific feature is located.
And this feature map we've detected
where a certain other feature is located on the image.
So that's what we are doing.
And let's have a look at a couple of examples.
So here we're using, and this is from gib.org
their documentation, it's a free like of kind of tool
like paint and you can use it to adjust your images
or work with your images.
But basically they have some valuable examples
in their documentation.
And here they have a picture of the Taj Mahal
and you can choose which filter you want to apply.
So if you download this program
and you upload a photo into it
and then you can actually start a convolution matrix
and apply filters
and you will see that these things, these convolution matrix
I actually applied in image processing and design and so on.
So let's have a look at what we get.
So if we apply this filter five in the middle
minus one, minus one, minus one minus one
you can see that it sharpens the image.
And yeah, so this is
it's quite intuitive if you think of it.
So five is the pixel, the main pixel,
like in the middle of the filter or the feature detector.
And then minus one, minus one, minus one
just you kind of like reduces the pixels around it.
In an intuitive sense then blur.
So basically it takes equal significance
gives equal significance to all of the pixels
around the one in the center
and therefore it combines them together
and you get a blur edge in hands.
So here you can see that minus one and one
and then you get zeros, right?
So you delete, remove the pixels around the main one
in the middle and you only keep this one at a minus one.
And it gives you an edge.
This one's a bit harder to understand how it works.
Like probably harder just to think
of it intuitively edge detect, right?
So this one probably makes more sense, right?
You take the middle one, you reduce the middle one
probably like the strength of the middle pixel.
And then you look for the ones, you look for these ones
you increase the strength of the ones around them.
So you have the ones there.
Yeah, so that gives you like an edge text
and you can see what you get there.
And boss another one.
So the key here is that it's asymmetrical
and you can see the image becomes asymmetrical as well.
So you've got like that kind of feeling
that it's standing out towards you.
And that's what you get when you have like minuses here
And pluses here, again, this is very
this is getting a bit technical now,
but at least we can get
some kind of intuitive understanding.
Lets just go quickly through them again.
So there's sharpen, there's blur, there's edge enhance
there's edge detect, there's emboss.
And so as you can see, these are great examples
of the same image, but we are getting feature maps.
So we use different feature detectors
to get different feature maps of the same image.
And therefore now we have lots of this versions
of this image where in each one we've tried
to detect certain things.
These terms, they're not applicable to us.
Like I said, like in boss is probably not applicable
to us in terms of convolutional neural networks.
But edge detect that's important.
We want to detect the edges,
edge enhance probably not, blur, sharpen.
So certain things like edge detect
is probably the most important one for our type of work.
And in terms of understanding,
like computers they will decide for themselves
their neural network will decide for itself
what's important, what's not.
And it probably won't be even recognizable to the human eye.
You won't be able to understand what those features mean
but the computer will decide.
And that's the beauty that of neural networks
that they can process so many different things
and understand without even having that intuition
without having that explanation why
they will understand which features are important to them
whether we have a name for them or not,
that's an irrelevant question
for their artificial neural network.
And my favorite one, here's a image of Jeffrey Hinton
photo of Jeffrey Hinton passed through one of these filters.
All right, so that brings us to the end of today's tutorial.
I hope you enjoyed learning about convolution.
The key takeaway is that a convolution,
the primary purpose of a convolution
is to find features in your image using the feature detector
put them into a feature map.
And by having them in a feature map,
it still preserves the spatial relationships between pixels
which is very important for us to,
because if they're completely jumbled up,
then we've lost the pattern.
And at the same time, it's important to understand
that most of the time the features a neural network
will detect and use to recognize certain images
and classes will mean nothing to humans.
But nevertheless, they work.
And that's what convolution is.
And I look forward to seeing you on the tutorial.
Until then, enjoy Deep Learning.