Hello and welcome back to the course on Deep Learning.
Today we're talking about max pooling.
And we've got some very exciting slides coming up ahead.
And even a special surprise at the very end of the tutorial.
So let's get started.
The first question is what is pooling and why do we need it?
Well, to answer that question, let's have a look at these images.
On these three images we've got a cheetah.
In fact, it is the same exact cheetah on the first image.
The image is positioned properly and the cheetah is looking straight at you.
On the second image it's a bit rotated and the third image is a bit squashed.
And the thing here is that we want the neural network to be able
to recognize the cheetah in every single one of these images.
In fact, this is just one cheetah.
What if we have lots of different cheetahs?
Here's a cheetah, here's a cheetah, here's another cheetah.
Here's a cheetah, here's a cheetah, and here's a cheetah.
And we want the neural network to recognize all of these cheetahs
as cheaters.
And how can it do that if they're all looking in different directions?
They're all in different parts of the image.
They're like their faces are positioned in different parts of the image.
Somebody on the right hand side, somebody in the left corner,
somebody is in the middle.
They're all a bit different.
The texture's a little bit different.
The lighting is a bit different.
There's lots of little differences.
And so if the neural network looks for exactly a certain feature, for instance,
a distinctive feature of the cheetah is
the tears that are, on its face going from the eyes or the,
the shadows that look like tears,
the texture or the pattern that is going from its eyes down.
It's, on the size of its nose.
It looks like tears.
That's a distinctive feature of this, cheetah.
But if it's looking for that feature, which it learned from, certain cheetahs,
in an exact location or an exact shape
or form or texture, it'll never find these other cheetahs.
So we have to make sure that our neural network,
has a property called spatial invariance, meaning that it doesn't care
where the, features are located.
Not not so much as in which part of the image,
because we we've kind of taken that into consideration
with our map, with our, with our convolution layer.
But it doesn't have to care if the features are a bit tilted,
if the features are a bit different in texture,
if the features are a bit closer, if features are a bit further
apart relative to, relative to each other.
So if the feature itself is a bit distorted, we our neural network
has to have some level of flexibility to be able to still find that feature.
And that is what pooling is all about.
So let's have a look at how pooling works.
Here's our feature map.
So we've already done our convolution and we've completed that part.
And now we're working with the convolution layer.
Now we're going to apply pooling. So how does it work.
We're going to be applying max pooling.
there's several different types
of pooling complies mean pooling max pooling some pooling.
And we'll comment on those towards the end of this tutorial.
But for now we're just applying max pooling.
So we take a box of two by two pixels like that.
And again it doesn't have to be two by two.
You can choose any size of box.
And again we'll comment on that towards our tutorial.
And you place it in the top left hand corner
and you find the maximum value in that box.
And then you record only that value and you disregard the other three.
So in your box you have four values.
You just disregard three.
You only keep one the maximum, which is one. In this case.
Then you move your box to the right by a stride.
You select the stride once again.
So here we select a stride of two and you that's what you normally select.
You can select a straight of one. You can select.
So there are overlapping boxes.
You can select any kind of stride that you like even three if you want.
But we're selecting a stride of two here and that's what is commonly used.
And then you repeat the repeat the process.
You record the maximum here.
If you crossover and it doesn't matter, you just keep continue
doing what you're doing.
So, you still record the max over here.
Zero.
here the maximum is four.
Here the maximum is two here.
The maximum is one, zero one as a row, two and then one.
So as you can see, a few things happened.
First of all, we still were able to preserve the features.
Right.
the maximum numbers they represent because we know how the convolution
layer works.
We know that the maximum or the bit large numbers in your feature map,
they represent where you actually found the closest similarity to a feature.
But by then pooling these features,
we are first of all getting rid of 75% of the information
that, is not the feature which is which is not,
the important things that we're looking out for,
because we are disregarding three pixels out of four.
so we're only keeping 25%.
And then also because
we are taking the maximum of the,
pixels that way or the values that we have,
we are therefore accounting for any distortion.
So for instance, two images in which, for example,
the cheetah's, tears on the eyes are
in one image, they're a bit to the left, or a bit rotated to the left.
And another one, they're a bit and they're how they're supposed to be or
how we, like, if we take one as the bases and another one, they're a bit
rotate to the left, the the pooled feature will be exactly the same.
So you can see here, if we are talking about the cheetah's tears,
then let's say this is the four and this is where it was here.
Then if it was a bit rotated.
So for instance the four ended up over here.
Then when we're doing the pooling
we're still going to get the same pooled feature map.
And that's kind of the the principle behind it.
It's a very, rough explanation.
Again, intuitive explanation, but that's the point of pooling
that we're still being able to preserve the features.
And moreover, accounts for, their possible
spatial or textural or other kind of distortions.
And in addition to all of that, we are reducing the size.
So there's another benefit.
So we've got we're preserving the features.
We're introducing spatial invariance.
We're reducing the size by 75%,
which is huge, which is really going to help us in terms of processing.
And moreover, another benefit of pooling is
we are reducing the number of parameters.
So we're reducing again by 75%.
We're reducing the number of parameters
that are going to go into our final layers of the neural network.
And therefore we're preventing overfitting.
It is a very important benefit of pooling that we're removing information.
And that is a good thing.
That is a good thing because that way,
our model won't be able to overfit
onto that information because especially because that information is not real.
And remember, like at the very start we were talking about even for humans,
us as humans, it's important to see exactly the features
rather than all this other noise that is coming into our eyes.
Well, same thing for neural networks.
They by disregarding the unnecessary,
not important information we're helping with preventing of overfitting.
So there we go. That is what pooling is about.
And the question here is, of course, why why max pooling.
Right. There's lots of different types of pooling.
And you know why why a stride of two?
Why a size of two by two pixels.
Lots of all these things.
And on that note, I'd like to introduce you to this,
a lovely research paper called Evaluation of Pooling Operations
in Convolutional Architectures for Object Recognition
by Dominic Scherrer from University of Bonn.
There's the link.
And the beauty about this paper is that it's very,
very simple, very straightforward.
So if you've never read a research paper
before which you'd like to give it a go, this is a great place to start.
It's very short, only ten pages, very easy to read.
And plus the extra benefit is that now that we've discussed convolution
and pooling, you will be totally comfortable
with everything that they're talking about in this paper.
And you, this is a great way to actually reinforce you knowledge.
So I highly recommend checking this paper out.
I will take 20 minutes to read it.
And you can even skip part two, which is called related work
if it feels a bit farfetched or alienating, just don't read that part.
Go straight to from part one to part three.
And the one thing that you do need to know about this paper.
They talk about a concept called subsampling,
while subsampling is basically average pooling.
So remember how here we were taking, we were taking the maximum.
So you know square we're taking the maximum value.
There's a concept called mean pooling or some pooling.
Some pooling is you just sum these values up.
Average pooling or mean pooling.
You take the average value out of all of these.
And subsampling is kind of like a generalization of mean pooling.
It's it's a more kind of generalized approach
to taking the average of, of these values.
And you can read a bit more about it in the paper.
But otherwise, just think of it as average pooling when you're reading that paper.
And so that's where you can get some additional information on this topic.
And now kind of let's recap where have we got into it.
So there's our input image.
Then we applied the convolution operation and we got the convolution layer.
And now to each of those feature maps
that we get we've applied the pooling layer.
So we've got, we've done these two steps convolution and pooling.
And now we're going to do something very fun, something exciting.
We're going to, experiment with this.
So this is a screenshot I took from a, tool
created by Adam Harley from,
well, back when he was at Ryerson University of Computer Science,
and now he's at Carnegie Mellon, I think, doing his PhD and great tool.
So let's open up.
Let's have a look so you can find it.
You can't actually find it through Google.
You have to know the URL.
It's it's it's just hard to find this on Google because there's no text here.
See we're just this URL x dot Ryerson okay.
And then this stuff on then and basically this
is exactly what we're doing but visualized.
So here you need to draw a number.
So let's say I draw number four.
And this tool will put the number four here.
That's your image in our first step.
Then this is the convolution step right.
And this is the pooling step.
And also pooling by the way is also called downsampling.
So pooling and downsampling are the same things.
So you can see it's applied convolution.
Then it's applied pooling.
And you can see how it exactly works.
So you can see what kind of convolutions that it has applied or
what kind of filters it applied, what they look like.
You can see what features it's looking out for.
and then it's applying pooling.
So it's reducing the size.
And you can see here that this is important right.
So you can see, that this is the convolved image
and this is the pooled image.
And you can still see the same features.
It's just lesson information, but same features, right?
The features are preserved.
That's the important part.
and moreover, if you know, if all four was a bit to the kind of like rotated
a bit to the side, it would still be able to pick up very similar pooled layers.
And then after that it's got more layers.
We haven't talked about that yet.
So then it's got another convolutional, convolution
layer here, which, we actually won't have.
and then it has another pool layer, but it's basically just repeating that
same process.
And then after that,
this is what we're going to be talking further down in the course,
is got the fully connected layers and so on.
But you can definitely play around with that.
So if I delete that, you like if I draw a seven
you will see that, it actually tells you the guesses.
It guesses that this is a seven.
And the second guess the second likelihood is a three.
So you can draw it some some challenging things and see if it can pick them up.
So let's say if I draw something that looks like a zero
but it's not a finished zero, will it pick it up
now this this time didn't pick it up.
Looks like a nine to it to the image.
What if I kind of like finished like that?
So now it thinks it's a zero or a nine
and you can see over there what's lighting up the zero or the nine.
But we'll talk about that part for the dog.
Let's do one more.
Let's say like like eight.
I think it's a pretty hard for this now.
Picked up an eight.
So you can see that goes into an eight and then like after that
it stops being recognizable.
The subs making sense to us humans, right?
These, features that it's working with.
But at the same time, it is correctly recognizing that it's an eight.
Yeah. So definitely play around with that.
You can draw a smiley face, see what happens then.
Looks like a three to this, to this tool
because the tool is obviously trained up only on digits from 0 to 9.
So it has to recognize something.
There are of those.
And it recognizes a three.
It's like in life when you when you see something like a a type of fruit
that you've never seen before, like a, custard apple or something.
And you think that it's, like it's, it's a pear
because you've never actually seen one before.
You don't know what to classify it as.
Same thing here.
So it hasn't actually trained on the smiley faces.
And that's why it's thinks it's a tree. It's a three.
So there you go. It's a very powerful, powerful tool.
It'll be helpful for you to play around with.
It actually when you put your mouse over a pixel, it pixels, it shows you,
where the, feature detector was to pick up that pixel
so you can see where those, this pixel is coming from.
And, also so you can see how the filter was
kind of like going through the image, exactly how we talked about in the course.
And here you can see,
you can see the, the pooling, you can see that the pooling is done with,
the pooling is done with a,
little square size of two by two.
And you can see that it's, it's a stride of two as well.
Just as we discussed in, today's tutorial.
So there you go.
Have a play around with that.
And I hope you enjoyed today's, session.
I look forward to seeing you next time.
And until then, enjoy deep learning.