1
00:00:00,350 --> 00:00:03,570
In the previous lesson,
we saw how to do transfer learning.

2
00:00:03,570 --> 00:00:07,720
But when we retrain the inception
classifier features for cats versus dogs,

3
00:00:07,720 --> 00:00:09,970
we ended up over-fitting again.

4
00:00:09,970 --> 00:00:15,530
We also had augmentation, but despite
that, we still suffered from over-fitting.

5
00:00:15,530 --> 00:00:18,845
So let's discuss some ways that
we can avoid that in this lesson.

6
00:00:18,845 --> 00:00:23,300
Now here's the accuracy of our
training set versus our validation set

7
00:00:23,300 --> 00:00:24,650
over 100 epochs.

8
00:00:24,650 --> 00:00:25,550
It's not very healthy.

9
00:00:26,570 --> 00:00:29,980
There's another layer take
in Keras called a dropout.

10
00:00:29,980 --> 00:00:34,680
And the idea behind the dropout is that
layers in a neural network can sometimes

11
00:00:34,680 --> 00:00:40,430
end up having similar weights and possible
impact each other leading to over-fitting.

12
00:00:40,430 --> 00:00:42,690
With a big complex model like this,
that's a risk.

13
00:00:42,690 --> 00:00:46,770
So if you can imagine the dense layers
can look a little bit like this.

14
00:00:47,800 --> 00:00:51,090
By dropping some out,
we make it look like this.

15
00:00:51,090 --> 00:00:55,390
And that has the effect of neighbors
not affecting each other too much and

16
00:00:55,390 --> 00:00:57,110
potentially removing overfitting.

17
00:00:58,140 --> 00:01:00,090
So how do we achieve this in code?

18
00:01:00,090 --> 00:01:02,590
Well, here's our model
definition from earlier.

19
00:01:02,590 --> 00:01:04,530
And here's where we add the dropout.

20
00:01:04,530 --> 00:01:08,620
The parameter is between 0 and 1 and
it's the fraction of units to drop.

21
00:01:09,630 --> 00:01:13,480
In this case,
we're dropping out 20% of our neurons.

22
00:01:13,480 --> 00:01:17,210
For comparison, here's the chart
of training against accuracy

23
00:01:17,210 --> 00:01:19,580
from before the dropout was added.

24
00:01:19,580 --> 00:01:23,840
When you see validation diverging away
from the training like this over time,

25
00:01:23,840 --> 00:01:26,240
it's actually a great candidate
to try using a dropout.

26
00:01:27,270 --> 00:01:29,620
And here's the impact of the dropout.

27
00:01:29,620 --> 00:01:31,270
You can see that it's very significant.

28
00:01:32,900 --> 00:01:34,360
So lets take a look at
this in the workbook.