In the previous lesson,
we saw how to do transfer learning. But when we retrain the inception
classifier features for cats versus dogs, we ended up over-fitting again. We also had augmentation, but despite
that, we still suffered from over-fitting. So let's discuss some ways that
we can avoid that in this lesson. Now here's the accuracy of our
training set versus our validation set over 100 epochs. It's not very healthy. There's another layer take
in Keras called a dropout. And the idea behind the dropout is that
layers in a neural network can sometimes end up having similar weights and possible
impact each other leading to over-fitting. With a big complex model like this,
that's a risk. So if you can imagine the dense layers
can look a little bit like this. By dropping some out,
we make it look like this. And that has the effect of neighbors
not affecting each other too much and potentially removing overfitting. So how do we achieve this in code? Well, here's our model
definition from earlier. And here's where we add the dropout. The parameter is between 0 and 1 and
it's the fraction of units to drop. In this case,
we're dropping out 20% of our neurons. For comparison, here's the chart
of training against accuracy from before the dropout was added. When you see validation diverging away
from the training like this over time, it's actually a great candidate
to try using a dropout. And here's the impact of the dropout. You can see that it's very significant. So lets take a look at
this in the workbook.