1
00:00:00,270 --> 00:00:08,040
Now, as usual, we just copy this and then we could check out on this documentation for the significance

2
00:00:08,040 --> 00:00:10,200
of each and every one of these arguments.

3
00:00:10,350 --> 00:00:12,120
Here we have the file path.

4
00:00:12,120 --> 00:00:18,480
So this is the file is where we are going to be saving our model.

5
00:00:19,290 --> 00:00:24,480
So here we have our checkpoint called back and that's it.

6
00:00:24,480 --> 00:00:26,400
We have model checkpoint here.

7
00:00:26,400 --> 00:00:28,530
We're going to define this file path.

8
00:00:28,530 --> 00:00:30,510
And then what are we going to be monitoring?

9
00:00:30,510 --> 00:00:37,980
We're going to be monitoring the validation list, which is what is given to us by default, just like

10
00:00:37,980 --> 00:00:42,570
with the URL is stopping where we had this monitor right here.

11
00:00:42,570 --> 00:00:50,580
We are going to look at our validation laws and suppose we have this validation loss and then our training

12
00:00:50,580 --> 00:00:51,170
loss.

13
00:00:51,180 --> 00:00:58,230
So we're going to look at our validation loss, which is this, and then save the model weights when

14
00:00:58,230 --> 00:01:03,400
we have this minimal or the smallest validation loss right here.

15
00:01:03,420 --> 00:01:11,520
And then if you set the safe best only to true, then we'll save only these weights, whereas one is

16
00:01:11,520 --> 00:01:16,980
false, we'll save this and also save the latest weights.

17
00:01:17,640 --> 00:01:24,360
And then for this argument, save weights only if it's saved to if it's set to true, then we're going

18
00:01:24,360 --> 00:01:26,130
to save only the weights.

19
00:01:26,130 --> 00:01:32,190
Whereas if it's set to false, then we're going to be saving a model that has a full model with its

20
00:01:32,730 --> 00:01:33,870
parameters.

21
00:01:33,870 --> 00:01:40,680
Then we have the mode which is automatic, like since it's validation loss, since it's a loss automatically,

22
00:01:40,680 --> 00:01:42,090
this would be a mean.

23
00:01:42,090 --> 00:01:47,820
Whereas if this is an accuracy or precision, we're going to have a max right here.

24
00:01:48,120 --> 00:01:54,020
So let's take this back to auto and then we have validation loss.

25
00:01:54,030 --> 00:02:00,480
Now to say frequency when set to epoch simply means we're going to do this saving after each and every

26
00:02:00,480 --> 00:02:03,360
epoch that we could modify this to say three.

27
00:02:03,360 --> 00:02:11,640
So after three epochs we are going to verify if our current model is the best of all the previous models.

28
00:02:11,640 --> 00:02:13,770
That's if we've set this to true.

29
00:02:13,800 --> 00:02:19,230
So if we set this to true, we're going to verify if this model, the current model, is the best.

30
00:02:19,230 --> 00:02:25,860
If it's the best, then we override this file path that's overwrite the file, the weight file we've

31
00:02:25,860 --> 00:02:28,770
saved, and if it's not the best, we continue training.

32
00:02:29,160 --> 00:02:36,120
So that said, let's modify our file path and then let's call this check points.

33
00:02:36,120 --> 00:02:36,750
That's it.

34
00:02:36,750 --> 00:02:41,400
So we have this check points and then we run this.

35
00:02:41,400 --> 00:02:47,370
So we have that and then we add this callback right here.

36
00:02:47,370 --> 00:02:52,110
So we have the checkpoint, checkpoint callback.

37
00:02:52,730 --> 00:02:56,160
And let's find we run the training now and see what we get.

38
00:02:57,680 --> 00:03:03,770
After training for over ten epochs, we have this two locks, which you could notice here, this one

39
00:03:03,770 --> 00:03:10,730
where we've been told assets written to checkpoints, assets so you could open up this checkpoint here

40
00:03:10,730 --> 00:03:14,390
and then you'll see that we actually saving this model.

41
00:03:14,390 --> 00:03:17,290
And then we also have this locked here.

42
00:03:17,300 --> 00:03:19,430
That's after the CID epoch.

43
00:03:19,460 --> 00:03:25,550
Now, let's understand why this login is done after the first and also after the seed epoch.

44
00:03:26,850 --> 00:03:30,990
Just when training starts, we have a validation loss of 0.47.

45
00:03:30,990 --> 00:03:37,560
And so we supposing that since this is the first then the model such is best state, then it moves to

46
00:03:37,560 --> 00:03:38,970
one which is greater than this.

47
00:03:38,970 --> 00:03:46,710
So the model state or the best state is this one after the first epoch, this the model, the first

48
00:03:46,710 --> 00:03:49,920
epoch or the model state are the first epoch is to the best.

49
00:03:50,040 --> 00:03:51,750
This doesn't beat the record.

50
00:03:51,750 --> 00:03:52,950
This doesn't.

51
00:03:53,100 --> 00:04:00,030
And then you'll see that here we have this validation loss of 0.24.

52
00:04:00,120 --> 00:04:08,040
That is why the model is now saved from this model check point and section will look at the reduced

53
00:04:08,040 --> 00:04:14,880
learning rate on plateau and then later on we'll have a dedicated section for TensorFlow.

54
00:04:14,880 --> 00:04:23,160
But the way it is learning rate are rather reduce learning rate on plateau works is that if you start

55
00:04:23,160 --> 00:04:31,950
training with a fixed learning rate and then say you've trained for over 100 epochs and that the model

56
00:04:31,950 --> 00:04:35,760
performance after 110 epochs.

57
00:04:35,760 --> 00:04:38,910
So let's say we have 110 right here.

58
00:04:38,910 --> 00:04:43,980
So after this 110 epochs, our model model performance doesn't improve.

59
00:04:43,980 --> 00:04:49,080
So we still fix our learning rate, but since for the past.

60
00:04:49,940 --> 00:04:50,930
Ten epochs.

61
00:04:50,930 --> 00:04:53,590
We haven't had an improvement in the model's performance.

62
00:04:53,600 --> 00:04:59,780
What we'll do is we'll reduce the model's learning rate by a given factor.

63
00:04:59,780 --> 00:05:01,910
So here is the factor.

64
00:05:02,630 --> 00:05:07,700
The patients our patients here is 10 million that we're going to wait for ten epochs before deciding

65
00:05:07,700 --> 00:05:10,060
whether we're going to reduce the learning rate or not.

66
00:05:10,070 --> 00:05:14,990
The verbosity, as usual, the quantity to be monitored.

67
00:05:14,990 --> 00:05:20,210
In this case, the default is the validation loss, the mode automatic.

68
00:05:20,210 --> 00:05:26,090
So if you wanted to be mean, you could specify mean or max.

69
00:05:26,090 --> 00:05:31,670
But setting it to automatic allows TensorFlow to automatically decide whether it's a mean or max.

70
00:05:31,670 --> 00:05:37,700
In the case of validation loss, obviously it's going to be a mean since our aim is to reduce the validation

71
00:05:37,700 --> 00:05:39,140
loss as much as we can.

72
00:05:39,140 --> 00:05:46,040
Then we have the mean delta, which takes in a default value of 0.0001.

73
00:05:46,040 --> 00:05:54,200
And this means that if after say, ten epochs we have a change in the validation loss.

74
00:05:54,200 --> 00:06:02,270
But this change is less than this mean delta, then we'll consider that that isn't a change and hence

75
00:06:02,270 --> 00:06:06,650
we are going to still reduce the learning rate by this factor right here.

76
00:06:06,650 --> 00:06:12,710
We also have the cool down and then the minimum learning rate below, which we wouldn't we wouldn't

77
00:06:12,710 --> 00:06:16,610
want to drop our learning rate as described here.

78
00:06:16,610 --> 00:06:21,920
This cool down is a number of epochs to wait before resuming normal operation.

79
00:06:21,920 --> 00:06:24,200
After the learning rate has been reduced.

80
00:06:24,470 --> 00:06:30,500
We are going to create this callback very easily and this time around we're going to monitor the validation

81
00:06:30,500 --> 00:06:31,280
accuracy.

82
00:06:31,280 --> 00:06:38,960
So if the validation accuracy doesn't increase after two epochs, we are going to reduce the learning

83
00:06:38,960 --> 00:06:50,060
rate by a factor of 0.1, meaning that if we had a learning rate of, say, 0.01, and that this validation

84
00:06:50,090 --> 00:06:57,440
accuracy hasn't increased after the two epochs, we are going to reduce this such that this becomes

85
00:06:57,440 --> 00:07:06,650
0.01 times 0.1, which is this factor right here just as before.

86
00:07:06,650 --> 00:07:11,930
We are going to add this plateau callback and then train our model.

87
00:07:12,200 --> 00:07:20,240
After training for five epochs, we can observe that we have this learning rate which has been reduced

88
00:07:20,570 --> 00:07:22,370
after the third epoch.

89
00:07:22,370 --> 00:07:32,210
And this is because after this two epochs there is no improvement in this initial validation accuracy.

90
00:07:32,210 --> 00:07:39,680
And so that's why the accuracy is going to drop from 0.01 to 0.001.

91
00:07:40,250 --> 00:07:42,020
Now that's it for this section.

92
00:07:42,020 --> 00:07:45,650
Thank you for getting up to this point and see you next time.