1
00:00:01,020 --> 00:00:01,380
Hello.

2
00:00:01,410 --> 00:00:02,380
Welcome back.

3
00:00:02,400 --> 00:00:06,390
In this lesson where are you going to build our multilayered neural network.

4
00:00:06,390 --> 00:00:10,130
I'm going to open our mortal layer on the score and in script

5
00:00:13,680 --> 00:00:17,360
Okay so this is simply a copy of our Trulia.

6
00:00:17,450 --> 00:00:25,650
And then we are going to call the new functions we created in the previous lesson over here.

7
00:00:26,240 --> 00:00:28,250
So we don't have to change much.

8
00:00:29,250 --> 00:00:39,740
What we may need to change is uh we need to call our new parameter initialization function and object.

9
00:00:39,740 --> 00:00:41,130
Oh okay.

10
00:00:41,150 --> 00:00:43,070
I'm simply going to create a new function.

11
00:00:43,070 --> 00:00:45,940
Course this is our model to layer and then model.

12
00:00:46,370 --> 00:00:51,140
So I'm going to create a new function known as multilayer and then model and in that function I'm going

13
00:00:51,140 --> 00:00:58,150
to call a 1 new initialize parameters multi-layer function and the other new functions we created.

14
00:00:58,280 --> 00:01:00,230
Um.

15
00:01:01,160 --> 00:01:02,660
So let's see let's see.

16
00:01:02,690 --> 00:01:08,450
I'm going to in order to simplify what we have over here.

17
00:01:09,020 --> 00:01:10,530
Some of these things will need them.

18
00:01:10,550 --> 00:01:19,380
But I see them when I put this new function here I'm going to create a new function here actually.

19
00:01:19,810 --> 00:01:20,120
Yeah.

20
00:01:20,120 --> 00:01:21,390
Being a bit indecisive.

21
00:01:21,410 --> 00:01:25,550
I'm just going to comment out the things that have to do with two layers.

22
00:01:25,640 --> 00:01:28,480
Once we are done we just clean this from this script.

23
00:01:28,490 --> 00:01:31,200
So comment this out right.

24
00:01:32,030 --> 00:01:33,990
So there's the layer dimension.

25
00:01:34,010 --> 00:01:35,780
We created this for the layer dimensions.

26
00:01:36,460 --> 00:01:38,200
Okay.

27
00:01:38,450 --> 00:01:42,140
This also we can clean this.

28
00:01:42,260 --> 00:01:42,620
Okay.

29
00:01:42,650 --> 00:01:46,180
So now we just have to read the data.

30
00:01:46,220 --> 00:01:51,000
We run our pre processing including our normalization.

31
00:01:51,050 --> 00:01:53,050
And yeah.

32
00:01:53,080 --> 00:01:57,380
And then down here we check our wrong predictions and other things.

33
00:01:57,380 --> 00:02:01,750
So I'm gonna come here and I'm gonna create a new function.

34
00:02:01,790 --> 00:02:06,880
This function was called to layer in and model.

35
00:02:07,250 --> 00:02:08,450
I'll call this 1 def

36
00:02:12,970 --> 00:02:17,960
multi layer and then model right.

37
00:02:18,000 --> 00:02:27,250
And this function is going to take our training set the Y labels.

38
00:02:29,410 --> 00:02:30,730
The layer dimensions

39
00:02:35,610 --> 00:02:36,570
lending rate

40
00:02:41,040 --> 00:02:46,750
although we can initialize the learning rate to to um what did we have before.

41
00:02:46,910 --> 00:02:48,000
Um yes.

42
00:02:49,140 --> 00:02:55,140
I'm gonna take the linen rate that we'll use for the two layer as well as the number of into uh number

43
00:02:55,140 --> 00:02:57,000
of iterations.

44
00:02:57,000 --> 00:02:58,330
The default values.

45
00:02:58,350 --> 00:03:09,760
I'm gonna paste them here like this.

46
00:03:10,670 --> 00:03:17,880
Okay so this is going to be our new function.

47
00:03:20,060 --> 00:03:20,370
Okay.

48
00:03:20,390 --> 00:03:26,840
So let's start off by seeding our random number generator and Peter at random um we actually doing this

49
00:03:26,840 --> 00:03:35,150
twice because when we call our parameter initialization function it cedes this so we can skip over this

50
00:03:36,080 --> 00:03:46,790
one to create a create a list to hold costs like we did initially and then I'm going to call our function

51
00:03:47,120 --> 00:03:52,160
a new function just the feed two of our library

52
00:03:59,700 --> 00:04:00,330
okay.

53
00:04:00,390 --> 00:04:01,160
Where's the function.

54
00:04:01,160 --> 00:04:12,630
This is a initialize parameters multilayer I'll copy this and then I'm gonna call this function here.

55
00:04:13,480 --> 00:04:16,830
And um let's declare the dimensions here.

56
00:04:16,850 --> 00:04:20,330
I'm gonna have a list here layout dimensions

57
00:04:28,290 --> 00:04:34,890
I hate using the same name for the argument and the variable of Python is so flexible so low dimensions

58
00:04:34,890 --> 00:04:35,130
here.

59
00:04:35,160 --> 00:04:37,590
Let's say we have we want to finally on your network.

60
00:04:37,590 --> 00:04:40,970
We know the the size of our input image.

61
00:04:41,130 --> 00:04:49,770
Um as we computed earlier it's sixty four multiplied by 64 multiplied by three.

62
00:04:49,770 --> 00:04:57,660
That is the inputs there and this gives us what this gives us twelve thousand two hundred and eighty

63
00:04:57,660 --> 00:04:58,950
eight.

64
00:04:58,950 --> 00:05:04,740
So this is going to be the number of notes for the input layer twelve thousand two hundred and twenty

65
00:05:04,740 --> 00:05:05,940
eight.

66
00:05:05,940 --> 00:05:07,220
Let's see.

67
00:05:07,590 --> 00:05:08,160
Right.

68
00:05:12,030 --> 00:05:20,280
So we're going to have twelve thousand from zero to twelve thousand two hundred and eighty seven.

69
00:05:20,280 --> 00:05:23,360
So if we add zero we have twelve thousand two hundred and eighty eight.

70
00:05:23,370 --> 00:05:26,700
That's going to be the input layer but this is not our neural network.

71
00:05:26,700 --> 00:05:28,270
This is the previous one.

72
00:05:28,560 --> 00:05:34,740
This one here is the they are one in currently the one we are creating is gonna have any number of layers

73
00:05:34,740 --> 00:05:35,820
we want.

74
00:05:35,820 --> 00:05:39,660
So this is just to show you just the the number of nodes for the input.

75
00:05:39,960 --> 00:05:44,160
We'll get the first aid in layer less if we wanted to have 20 nodes the one after that let's say we

76
00:05:44,160 --> 00:05:49,900
wanted to have eight nodes to one after that let's say we wanted to have sex nodes and the last one.

77
00:05:49,910 --> 00:05:53,850
1 so we have 1 2 3 4 5.

78
00:05:54,030 --> 00:06:01,350
So you could say this this is if we want to count the other ones which we should experiment with.

79
00:06:02,230 --> 00:06:09,540
So anyway our function Initialize parameters multilayer it's extra layer dimensions as its argument

80
00:06:10,290 --> 00:06:21,600
the teams pass this over here and it returns it's returns at Pictionary court parameters which the weight

81
00:06:21,600 --> 00:06:23,680
and bias values.

82
00:06:23,680 --> 00:06:30,180
So I'm gonna store the return in this dictionary we call parameters once that is done I can start the

83
00:06:30,180 --> 00:06:33,730
loop for gradient descent come over here.

84
00:06:35,140 --> 00:06:37,050
So for I in range

85
00:06:41,590 --> 00:06:46,810
starting from zero to the argument that you so pass known as number of iterations

86
00:06:49,510 --> 00:06:54,010
and then we going to call our forward propagation

87
00:06:58,750 --> 00:07:03,350
we performed for prop somewhere.

88
00:07:04,960 --> 00:07:11,010
Um yeah let's go to our library we have a function for the

89
00:07:17,250 --> 00:07:18,120
Where's the library.

90
00:07:18,120 --> 00:07:19,650
This is a multilayer and then

91
00:07:24,480 --> 00:07:32,070
we have a function somewhere module forward prop it takes two arguments The X which is the training

92
00:07:32,070 --> 00:07:39,810
input and the parameters and when I copy this and it returns the E L and the cache to Ellis like the

93
00:07:39,810 --> 00:07:48,510
Y hot activation of the final layer and the cache is just the things that we store Um so I'll copy this

94
00:07:48,510 --> 00:07:51,690
function and I'm going to pass X over here

95
00:07:54,960 --> 00:08:06,060
onto parameters over here and um I'm going to store the return in a variable call E L and another one

96
00:08:06,060 --> 00:08:07,050
called caches

97
00:08:11,360 --> 00:08:13,350
right.

98
00:08:13,430 --> 00:08:20,270
Okay so once that's done we're going to compute the cost or simply call our computer cos the function

99
00:08:23,970 --> 00:08:31,200
in this function takes y hat and y e l which is activation of the final layer is that what y hat and

100
00:08:31,200 --> 00:08:39,240
we already have Y which is the labels of the training set going to store the results in a variable called

101
00:08:39,240 --> 00:08:40,080
cost.

102
00:08:40,080 --> 00:08:41,030
This single cost.

103
00:08:41,760 --> 00:08:50,400
We have a list or on our record costs we shall append each single cost to these to this to create a

104
00:08:50,400 --> 00:08:52,060
list of costs.

105
00:08:52,140 --> 00:08:53,430
Right.

106
00:08:54,080 --> 00:08:54,530
Okay.

107
00:08:54,570 --> 00:09:07,450
So once that is done we can call our newly minted function the backward propagation function as computers.

108
00:09:09,590 --> 00:09:12,920
So does the function model.

109
00:09:13,010 --> 00:09:25,010
But it takes the e l y in the cache is going to copy this and I'm gonna come back here pasted over here

110
00:09:26,990 --> 00:09:34,520
and I'm going to pass a L and then Y and then caches.

111
00:09:34,790 --> 00:09:36,920
And this is going to return gradients.

112
00:09:36,920 --> 00:09:42,770
I'm going to store it in a variable of a similar name gradient.

113
00:09:42,980 --> 00:09:43,390
Right.

114
00:09:43,460 --> 00:09:46,120
So after this we would have performed back propagation.

115
00:09:47,090 --> 00:09:52,280
Once this is done we need to update our parameters.

116
00:09:52,280 --> 00:09:56,330
So we simply call or update the parameters function.

117
00:09:56,330 --> 00:09:57,280
Let's take a look at it

118
00:10:00,640 --> 00:10:05,070
update parameters okay.

119
00:10:05,090 --> 00:10:08,490
So it takes parameters gradients and the lending rate.

120
00:10:08,490 --> 00:10:11,160
And then it just runs through the loop.

121
00:10:11,160 --> 00:10:12,390
Okay I'm gonna copy this

122
00:10:16,020 --> 00:10:19,220
put it over here.

123
00:10:19,310 --> 00:10:24,270
The first document is parameters.

124
00:10:24,450 --> 00:10:26,220
The second document gradient

125
00:10:29,470 --> 00:10:33,980
the third argument is the lending rate that the user passes to the function.

126
00:10:34,170 --> 00:10:41,230
There's a typo here and um this show return to New parameters.

127
00:10:41,230 --> 00:10:49,960
So I should store it in the variable of a similar name parameters like this right.

128
00:10:50,050 --> 00:10:56,010
And then we going to print the cost for every 100 iterations.

129
00:10:56,260 --> 00:10:58,430
We can simply copy what we wrote to do the.

130
00:10:58,510 --> 00:11:00,450
And then append it.

131
00:11:00,520 --> 00:11:00,790
Okay.

132
00:11:00,820 --> 00:11:03,980
So this is from our previous function at a function for the two.

133
00:11:04,030 --> 00:11:07,330
I'm going to copy this and then bring it back here and then

134
00:11:14,200 --> 00:11:15,640
take away the comment.

135
00:11:18,190 --> 00:11:21,680
Okay.

136
00:11:21,820 --> 00:11:23,750
Right.

137
00:11:23,920 --> 00:11:24,370
Okay.

138
00:11:24,430 --> 00:11:30,820
So once this is done we can print out the um the cost and return the parameters.

139
00:11:30,820 --> 00:11:32,990
This the same as what we wrote for the 2 layer.

140
00:11:33,400 --> 00:11:34,780
So I'm gonna copy this as well.

141
00:11:46,680 --> 00:11:52,340
And I'm going to come over here on comment.

142
00:11:54,810 --> 00:11:55,380
Right.

143
00:11:55,440 --> 00:11:59,400
And this we have to make sure it's not in the for loop.

144
00:11:59,400 --> 00:12:00,100
It's not.

145
00:12:00,750 --> 00:12:02,470
Okay.

146
00:12:02,910 --> 00:12:04,880
So this is how a model.

147
00:12:05,190 --> 00:12:09,300
Now we can call it a model and see what we have

148
00:12:13,880 --> 00:12:14,980
so how.

149
00:12:14,990 --> 00:12:18,590
We test out this model in the next lesson.

150
00:12:18,980 --> 00:12:19,210
Yeah.

151
00:12:19,910 --> 00:12:21,620
Um but before we do that.

152
00:12:21,800 --> 00:12:22,210
Okay.

153
00:12:22,280 --> 00:12:26,150
I wanted to clean this up a bit but we can clean it up once we've tested it.

154
00:12:26,720 --> 00:12:28,000
Okay I'll see you in the next list.