1
00:00:00,610 --> 00:00:00,980
Hello.

2
00:00:01,170 --> 00:00:01,870
Welcome back.

3
00:00:02,220 --> 00:00:11,370
So in the last listen we saw how to build a to a Europe network and then we used the functions we created

4
00:00:11,400 --> 00:00:18,300
in our DNA and that's called lip version one script to create on your own network.

5
00:00:18,330 --> 00:00:25,710
In this lesson we are going to create functions that would allow us to be able to use a neural network

6
00:00:25,710 --> 00:00:28,140
size that we want.

7
00:00:28,140 --> 00:00:30,880
The first one was just to neural networks.

8
00:00:30,900 --> 00:00:36,600
Now we're going to write a function that would accept the dimension of the of the layers such that you

9
00:00:36,600 --> 00:00:43,080
can have a deep neural network with three hidden layers for hidden layers five hidden layers as much

10
00:00:43,080 --> 00:00:43,850
as you want.

11
00:00:43,860 --> 00:00:50,310
We're going to create a function that allows us to automatically include multiple layers rather than

12
00:00:50,310 --> 00:00:52,470
fix it to two layers only.

13
00:00:52,530 --> 00:00:54,630
So I'm gonna make a copy of the last project.

14
00:00:54,630 --> 00:00:58,740
This one here I'll just copy and paste the photo here

15
00:01:03,570 --> 00:01:05,730
and I'm going to rename this.

16
00:01:06,870 --> 00:01:08,100
I'm going to rename this to

17
00:01:10,870 --> 00:01:11,940
multilayer and then

18
00:01:20,560 --> 00:01:21,830
right.

19
00:01:22,340 --> 00:01:23,500
Can gonna open it.

20
00:01:23,500 --> 00:01:26,110
I'm going to rename CNN leap.

21
00:01:26,170 --> 00:01:32,780
I'm going to change this to version 2 and then this over here to layer.

22
00:01:32,800 --> 00:01:34,870
I'm going to call it Morty here

23
00:01:39,850 --> 00:01:40,270
right.

24
00:01:40,270 --> 00:01:42,730
So let's start with our library.

25
00:01:42,970 --> 00:01:50,050
Going to right click right click open with I do.

26
00:01:50,050 --> 00:01:50,870
Right.

27
00:01:50,890 --> 00:01:52,920
So let's see over here.

28
00:01:52,920 --> 00:01:58,450
Now I initialize parameters function as fixed for this size.

29
00:01:58,450 --> 00:02:03,820
We have an input an input layer a hidden layer and then the output.

30
00:02:03,820 --> 00:02:06,890
Now we want to be able to have multiple layers.

31
00:02:06,940 --> 00:02:14,770
So what we're going to do is we're going to write a new function where you can simply pass the D layer

32
00:02:14,770 --> 00:02:21,210
dimensions and based on the argument you pass here it's going to determine it's going to initialized

33
00:02:21,210 --> 00:02:24,340
a number of layers you select for you automatically.

34
00:02:24,370 --> 00:02:31,190
So we would have to write a new function that would do that for us so I'm going to come up here and

35
00:02:31,190 --> 00:02:32,960
then I'm going to see def

36
00:02:38,010 --> 00:02:38,880
initialize

37
00:02:41,450 --> 00:02:44,110
initialize parameters so call it

38
00:02:53,050 --> 00:02:55,580
multilayer right.

39
00:02:55,760 --> 00:02:56,340
Okay.

40
00:02:56,410 --> 00:03:03,730
I think there's a very descriptive name and this function is going to take the estimations as I comment

41
00:03:10,820 --> 00:03:11,710
right.

42
00:03:11,730 --> 00:03:21,660
So we start off by I'll see to my random number generator and P random dot run in

43
00:03:25,370 --> 00:03:28,250
just to random the seed actually not run down.

44
00:03:28,780 --> 00:03:29,370
Okay.

45
00:03:29,380 --> 00:03:34,840
And then once that's done I'm going to create parameters here.

46
00:03:34,870 --> 00:03:38,110
I'm going to start the parameters in this dictionary when we are done.

47
00:03:38,500 --> 00:03:40,840
I'm going to just initialize it to be empty

48
00:03:45,000 --> 00:03:45,510
and then

49
00:03:48,540 --> 00:03:53,190
I'm going to get a number of layers by finding the length of this list.

50
00:03:53,190 --> 00:03:59,700
So this is going to be a list and in the list we're going to give the dimension of each layer and a

51
00:03:59,700 --> 00:04:01,500
dimension means the number of nodes.

52
00:04:01,830 --> 00:04:05,480
So if our input node has a thousand.

53
00:04:06,060 --> 00:04:10,680
If we have a thousand input we'll have a thousand and then the next layer.

54
00:04:10,710 --> 00:04:12,300
Let's say the first aid in there.

55
00:04:12,330 --> 00:04:15,030
If it has three nodes then it's gonna be three.

56
00:04:15,210 --> 00:04:19,420
And then if the one of two that has five nodes it's going to be five.

57
00:04:19,620 --> 00:04:21,270
So he's gonna be a thousand.

58
00:04:21,300 --> 00:04:21,920
Come on three.

59
00:04:21,930 --> 00:04:24,650
Come on five that's gonna be like the dimension.

60
00:04:24,720 --> 00:04:31,170
So just by taking the length the number of items in this list that we shall call list dimension we can

61
00:04:31,170 --> 00:04:35,050
know the total number of layers of the neural network.

62
00:04:35,250 --> 00:04:35,960
Right.

63
00:04:36,000 --> 00:04:43,440
So I'm going to create a variable capital L over here and I'll say this is equal to length of this argument

64
00:04:43,500 --> 00:04:47,070
which is going to be passed over here.

65
00:04:47,690 --> 00:04:48,550
Right.

66
00:04:48,570 --> 00:04:51,410
So once we've done this we can open a for loop.

67
00:04:51,420 --> 00:04:58,470
I'll say for l in range from 1 to Cup 2 L

68
00:05:01,960 --> 00:05:03,880
and I'm going to see parameters

69
00:05:12,040 --> 00:05:18,020
w plus S T R L

70
00:05:24,310 --> 00:05:34,470
E course then remember we generate in the m the weights so we use random and P2 at random dot round

71
00:05:34,750 --> 00:05:45,550
and decides is going to be the layout dimensions I read or list index index l by

72
00:05:48,970 --> 00:05:50,020
layout dimensions

73
00:05:57,400 --> 00:06:04,420
have to be careful with the name the spellings so the argument is called layout dimensions so they add

74
00:06:04,410 --> 00:06:08,440
dimensions over here the estimation is they add dimensions.

75
00:06:08,440 --> 00:06:10,510
Okay index they are dimensions

76
00:06:13,150 --> 00:06:14,620
L minus 1

77
00:06:20,440 --> 00:06:20,950
and

78
00:06:24,250 --> 00:06:27,640
I'm going to divide it by a square root

79
00:06:32,650 --> 00:06:35,560
of the previous layer.

80
00:06:35,800 --> 00:06:40,950
The previously estimations which is L minus one right.

81
00:06:41,270 --> 00:06:43,390
I'll reshape this a bit.

82
00:06:44,020 --> 00:06:52,840
So once this is done we can initialize the bias and we do parameters.

83
00:06:59,650 --> 00:07:03,040
We're going to give it a key of B like we did earlier

84
00:07:06,190 --> 00:07:09,160
plus SDR index L

85
00:07:12,040 --> 00:07:17,310
because we initialize the bias to 0.

86
00:07:17,310 --> 00:07:19,570
Source I'll do MP dot zeros

87
00:07:22,640 --> 00:07:28,640
and then the dimension is low dimension by 1

88
00:07:37,300 --> 00:07:38,320
right.

89
00:07:40,340 --> 00:07:43,540
So next we can simply return to parameters

90
00:07:55,390 --> 00:08:00,550
so return parameters here like this.

91
00:08:00,820 --> 00:08:08,410
Okay so we can just pass we can just pass a 20 layer neural network and this will be able to generate

92
00:08:08,440 --> 00:08:11,810
the weight and biases for us right.

93
00:08:11,950 --> 00:08:17,260
So does the first addition we've added to our library.

94
00:08:17,270 --> 00:08:21,200
Are we going to create.

95
00:08:21,490 --> 00:08:22,070
Let's see.

96
00:08:22,140 --> 00:08:24,310
Well we have everything we need.

97
00:08:24,620 --> 00:08:28,090
We we can reuse almost everything here.

98
00:08:29,600 --> 00:08:29,950
Okay.

99
00:08:29,980 --> 00:08:34,030
So over here we have a model forward prop.

100
00:08:34,420 --> 00:08:43,490
Okay let's write another function for back prop I'm going to come down here and write a function I say

101
00:08:43,520 --> 00:08:48,370
def model back prop

102
00:08:51,220 --> 00:08:55,900
so we going to use this to perform debug propagation

103
00:09:00,310 --> 00:09:10,210
and this is going to take it's gonna take three argument it's gonna take the M L which is the activation

104
00:09:10,210 --> 00:09:18,230
of the final layer which is the same as y hat y our labels y and then some caches.

105
00:09:18,430 --> 00:09:19,720
Yeah the cautious we pass.

106
00:09:20,380 --> 00:09:23,020
Okay so let's start off by

107
00:09:26,140 --> 00:09:28,830
I start off by creating some gradient.

108
00:09:28,950 --> 00:09:36,750
Create this list to hold a gradients and I'm going to get the length the number of layers from the um

109
00:09:37,290 --> 00:09:43,080
the length of the list that would hold a cautious associate length of cautious

110
00:09:49,210 --> 00:09:52,970
once that is done.

111
00:09:53,780 --> 00:10:08,990
I'm going to reshape y to be the same shape as a L or C Y equals Y dots reshape and reshaping it to

112
00:10:08,990 --> 00:10:10,270
the shape of it.

113
00:10:10,850 --> 00:10:16,000
In a sense we're making y the same as the shape of y hat.

114
00:10:16,070 --> 00:10:24,110
Now I'm going to compute d e l e l e course

115
00:10:29,710 --> 00:10:35,800
and P divide we divide in here.

116
00:10:35,860 --> 00:10:42,850
Y comma AOL and then minus and a minus here

117
00:10:45,780 --> 00:10:47,550
minus n Peter to divide

118
00:10:51,230 --> 00:10:57,850
1 minus Y comma one minus a l like this

119
00:11:02,980 --> 00:11:03,400
right.

120
00:11:03,730 --> 00:11:07,780
So I'm going to create a variable quote current cash

121
00:11:12,760 --> 00:11:16,090
and I'm going to grab elk minus one

122
00:11:19,730 --> 00:11:25,520
crushes elk minus one.

123
00:11:25,570 --> 00:11:36,480
And once that's done I'm going to um I'm going to set the content of the gradient.

124
00:11:36,470 --> 00:11:42,820
I'm going to sort of collect.

125
00:11:42,820 --> 00:11:50,950
I am simply going to call out linear propagation function and then collect the return value some put

126
00:11:50,950 --> 00:11:52,120
it into the gradient.

127
00:11:53,050 --> 00:11:58,660
So we created a function earlier known as linear activation Buchwald over here.

128
00:11:58,960 --> 00:12:05,260
I'm gonna call this and this function as you can see it returns the previous the W NDP.

129
00:12:05,290 --> 00:12:13,600
I'm going to store this in a while gradient dictionary here so I'll call the function.

130
00:12:13,960 --> 00:12:23,080
The function is going to take a loss argument that is the previous the a and then the current cash

131
00:12:32,110 --> 00:12:35,650
and then the activation we're going to use is the sigmoid

132
00:12:45,800 --> 00:12:46,780
okay.

133
00:12:47,230 --> 00:12:52,730
So I'm going to store the return in the dictionary.

134
00:12:53,300 --> 00:12:54,280
I'll come over here.

135
00:12:55,810 --> 00:13:00,770
It's going to return d w e and DP.

136
00:13:00,770 --> 00:13:03,640
So I'm gonna just see gradient

137
00:13:07,380 --> 00:13:20,550
D a comma D A plus cause if we're dealing with the iterator o append L over here and over here like

138
00:13:20,550 --> 00:13:23,820
this this the um the last layer

139
00:13:32,670 --> 00:13:43,200
and the next return value is the w the next return values the top abuse or simply do comma creates and

140
00:13:43,200 --> 00:13:50,040
then when I start this I'll give it a key w plus

141
00:13:53,500 --> 00:13:56,250
string L then the next one is DP

142
00:14:04,240 --> 00:14:13,930
DP plus.

143
00:14:14,330 --> 00:14:16,340
So this is what we lost.

144
00:14:17,360 --> 00:14:20,020
This is for the last layer capital L.

145
00:14:20,060 --> 00:14:28,190
Now we have to just use the we have to iterate to do all previous layers L minus 1 L minus 2 L minus

146
00:14:28,220 --> 00:14:29,810
3 etc..

147
00:14:30,200 --> 00:14:31,370
Right.

148
00:14:31,430 --> 00:14:35,550
And we simply call in our linear activation backward function.

149
00:14:35,570 --> 00:14:43,540
This function returns the B TWD a We've stored them in our gradient over here.

150
00:14:43,700 --> 00:14:48,420
So I'm going to create a loop to do the rest.

151
00:14:49,310 --> 00:14:53,450
Or C for l in reversed

152
00:14:59,210 --> 00:14:59,880
range

153
00:15:04,150 --> 00:15:07,150
L minus 1 like this.

154
00:15:11,350 --> 00:15:14,020
I'm going to do current cash of course L

155
00:15:17,570 --> 00:15:25,860
because cash is cautious it's going to be past us like you meant to the function to a cautious index

156
00:15:25,980 --> 00:15:33,050
l will be the current cash and then once that is done I'm going to call our function again

157
00:15:37,190 --> 00:15:45,230
reach call a function Lena activate DNA activation backwards.

158
00:15:45,320 --> 00:15:48,960
Copy this piece over here.

159
00:15:49,670 --> 00:15:55,900
The first document is to be gradients the a

160
00:16:00,800 --> 00:16:14,360
secret key to a plus SDR L plus two one after that

161
00:16:20,000 --> 00:16:21,430
the next argument.

162
00:16:21,530 --> 00:16:26,000
We have three arguments so we've got a positive cash.

163
00:16:26,000 --> 00:16:32,480
This is previous the E and then after that the next argument I see the function the next argument is

164
00:16:32,480 --> 00:16:42,080
the cash the third one is the activation type so the cash is current cash and current cash is taken

165
00:16:42,110 --> 00:16:48,980
the edge of the cash no l in our list of cash is so current cash

166
00:16:54,490 --> 00:16:54,920
and then

167
00:16:59,000 --> 00:17:01,520
activation equals random

168
00:17:04,990 --> 00:17:07,180
linear activation backwards.

169
00:17:07,180 --> 00:17:07,590
Okay.

170
00:17:08,980 --> 00:17:16,630
So we're going to return the um the return we're going to keep them in temporary variables I'm going

171
00:17:16,630 --> 00:17:36,220
to have a temp over here and then uh I'll call this the a previous temp and then t w w temp and then

172
00:17:36,220 --> 00:17:37,000
to be temp

173
00:17:41,460 --> 00:17:42,300
right.

174
00:17:42,360 --> 00:18:00,020
So now when I come over here and then I'll see gradients are dictionary D A plus SDR plus 1

175
00:18:02,990 --> 00:18:08,510
equals D a previous temp.

176
00:18:09,050 --> 00:18:10,990
You can see where this is going.

177
00:18:11,210 --> 00:18:14,660
I'm gonna do the same for DP but just copy and paste.

178
00:18:19,170 --> 00:18:25,170
I start with the W the W four here plus L plus 1.

179
00:18:25,170 --> 00:18:33,270
This is simply D W temp and then DP

180
00:18:36,400 --> 00:18:38,050
This is simply DP temp

181
00:18:43,230 --> 00:18:46,170
right and we're going to return to gradients

182
00:18:52,920 --> 00:18:53,620
okay.

183
00:18:54,050 --> 00:19:02,150
So these are the two new functions we need in order to be able to perform the same computation with

184
00:19:02,270 --> 00:19:06,430
multiple layers rather than just two layers.

185
00:19:06,500 --> 00:19:12,750
So when we go to our our other script we're going to call these two new functions.

186
00:19:13,070 --> 00:19:15,060
Right.

187
00:19:15,800 --> 00:19:22,960
So this is all there is for this this lesson and if you have any questions just send me a message I'm

188
00:19:22,980 --> 00:19:24,650
leaving the questions area.

189
00:19:24,950 --> 00:19:31,860
The next lesson we should develop our new neural network have in more than two layers.

190
00:19:31,860 --> 00:19:36,440
And yeah we shall call these two new functions that we've created.

191
00:19:36,500 --> 00:19:37,370
I'll see you later.

192
00:19:37,370 --> 00:19:38,130
Have a nice day.