1
00:00:01,340 --> 00:00:09,260
‫In the last couple of videos, we learn how to create neural network with keras sequential api

2
00:00:10,760 --> 00:00:16,790
‫the sequential Api was quite easy to use, but it has some limitation.

3
00:00:19,010 --> 00:00:26,840
‫With sequential api, you can only create neural network with simply sequential architecture.

4
00:00:28,130 --> 00:00:37,070
‫You cannot create complex topologies like this where you have multiple inputs or multiple outputs.

5
00:00:38,180 --> 00:00:40,910
‫For that, you have to use functional api.

6
00:00:42,350 --> 00:00:43,370
‫in functional api.

7
00:00:44,150 --> 00:00:48,530
‫We create each of these layers in the form of functions.

8
00:00:49,270 --> 00:00:52,520
‫Or you can say a building block of your neural network.

9
00:00:53,330 --> 00:00:57,930
‫And you can use this functions to create a complex structure.

10
00:00:58,250 --> 00:01:01,520
‫By joining them according to your structure need.

11
00:01:03,600 --> 00:01:12,230
‫Before going into details, let's first delete the model that we created in our last lecture just

12
00:01:12,230 --> 00:01:12,750
‫just write del

13
00:01:12,960 --> 00:01:14,150
‫And your model name.

14
00:01:14,700 --> 00:01:19,100
‫And then you also need to clear the session of your keras.

15
00:01:19,740 --> 00:01:23,700
‫This will free up the resources for our next model training.

16
00:01:25,680 --> 00:01:27,710
‫So just write keras.backend

17
00:01:28,050 --> 00:01:29,400
‫dot clear session.

18
00:01:32,470 --> 00:01:37,350
‫run this, too, now we are ready to create and train our next model.

19
00:01:39,780 --> 00:01:43,020
‫For functional API, we will be using this example.

20
00:01:44,540 --> 00:01:53,750
‫This kind of neural network is known as wide and deep neural network deep because our input is going

21
00:01:53,750 --> 00:02:01,490
‫through two layer of dense hidden layer and wide because of what input is directly going for

22
00:02:01,580 --> 00:02:02,630
‫output as well.

23
00:02:04,040 --> 00:02:13,240
‫So along with the output of this hidden layers, it also connect all parts of our input directly

24
00:02:13,260 --> 00:02:14,360
‫to the output layer.

25
00:02:17,050 --> 00:02:24,740
‫This linkage, which I have marked as white, is not possible when we are using sequential api.

26
00:02:27,670 --> 00:02:35,050
‫The advantage of this architecture is that it makes possible for a neural network to learn both the

27
00:02:35,050 --> 00:02:36,030
‫deep patterns.

28
00:02:36,490 --> 00:02:42,490
‫by this deep linkage and the simple rules by this wide linkage.

29
00:02:45,040 --> 00:02:53,710
‫In our regular MLP, more than all the data flows through this full stacks of dense layers and thus

30
00:02:54,010 --> 00:03:00,460
‫some simple patterns in the data may end up being distorted by the sequence of this transformation.

31
00:03:03,800 --> 00:03:07,300
‫So now let's create this layers one by one.

32
00:03:08,870 --> 00:03:14,330
‫We will be using the same data set as we use for our last lecture.

33
00:03:18,000 --> 00:03:20,780
‫First, we need to create an input layer.

34
00:03:21,270 --> 00:03:30,510
‫We are calling it input and we are creating it with keras.layers, dot input, and then we have

35
00:03:30,510 --> 00:03:31,490
‫to provide the ship.

36
00:03:32,250 --> 00:03:32,940
‫We can either.

37
00:03:32,950 --> 00:03:33,360
‫write.

38
00:03:34,620 --> 00:03:40,830
‫Just 8 in brackets because we have eight independent variables or we can write it in this way.

39
00:03:41,130 --> 00:03:45,630
‫x train dot shape and then the first and so on attribute.

40
00:03:46,230 --> 00:03:47,760
‫So this is our input layer.

41
00:03:48,480 --> 00:03:51,700
‫We will create this kind of layers.

42
00:03:51,930 --> 00:03:55,680
‫And then we will connect these layers accordingly.

43
00:03:56,760 --> 00:04:04,200
‫So next we create a dense layer with 30 neurons using a RELU activation.

44
00:04:05,970 --> 00:04:07,770
‫And you can also notice that.

45
00:04:08,790 --> 00:04:12,210
‫We are calling this input player like a function.

46
00:04:12,990 --> 00:04:16,020
‫So this input layer is a input for this hidden layer one.

47
00:04:16,620 --> 00:04:18,600
‫And we are calling it like a function.

48
00:04:19,110 --> 00:04:22,250
‫And that's why we call it, functional api.

49
00:04:22,680 --> 00:04:31,020
‫We create this kind of layers and we use this layers as functions for our next players.

50
00:04:33,330 --> 00:04:37,040
‫So we have connected our input layer to our hidden layer 2

51
00:04:39,560 --> 00:04:41,670
‫Now we create our second hidden layer.

52
00:04:43,700 --> 00:04:50,690
‫Again, we are using Keras dot layers dot dense, and we are creating it with 13 neurons and activation

53
00:04:50,690 --> 00:04:51,440
‫is relu.

54
00:04:52,280 --> 00:04:57,380
‫And now for this hidden layer two our input should be hidden layer one.

55
00:04:57,710 --> 00:05:01,250
‫So we are passing our hidden layer one as a function here.

56
00:05:02,990 --> 00:05:05,480
‫Now, the next step is this.

57
00:05:05,900 --> 00:05:09,410
‫Here we want the output of this hidden layer 2.

58
00:05:10,130 --> 00:05:13,590
‫And also we want all our inputs variable here.

59
00:05:14,270 --> 00:05:15,560
‫You can see this linkage.

60
00:05:15,770 --> 00:05:19,330
‫We want these two to go into this concat layer.

61
00:05:20,860 --> 00:05:26,560
‫Concat layer is just merging the output of this hidden layer and all the inputs.

62
00:05:29,760 --> 00:05:37,240
‫So we can write this layer as concat and then we'll use keras dot layer, dot concat.

63
00:05:39,570 --> 00:05:43,610
‫And we are passing the list of ourput of hidden layer two.

64
00:05:44,360 --> 00:05:45,430
‫input layer

65
00:05:47,210 --> 00:05:55,490
‫If we wanted output of hidden layer one as well, we can write hidden layer one here and it will add

66
00:05:55,580 --> 00:06:04,190
‫a linkage like this also, so you can see you can customize all this linkages very easily using a functional

67
00:06:04,190 --> 00:06:04,670
‫api

68
00:06:05,920 --> 00:06:12,720
‫Now next step is to create our output layer, our output should get input from the concat layer.

69
00:06:13,570 --> 00:06:15,280
‫And this should be a single neuron.

70
00:06:15,850 --> 00:06:21,260
‫So we are creating output equal to keras dot layers dot dense.

71
00:06:22,060 --> 00:06:29,340
‫And then a single neuron without an activation function and input as the output of contact layer.

72
00:06:29,680 --> 00:06:32,390
‫So we are passing concrete layer as a function here.

73
00:06:34,470 --> 00:06:36,470
‫Now, we have created all the layers.

74
00:06:37,410 --> 00:06:41,700
‫The next step is to combine all this layers and create a model.

75
00:06:44,100 --> 00:06:49,910
‫So we are creating a model object and then we are calling keras.models.model

76
00:06:50,460 --> 00:06:56,100
‫And here we are mentioning what we want as our input and what we want as our output.

77
00:06:56,730 --> 00:07:01,090
‫So our input is this first input layer and our output

78
00:07:01,320 --> 00:07:02,960
‫Is this last output layer

79
00:07:04,710 --> 00:07:12,800
‫Think sequential api we first create a model and then we create each layer, layer by layer, but

80
00:07:12,830 --> 00:07:13,910
‫in functional api

81
00:07:15,170 --> 00:07:21,350
‫We create this layers and then we join this layers to create this whole network.

82
00:07:22,750 --> 00:07:29,680
‫Just run this now again, just to look at the structure of the model that you have created.

83
00:07:29,950 --> 00:07:31,750
‫You can call dot summary.

84
00:07:33,070 --> 00:07:34,850
‫So our object name is model

85
00:07:35,380 --> 00:07:36,780
‫And we are calling dot summary method.

86
00:07:38,650 --> 00:07:40,960
‫You run this, you will get all the details.

87
00:07:41,440 --> 00:07:45,030
‫First, we have an input layer with eight input variables.

88
00:07:45,940 --> 00:07:50,530
‫Then we have a dense layer with 30 neurons than we hav our second dense

89
00:07:50,650 --> 00:07:53,140
‫layer again with 13 neurons.

90
00:07:54,430 --> 00:08:01,130
‫Then we have a concat layer where we are contacting the input of this second dense layer

91
00:08:01,600 --> 00:08:02,890
‫And our input layer.

92
00:08:03,790 --> 00:08:08,320
‫So you can see we have 30 plus 8, so 38 neurons here.

93
00:08:10,300 --> 00:08:15,110
‫And then we have our output where we have only 1 neuron.

94
00:08:16,540 --> 00:08:18,480
‫Now the next step is to compile.

95
00:08:19,480 --> 00:08:29,260
‫And just as in our previous lecture, we will be using mean squared error as lost function. Sgd

96
00:08:29,260 --> 00:08:39,580
‫learning rate of zero point zero zero one as our optimizer and mae or mean absolute error as our additional

97
00:08:39,760 --> 00:08:40,870
‫metrics to calculate.

98
00:08:42,760 --> 00:08:43,450
‫run this.

99
00:08:45,650 --> 00:08:51,410
‫Now, fitting the model is same. we will just write model.fit provide our training

100
00:08:51,550 --> 00:08:54,270
‫dataset, provide the number of epoch values

101
00:08:55,290 --> 00:09:02,350
‫And then the validation dataset, since last time we ran that regression model for 40 epochs.

102
00:09:04,080 --> 00:09:06,240
‫So changing it for 40 epochs

103
00:09:07,770 --> 00:09:11,730
‫So that's just been more than.

104
00:09:16,460 --> 00:09:22,910
‫Again, you will see the lost functions, validation loss and the value of metrices that you have given

105
00:09:23,090 --> 00:09:23,360
‫here.

106
00:09:26,460 --> 00:09:27,640
‫You can see the lost value.

107
00:09:27,660 --> 00:09:30,510
‫We are getting on our training set is zero point three six.

108
00:09:30,540 --> 00:09:35,410
‫This is the mse value and the validation loss is zero point three six three eight.

109
00:09:37,070 --> 00:09:40,780
‫Let's calculate the value on our test data.

110
00:09:44,000 --> 00:09:52,020
‫You can see the lost value here is zero point three zero six four earlier in our last case

111
00:09:56,560 --> 00:10:00,060
‫I guess we were getting loss as 0.2515

112
00:10:01,630 --> 00:10:10,350
‫So this model is not performing that good, but in some situations, this kind of deep and wide network

113
00:10:10,380 --> 00:10:13,450
‫performs better than a normal MLP

114
00:10:13,510 --> 00:10:14,170
‫mdoels

115
00:10:15,930 --> 00:10:21,640
‫In this case our normal MLP model is platforming better than this wide and deep network.

116
00:10:23,310 --> 00:10:26,390
‫Again, you have all the parameters available to you.

117
00:10:26,610 --> 00:10:33,090
‫So if you just write model underscored history.history, you will get all the lost values.

118
00:10:34,850 --> 00:10:43,640
‫And the mae value that we got during the training, and you can also plot this on a graph, just

119
00:10:43,640 --> 00:10:44,840
‫like we did earlier.

120
00:10:47,460 --> 00:10:56,370
‫So the important thing here is that our validation loss or validation and mae value is still decreasing.

121
00:10:56,880 --> 00:11:02,280
‫So there is a scope of further improvement in the accuracy of our model.

122
00:11:03,390 --> 00:11:05,280
‫So let's just run this.

123
00:11:05,470 --> 00:11:08,550
‫model for 40 more epoch as well.

124
00:11:20,320 --> 00:11:22,540
‫Now, let's calculate the.

125
00:11:23,420 --> 00:11:24,840
‫test value as well.

126
00:11:27,840 --> 00:11:32,730
‫You can see now the loss has decreased to zero point two six.

127
00:11:33,630 --> 00:11:36,270
‫Earlier, we will getting around zero point three.

128
00:11:37,260 --> 00:11:40,890
‫Now we have zero point two six as our loss value.

129
00:11:42,980 --> 00:11:50,190
‫You can also plot this graph again to see whether the model have converge this time or not.

130
00:11:54,220 --> 00:12:00,250
‫You can see this is almost a straight line, so we can see that model has converged.

131
00:12:00,940 --> 00:12:07,940
‫And the test value for our model and the mse value for our model is.

132
00:12:08,470 --> 00:12:10,880
‫Zero one two six one on the test set.

133
00:12:13,030 --> 00:12:18,730
‫This is somewhat similar to the accuracy we bought earlier with our normal MLP model.

134
00:12:20,050 --> 00:12:20,210
‫But.

135
00:12:21,100 --> 00:12:27,010
‫I would say that our normal MLP model was performing better than this deep and wide network.

136
00:12:29,170 --> 00:12:31,770
‫So that's all for this lecture in the next lecture.

137
00:12:31,800 --> 00:12:39,210
‫We will see how to save our model model and how to save check points at the end of each epochs.

138
00:12:39,840 --> 00:12:40,260
‫Thank you.