1
00:00:00,700 --> 00:00:08,450
In previous session, we just learned the basics of Advancer script, and I explain some of these functions

2
00:00:08,460 --> 00:00:11,310
now let's create another events creep.

3
00:00:11,490 --> 00:00:19,150
This time I want to change the structure of the file and have a more control over neural network.

4
00:00:19,190 --> 00:00:23,400
So I'm going to select all copy them, open a new tab.

5
00:00:23,400 --> 00:00:26,310
Pace and Control is to save your file.

6
00:00:26,610 --> 00:00:29,700
This time I will call it advanced script number two.

7
00:00:31,140 --> 00:00:35,020
Now, let's start from the beginning and see what can we change here.

8
00:00:35,790 --> 00:00:40,260
The first thing that I want you to change here is hidden layer size.

9
00:00:40,890 --> 00:00:43,340
This is only for one hidden layer.

10
00:00:43,350 --> 00:00:50,550
But what if we want to design a network that has more hidden layers like Tree Hill and layers or forehead

11
00:00:50,550 --> 00:00:52,750
and layers in order to do that?

12
00:00:52,770 --> 00:00:58,020
I'm going to remove this tree and inside the bracket I will put one tree in space.

13
00:00:58,020 --> 00:01:02,460
And then for now, let's run your network and see what's the result.

14
00:01:03,550 --> 00:01:08,110
OK, here we can see the structure of our neural network change.

15
00:01:08,130 --> 00:01:09,450
We can see better here.

16
00:01:10,050 --> 00:01:12,990
We have hidden layer number one between neurons.

17
00:01:12,990 --> 00:01:15,930
This is number of neurons that I assign to it.

18
00:01:15,930 --> 00:01:20,520
And for that he didn't layer number two, we have four nerit.

19
00:01:21,000 --> 00:01:23,730
Of course, it has also an output layer.

20
00:01:24,600 --> 00:01:26,550
Now let's just check it here.

21
00:01:26,550 --> 00:01:28,110
We can have more.

22
00:01:28,260 --> 00:01:36,300
For example, if you space and put two here, then run it again, you will have a network with four

23
00:01:36,630 --> 00:01:40,140
different layers, hidden layer and one output layer.

24
00:01:40,500 --> 00:01:46,380
But there is a point here because I'm going to be getting this question of what would be the maximum

25
00:01:46,380 --> 00:01:49,650
number of layers, the hidden layers that we should use.

26
00:01:50,010 --> 00:01:57,720
It will depend on the structure of your data, but in reality, for optimization, for classification

27
00:01:57,720 --> 00:01:58,440
problems.

28
00:01:58,680 --> 00:02:06,960
If you cannot get a good result with maximum of three hidden layers, then don't try for adding an extra

29
00:02:06,960 --> 00:02:07,380
layer.

30
00:02:07,380 --> 00:02:08,540
It won't help you.

31
00:02:08,970 --> 00:02:15,450
So usually we will use only one hidden layer, maybe two or maximum three on this.

32
00:02:15,450 --> 00:02:19,880
That's a deep learning problem and then it's a different thing.

33
00:02:19,890 --> 00:02:26,380
But for optimization problem, for fitting tools, it's going to be enough to or tweet in the years.

34
00:02:26,400 --> 00:02:33,960
Now, let me just remove this one and stay with two hidden layers fitness.

35
00:02:34,080 --> 00:02:43,590
If we select fitness and press EF1 on your keyboard, you can have the help of Matlab with some explanation

36
00:02:43,590 --> 00:02:44,310
about these.

37
00:02:44,520 --> 00:02:46,560
Features it on my keyboard.

38
00:02:46,560 --> 00:02:51,810
I should press F and plus Evren to open this window.

39
00:02:53,510 --> 00:02:54,110
Fitness.

40
00:02:54,350 --> 00:02:57,420
It's a function for fitting neural network.

41
00:02:57,560 --> 00:03:03,290
This is the same tags it means we can use reduced to syntax in this example by default.

42
00:03:03,290 --> 00:03:10,550
It just used the second syntax, which we can define a number of hidden layers and also we can mention

43
00:03:10,560 --> 00:03:12,050
about the training function.

44
00:03:12,470 --> 00:03:16,820
But this has a limited options.

45
00:03:17,210 --> 00:03:20,450
And for example, we cannot change the activation function.

46
00:03:20,450 --> 00:03:26,720
Activation function, for this example is a sigmoid and we can change it here.

47
00:03:27,210 --> 00:03:32,930
Let me open a document for another fitting tools, which is called New F.

48
00:03:34,250 --> 00:03:35,450
Let's check it here.

49
00:03:36,140 --> 00:03:38,000
This is the help window.

50
00:03:38,570 --> 00:03:42,410
New F create a feed for word back propagation network.

51
00:03:42,830 --> 00:03:43,710
Pay attention.

52
00:03:43,730 --> 00:03:46,820
This is absolutely relevant.

53
00:03:46,820 --> 00:03:54,080
2010 onwards, meaning this is a very old feeding tools, but we are going to use it today.

54
00:03:54,470 --> 00:04:03,030
The suggestion is for using a feed forward fee for what is almost the same redefeat net it holds.

55
00:04:03,030 --> 00:04:11,420
So has limited capability and features like we can only change Zahedan size and we can only change the

56
00:04:11,420 --> 00:04:19,100
train function, but we don't have any features to change the other parameters, like an activation

57
00:04:19,100 --> 00:04:19,640
function.

58
00:04:19,940 --> 00:04:24,250
This one is giving you a very good access to change parameters.

59
00:04:24,650 --> 00:04:30,470
For instance, this E for input Paton's this T chosen over target.

60
00:04:30,470 --> 00:04:35,620
This as I choose the size of hidden layers.

61
00:04:36,290 --> 00:04:40,430
There are two syntaxes to use this function.

62
00:04:40,430 --> 00:04:48,080
We can use new F F PTTs, which is for the simple one, and we can also use with more functions and

63
00:04:48,080 --> 00:04:48,870
more features.

64
00:04:49,190 --> 00:04:52,010
Now I'm going to use this one to are more explanation.

65
00:04:52,220 --> 00:04:53,820
You can just read them if you need.

66
00:04:54,380 --> 00:05:00,380
Let me back here and I'm going to change this feed net.

67
00:05:00,680 --> 00:05:03,170
Need new F.

68
00:05:07,210 --> 00:05:15,760
As the syntax mentioned, we need to first define the inputs, my inputs here are X, so I'm going to

69
00:05:15,760 --> 00:05:18,100
put X, then the targets.

70
00:05:18,760 --> 00:05:22,660
I just define the targets as T, x, t.

71
00:05:22,990 --> 00:05:25,090
It might have a different name in your program.

72
00:05:25,090 --> 00:05:26,410
Please pay attention to it.

73
00:05:26,800 --> 00:05:34,480
And then another one, which is the size of my hidden layer, I'm just going to copy and paste it.

74
00:05:37,620 --> 00:05:39,180
That's very real.

75
00:05:39,390 --> 00:05:42,630
Now let's run our program and check it if it's working.

76
00:05:42,990 --> 00:05:44,370
Yeah, very well.

77
00:05:44,370 --> 00:05:46,500
It's working perfectly.

78
00:05:47,550 --> 00:05:51,690
The thing that I'm looking for is to change this activation functions.

79
00:05:51,750 --> 00:05:57,780
As we can see, these are sigmoid and for the output layer is just using the purely.

80
00:05:58,170 --> 00:06:03,060
But what are different activation function that we can use for training a neural net for.

81
00:06:03,150 --> 00:06:05,910
Let's check them here in a comment box.

82
00:06:07,020 --> 00:06:08,280
Let me first query.

83
00:06:08,970 --> 00:06:13,320
You can type help and then transfer.

84
00:06:16,140 --> 00:06:24,810
OK, these are the transfer functions that we can use to train other neural nets for this one is a Campath

85
00:06:24,810 --> 00:06:26,760
competitive transfer function.

86
00:06:27,150 --> 00:06:34,890
We have Hatherly, which is a positive hardly need transfer function if you need to see more information

87
00:06:34,890 --> 00:06:35,820
about each one.

88
00:06:36,720 --> 00:06:38,370
Let me show you, for example, this one.

89
00:06:38,370 --> 00:06:39,720
Just copy it.

90
00:06:39,720 --> 00:06:44,340
And in the health box, look for it here horridly.

91
00:06:44,610 --> 00:06:51,360
We can see some information graph and symbols the syntaxes if you want to use in the description.

92
00:06:51,720 --> 00:07:00,030
And also an example, we can see this is a hard line to transfer function and it's just limited between

93
00:07:00,210 --> 00:07:02,250
negative one, two plus one.

94
00:07:03,150 --> 00:07:06,660
Now, let's check more transfer functions here.

95
00:07:06,660 --> 00:07:11,340
We have a Luksik which is sigmoid to transfer function.

96
00:07:11,340 --> 00:07:16,390
We have Porres mean this one is a positive linear transfer form.

97
00:07:16,440 --> 00:07:20,650
And this is also a good one if you want to use it for an output layer.

98
00:07:20,880 --> 00:07:26,160
Let me just look for you to help us to show you the graph.

99
00:07:26,850 --> 00:07:28,260
Pause, Linda.

100
00:07:33,850 --> 00:07:41,560
This one is basically a linear system, but it will only accept a positive part of the linear system

101
00:07:41,560 --> 00:07:42,890
for the transfer function.

102
00:07:43,060 --> 00:07:49,110
You might need a specific art food that only accept a positive outcome.

103
00:07:49,150 --> 00:07:51,580
That would be a very good fit for it.

104
00:07:52,420 --> 00:07:56,750
And here we have a radio based transfer function.

105
00:07:57,280 --> 00:07:57,910
Look at this one.

106
00:07:57,920 --> 00:08:02,470
This is a rack, bass or radial basis transfer function.

107
00:08:02,870 --> 00:08:05,950
I'll search for it in a health box.

108
00:08:05,950 --> 00:08:10,480
And then I use this one, for instance, to train my network.

109
00:08:10,480 --> 00:08:19,810
In this example, the graph here is like a radial basis transfer function and there are more like a

110
00:08:19,810 --> 00:08:23,670
tank, which is almost the same as the lock sic.

111
00:08:23,680 --> 00:08:29,090
And this one, the three bass is a triangle basis transfer function.

112
00:08:29,410 --> 00:08:37,990
So let me choose the rad bass and tre bass for training my network back to advance a screen window.

113
00:08:39,620 --> 00:08:48,280
I'm going to just define a transfer function here as the three bass for the first layer I have here

114
00:08:48,280 --> 00:08:54,250
two hidden layers, meaning I have a total of three layers and I should define their transfer functions

115
00:08:54,250 --> 00:08:55,780
or activation functions here.

116
00:08:56,470 --> 00:08:57,220
Three bass.

117
00:08:57,220 --> 00:09:02,980
I'm going to choose also this one, the right bass.

118
00:09:05,110 --> 00:09:15,160
Bass for the second there and another one just superiorly was working, Vera puling for the pure linear

119
00:09:15,160 --> 00:09:17,410
for the last one, which is of her output.

120
00:09:19,960 --> 00:09:23,230
Add this t a variable here and run.

121
00:09:26,080 --> 00:09:33,580
Look at this one, the first hidden layer is like a triangle, because I used to bass, the second one

122
00:09:33,910 --> 00:09:37,990
is redBus and the last one is a purely.

123
00:09:38,440 --> 00:09:41,960
But let's see, what about the 15 to what happened?

124
00:09:42,550 --> 00:09:54,130
OK, we can see it's not a very good FT change these transfer functions to maybe a Luksik and 10 for

125
00:09:54,130 --> 00:09:54,610
this one.

126
00:09:58,100 --> 00:10:01,550
Which are basically the same, so let's try it again.

127
00:10:03,620 --> 00:10:07,970
OK, here I have my activation functions and let's take a look at Ft.

128
00:10:08,660 --> 00:10:13,640
It is actually a good fit, so I'm going to keep these two very low.

129
00:10:15,200 --> 00:10:24,380
So by using a new F f more able to change the transfer function and activation function for each hidden

130
00:10:24,380 --> 00:10:24,800
layer.