1
00:00:00,690 --> 00:00:01,040
Hello.

2
00:00:01,050 --> 00:00:02,270
Welcome back.

3
00:00:02,310 --> 00:00:08,040
Activation functions introduce non-linear properties to our neural network.

4
00:00:08,080 --> 00:00:14,880
The main purpose is to convert an input signal of a unit or a node in a 1 neural network to an output

5
00:00:14,880 --> 00:00:20,660
signal that outputs signal is then used as an input in the next layer.

6
00:00:20,730 --> 00:00:27,630
We bought a sum of product of input x and the corresponding weight and apply an activation function

7
00:00:27,630 --> 00:00:34,060
to it to get the output of the LIA and feed it as an input to the next layer.

8
00:00:35,070 --> 00:00:42,560
If we do not apply an activation function then the output signal will simply be a simple linear function.

9
00:00:42,690 --> 00:00:49,350
Linear equations are easy to solve but they are limited in the complexity and have less power to learn

10
00:00:49,980 --> 00:00:56,670
complex functional mapping from data and no their work without an activation function would simply be

11
00:00:56,670 --> 00:01:03,840
a linear regression model which has limited power and lacks good performance.

12
00:01:03,840 --> 00:01:10,530
Also without activation functions our neural network would be um would find it difficult or would not

13
00:01:10,530 --> 00:01:18,870
be able to even land and model other complicated types of data such as images videos audio speech etc.

14
00:01:19,860 --> 00:01:27,540
We often refer to activation functions as nonlinear routines nonlinear functions are those which have

15
00:01:27,540 --> 00:01:35,880
degrees more than one and have a curvature when plotted apart from requiring that a war activation function

16
00:01:35,970 --> 00:01:37,860
is non-linear.

17
00:01:37,890 --> 00:01:45,870
We also need it to be easily differential meaning we needed to we need to be able to compute the derivative

18
00:01:45,960 --> 00:01:52,020
of the activation function quickly we need it to be this way so that we can quickly perform something

19
00:01:52,020 --> 00:01:58,500
known as back propagation quickly in a what neural network the three most popular activation functions

20
00:01:58,500 --> 00:02:05,460
are the sigmoid the ton H and directed linear units also known as Red loop for short the sigmoid activation

21
00:02:05,460 --> 00:02:13,110
function has the deform f of c course one over one plus E raised to the power minus C and the curve

22
00:02:13,170 --> 00:02:21,690
has an S shape as we can see over here d ton H is expressed thus FFC equals E race the policy minus

23
00:02:21,810 --> 00:02:28,550
you raise the bar minus the all over you rush to to the policy plus E Race to the power minus the s

24
00:02:28,560 --> 00:02:37,920
we can see over here the range of the sigmoid is between 0 and 1 and the range of the of the 10 H is

25
00:02:38,010 --> 00:02:45,210
minus 1 and 1 as we can see in the graph over here this is zero point over here this is one point over

26
00:02:45,210 --> 00:02:53,130
here for the sigmoid and is the ton H this is minus one here and this is one the realm is expressed

27
00:02:53,190 --> 00:03:05,160
as Max zero Zi meaning f of C E course Max zero C this means if Z is less than zero the result is zero

28
00:03:05,610 --> 00:03:15,240
if C is greater than or equal to zero direct to resort is one the sigmoid function is good for output

29
00:03:15,240 --> 00:03:24,640
Lay is only the ton H function performs best when used in it in layers the realm is versatile or versatile

30
00:03:24,720 --> 00:03:32,160
meaning it's good for all this we also show over here the derivatives of these activation functions

31
00:03:32,450 --> 00:03:37,690
we shall implement both the functions and the derivatives in code.

32
00:03:38,220 --> 00:03:41,750
So this order is for this lesson and I shall see you in the next lesson.