WEBVTT 00:00.270 --> 00:01.110 Instructor: Hello and welcome back 00:01.110 --> 00:02.400 to the course on deep learning. 00:02.400 --> 00:03.930 Today we're talking about the neuron, 00:03.930 --> 00:06.270 which is the basic building block 00:06.270 --> 00:07.980 of artificial neural networks. 00:07.980 --> 00:09.360 So let's get started. 00:09.360 --> 00:11.310 Previously we saw an image which looked like this 00:11.310 --> 00:13.888 and these are actual real life neurons 00:13.888 --> 00:15.120 which have been smeared 00:15.120 --> 00:17.940 onto a glass and colored a little bit 00:17.940 --> 00:19.980 and they are observed through a microscope. 00:19.980 --> 00:22.170 So this is what they look like, as you can see 00:22.170 --> 00:24.390 quite an interesting structure, a body, 00:24.390 --> 00:27.780 and then lots of different tails, 00:27.780 --> 00:30.300 kind of branches coming out of them. 00:30.300 --> 00:33.510 And this is very interesting, but the question is 00:33.510 --> 00:36.150 how can we recreate that in a machine 00:36.150 --> 00:39.000 because we really need to recreate it in a machine, 00:39.000 --> 00:41.520 since the whole purpose of deep learning 00:41.520 --> 00:44.530 is to mimic how the human brain works 00:46.110 --> 00:48.227 in the hope that by doing so 00:48.227 --> 00:50.970 we are going to create something amazing. 00:50.970 --> 00:53.220 We're gonna create an amazing infrastructure 00:53.220 --> 00:55.200 for machines to be able to learn. 00:55.200 --> 00:56.820 And why do we hope for that? 00:56.820 --> 00:58.270 Well, because the human brain 00:59.460 --> 01:03.431 just happens to be one of the most powerful learning tools 01:03.431 --> 01:07.290 on the planet or learning mechanisms on the planet. 01:07.290 --> 01:09.810 And we just hope that if we recreate that, 01:09.810 --> 01:11.940 we'll have something as awesome as that. 01:11.940 --> 01:12.960 Our challenge right now, 01:12.960 --> 01:16.200 our very first step to creating artificial neural networks 01:16.200 --> 01:18.360 is to recreate a neuron. 01:18.360 --> 01:19.193 So how do we do that? 01:19.193 --> 01:20.026 Well, first of all 01:20.026 --> 01:23.850 let's have a closer look at what it actually is. 01:23.850 --> 01:25.680 This image was first created 01:25.680 --> 01:30.370 by Spanish neuroscientist Santiago Ramón y Cajal in 1899. 01:33.150 --> 01:36.870 And what he did was he dyed neurons 01:36.870 --> 01:37.800 in actual brain tissue 01:37.800 --> 01:39.870 and looked at them under a microscope. 01:39.870 --> 01:41.130 And while he was looking at them 01:41.130 --> 01:42.510 he actually drew what he saw. 01:42.510 --> 01:43.530 And this is what he saw. 01:43.530 --> 01:44.670 He saw two neurons 01:44.670 --> 01:46.818 or two large neurons over there at the top, 01:46.818 --> 01:50.040 which had all these branches coming out of them 01:50.040 --> 01:52.290 towards their top parts. 01:52.290 --> 01:54.730 And then each one of 'em had this rod 01:55.620 --> 01:58.260 or thread coming out towards the bottom, 01:58.260 --> 01:59.490 very long one. 01:59.490 --> 02:01.620 And yeah, so that's what he saw. 02:01.620 --> 02:04.110 And now, you know, technology has advanced quite a lot 02:04.110 --> 02:07.800 and we have seen neurons much closer, in more detail 02:07.800 --> 02:09.720 and now we can actually draw 02:09.720 --> 02:11.910 what it looks like diagrammatically. 02:11.910 --> 02:13.380 So let's have a look at that. 02:13.380 --> 02:14.220 Here's a neuron. 02:14.220 --> 02:15.390 This is what it looks like 02:15.390 --> 02:20.390 very similar to what Santiago Ramón drew over here. 02:20.550 --> 02:22.140 And here in this neuron, 02:22.140 --> 02:24.570 what we can see is that it's got a body, 02:24.570 --> 02:26.520 that's the main part of the neuron, 02:26.520 --> 02:28.170 and then it's got some branches at the top 02:28.170 --> 02:29.130 which are called dendrites, 02:29.130 --> 02:30.630 and it's also got an axon, 02:30.630 --> 02:33.300 which is that long tail of the neuron. 02:33.300 --> 02:35.970 And so what are these dendrites for 02:35.970 --> 02:36.803 and what's the axon for? 02:36.803 --> 02:39.810 Well, the key point to understand here is 02:39.810 --> 02:44.010 that neurons by themselves are pretty much useless. 02:44.010 --> 02:45.960 It's like ant, right? 02:45.960 --> 02:48.240 An ant on its own, can't do much. 02:48.240 --> 02:51.210 Five ants together, maybe they can pick something up 02:51.210 --> 02:54.300 but again, they can't build an ant hill, 02:54.300 --> 02:55.470 they can't establish a colony. 02:55.470 --> 02:59.370 They can't work together as a huge organism. 02:59.370 --> 03:01.560 But at the same time, when you have lots and lots of ants, 03:01.560 --> 03:04.320 like you have a million ants, they can build a whole colony. 03:04.320 --> 03:05.760 They can build an ant hill. 03:05.760 --> 03:06.600 Same thing with neurons. 03:06.600 --> 03:07.830 By itself, it's not that strong 03:07.830 --> 03:09.840 but when you have lots of neurons together, 03:09.840 --> 03:12.480 they work together to do magic. 03:12.480 --> 03:13.830 And how do they work together? 03:13.830 --> 03:14.663 That's the question. 03:14.663 --> 03:16.710 Well, that's what the dendrites and the axon are for. 03:16.710 --> 03:18.960 So the dendrites are kind of like the receivers 03:18.960 --> 03:20.040 of the signal for the neuron. 03:20.040 --> 03:21.600 And the axon is the transmitter 03:21.600 --> 03:23.190 of the signal for the neuron. 03:23.190 --> 03:26.550 And here's an image of how it all works conceptually. 03:26.550 --> 03:28.020 So at the top, you've got a neuron 03:28.020 --> 03:32.070 and you can see that its dendrites are connected to axons 03:32.070 --> 03:35.970 of other neurons that are even further away above it. 03:35.970 --> 03:40.050 And then the signal from this neuron travels down its axon 03:40.050 --> 03:42.870 and connects or passes onto the dendrites 03:42.870 --> 03:43.703 of the next neuron. 03:43.703 --> 03:45.000 And that's how they're connected. 03:45.000 --> 03:46.770 And in that small image over there, 03:46.770 --> 03:51.360 you can see that the axon doesn't actually touch 03:51.360 --> 03:52.441 the dendrite. 03:52.441 --> 03:54.900 A lot of machine learning 03:54.900 --> 03:57.330 or like a few machine learning scientists 03:57.330 --> 04:00.213 are very adamant about the fact that it doesn't touch. 04:02.850 --> 04:04.590 It doesn't touch, it has been proven 04:04.590 --> 04:06.960 that there is no physical connection there. 04:06.960 --> 04:09.630 But the point that we are interested in is that 04:09.630 --> 04:11.280 that connection between them, 04:11.280 --> 04:15.240 that the whole concept of the signal being passed, 04:15.240 --> 04:16.320 that's called the synapse. 04:16.320 --> 04:19.050 You can see over there in that little image 04:19.050 --> 04:22.260 that figure bracket is a synapse. 04:22.260 --> 04:23.820 And that's the term we're going to be using. 04:23.820 --> 04:27.900 So instead of calling our artificial neurons 04:27.900 --> 04:29.040 the lines that we're gonna have 04:29.040 --> 04:30.510 or the connectors for artificial neurons 04:30.510 --> 04:33.060 we're not gonna be calling 'em axons or dendrites 04:33.060 --> 04:35.220 because then the question is, whose connection is this? 04:35.220 --> 04:36.860 Is it that neuron's or is it this neuron's? 04:36.860 --> 04:39.450 We just call them synapses 04:39.450 --> 04:42.690 and that's kind of just answers all questions right away. 04:42.690 --> 04:45.150 That's just basically where the signal is passed, 04:45.150 --> 04:47.550 doesn't matter who that element belongs. 04:47.550 --> 04:50.280 So that's just a representation of the signal being passed. 04:50.280 --> 04:51.960 And we'll see that just now. 04:51.960 --> 04:55.140 So basically that's how a neuron works. 04:55.140 --> 04:58.860 And yeah, so let's move on to how we going to 04:58.860 --> 05:03.393 represent neurons or how create neurons in machines. 05:03.393 --> 05:06.420 So now we're moving away from neuroscience 05:06.420 --> 05:09.360 and moving into technology. 05:09.360 --> 05:10.350 And here we go. 05:10.350 --> 05:13.360 So here's our neuron also sometimes called the node. 05:13.360 --> 05:16.530 The neuron gets some input signals 05:16.530 --> 05:18.390 and it has an output signal. 05:18.390 --> 05:21.030 So dendrites and axons, remember, 05:21.030 --> 05:23.155 but again, we're gonna call these synapses. 05:23.155 --> 05:27.480 Then these input signals, we're gonna represent them 05:27.480 --> 05:29.070 with other neurons as well. 05:29.070 --> 05:32.970 So in this specific case, you can see that this neuron, 05:32.970 --> 05:35.880 this green neuron is getting signals from yellow neurons. 05:35.880 --> 05:39.109 And in this course we're going to try and stick to 05:39.109 --> 05:41.820 a certain color coding regime where yellow means 05:41.820 --> 05:42.653 an input layer. 05:42.653 --> 05:46.710 So basically all of the neurons that are on the outer layer 05:46.710 --> 05:51.060 or in the first front of where the signal's coming in. 05:51.060 --> 05:53.138 And by signal it might be 05:53.138 --> 05:57.120 a bit of an over overkill to call this a signal. 05:57.120 --> 05:59.340 It's just basically input values. 05:59.340 --> 06:02.040 So you know how even in a simple linear regression, 06:02.040 --> 06:03.000 you have input values 06:03.000 --> 06:05.070 and then you have a, a predicted value. 06:05.070 --> 06:05.903 Same thing here. 06:05.903 --> 06:07.200 So you have input values 06:07.200 --> 06:09.030 and there they are the yellow ones. 06:09.030 --> 06:11.250 And then on the right you'll see just now it'll be red, 06:11.250 --> 06:12.550 it'll be the output value. 06:13.560 --> 06:15.570 The thing that I wanted to point out here is 06:15.570 --> 06:17.580 that in this specific example, we're looking at a neuron, 06:17.580 --> 06:21.300 which is getting its signals from the input layer neurons. 06:21.300 --> 06:22.200 So they're also neurons 06:22.200 --> 06:24.510 but they're the input layer neurons. 06:24.510 --> 06:27.510 Sometimes you'll have neurons which get their signal 06:27.510 --> 06:30.510 from other hidden layer neurons 06:30.510 --> 06:31.770 so from other green neurons. 06:31.770 --> 06:33.330 And the concept is gonna be exactly the same. 06:33.330 --> 06:36.030 It's just in this case, for simplicity sake, 06:36.030 --> 06:37.620 we're portraying this example. 06:37.620 --> 06:39.030 And in terms of the input layer, 06:39.030 --> 06:40.480 the way to think about it is, 06:42.290 --> 06:45.330 in the analogy of the human brain, 06:45.330 --> 06:48.120 the input layer is your senses, right? 06:48.120 --> 06:52.470 So whatever you can see, hear, feel, touch or smell. 06:52.470 --> 06:55.860 And of course, there's a lot of things you can see. 06:55.860 --> 06:57.750 There's a lot of information coming in, 06:57.750 --> 07:00.150 but that's what your brain is limited to. 07:00.150 --> 07:05.150 It's pretty much lives in a box made out of bones. 07:06.060 --> 07:08.940 And it's a mind blowing concept 07:08.940 --> 07:10.590 to think about that your brain 07:10.590 --> 07:13.170 is just locked in a black box 07:13.170 --> 07:14.340 and the only thing- 07:14.340 --> 07:15.450 You can't see, you can't hear, 07:15.450 --> 07:17.370 the only thing it's getting is electrical impulses 07:17.370 --> 07:20.490 coming from these organs that you have, 07:20.490 --> 07:23.640 which are called your ears, nose, eyes, you know, 07:23.640 --> 07:26.895 your sense of touch and whatever 07:26.895 --> 07:28.380 and your taste, right? 07:28.380 --> 07:30.090 So it's just getting signals, 07:30.090 --> 07:32.190 but it basically lives in this dark black box 07:32.190 --> 07:36.030 and it's making sense of the world through your senses. 07:36.030 --> 07:37.801 It's phenomenal. 07:37.801 --> 07:41.940 And so, yeah, so you have these inputs that are coming in, 07:41.940 --> 07:44.040 in the terms of human brain, those are your five senses. 07:44.040 --> 07:47.640 And in terms of machine learning or deep learning 07:47.640 --> 07:50.640 that is basically your input values. 07:50.640 --> 07:51.900 So your independent variables 07:51.900 --> 07:52.830 and we'll get that too in a second. 07:52.830 --> 07:55.023 So your input values, 07:56.400 --> 07:58.860 the signal is passed through synapses to the neuron 07:58.860 --> 08:00.780 and then your neuron has an output value 08:00.780 --> 08:03.510 that it passes further on down the chain. 08:03.510 --> 08:04.350 In this specific case, 08:04.350 --> 08:07.020 in terms of color-coding, again, yellow means input layer. 08:07.020 --> 08:08.610 So we're kind of simplifying everything here. 08:08.610 --> 08:11.130 We're saying we're only gonna have like the input layer 08:11.130 --> 08:14.340 then we're gonna have one hidden layer with the green 08:14.340 --> 08:15.173 which is the hidden layer. 08:15.173 --> 08:17.520 And then we're gonna have right the output layer right away. 08:17.520 --> 08:20.373 So just so that we can get used to these colors for now. 08:21.570 --> 08:22.403 So there we go. 08:22.403 --> 08:24.030 That's the basic structure. 08:24.030 --> 08:26.100 So now let's look in a bit more detail 08:26.100 --> 08:28.380 at these different elements that we have. 08:28.380 --> 08:31.080 So we've got the input layer, and what do we have here? 08:31.080 --> 08:33.330 Well, we have these inputs 08:33.330 --> 08:35.490 which are in fact independent variables. 08:35.490 --> 08:37.260 So independent variable one, independent variable two 08:37.260 --> 08:38.670 independent variable m. 08:38.670 --> 08:40.920 The important thing to remember here is that 08:40.920 --> 08:43.410 these independent variables are all 08:43.410 --> 08:44.730 for one single observation. 08:44.730 --> 08:47.580 So think of it as just one row in your database. 08:47.580 --> 08:49.020 One observation. 08:49.020 --> 08:52.020 You just take all of the independent variables, 08:52.020 --> 08:54.827 you know, maybe it's the age of the person? 08:54.827 --> 08:57.810 The amount of money in the bank accounts? 08:57.810 --> 09:00.780 And then how do they drive or walk to work? 09:00.780 --> 09:03.060 What method of transportation do they use? 09:03.060 --> 09:06.120 So, but that's all descriptors of one specific person 09:06.120 --> 09:09.060 that you are either you're training your model on 09:09.060 --> 09:11.583 or you're performing some prediction on. 09:12.570 --> 09:14.100 And the other thing you need to know 09:14.100 --> 09:16.380 about these variables is that you need to standardize them. 09:16.380 --> 09:18.000 So you need to either standardize them 09:18.000 --> 09:20.160 which means make sure that they have a mean 09:20.160 --> 09:21.360 of zero and variance one, 09:21.360 --> 09:25.230 or you can also sometimes and Hadelin will point out 09:25.230 --> 09:28.050 these traces in a bit more detail. 09:28.050 --> 09:29.820 Perhaps in the practical tutorials, 09:29.820 --> 09:30.960 you might come across these, 09:30.960 --> 09:33.000 sometimes you might want to, not standardize, 09:33.000 --> 09:34.980 you might wanna normalize them, 09:34.980 --> 09:38.250 meaning that instead of making sure the mean is zero 09:38.250 --> 09:42.480 and variance is one, you just subtract the minimum value 09:42.480 --> 09:44.550 and then you divide by the maximum minus minimum, 09:44.550 --> 09:46.500 so by the range of your values, 09:46.500 --> 09:49.500 and therefore, you get values between zero and one. 09:49.500 --> 09:52.440 And it depends on the scenario. 09:52.440 --> 09:53.580 You might wanna do one or the other 09:53.580 --> 09:55.980 but basically you want all of these variables to be 09:55.980 --> 10:00.750 quite similar, about the same range of values. 10:00.750 --> 10:02.190 And why is that? 10:02.190 --> 10:04.290 Well, all of these values are going to go 10:04.290 --> 10:06.540 into a neural network where as we'll see just now, 10:06.540 --> 10:09.660 they'll be added up and multiplied by weight, 10:09.660 --> 10:10.493 added up and so on. 10:10.493 --> 10:13.950 And it's just going to be easier for the neural network 10:13.950 --> 10:17.190 to process them, if they're all about the same. 10:17.190 --> 10:19.469 And in fact, you know, 10:19.469 --> 10:24.210 that's just how it is going to be able to work properly. 10:24.210 --> 10:27.240 And if you want to read more about standardization, 10:27.240 --> 10:29.100 normalization and other things that you can do 10:29.100 --> 10:30.660 if you input variables, 10:30.660 --> 10:34.740 a good additional reading paper is called 10:34.740 --> 10:38.520 Efficient BackProp by Yann LeCun 1998. 10:38.520 --> 10:39.510 The links over there. 10:39.510 --> 10:41.940 So Yann LeCun, we're actually going to talk more 10:41.940 --> 10:45.930 about this phenomenal person in the space of deep learning 10:45.930 --> 10:48.660 in the part of the course where we're talking 10:48.660 --> 10:50.460 about convolutional neural networks. 10:50.460 --> 10:53.250 And you'll see that this is definitely a person 10:53.250 --> 10:55.260 who knows what he's talking about. 10:55.260 --> 10:57.720 He's a close friend of Geoffrey Hinton 10:57.720 --> 11:00.870 who we've already seen, who've already mentioned. 11:00.870 --> 11:03.630 So in this paper you'll learn more 11:03.630 --> 11:05.670 about standardization and normalization 11:05.670 --> 11:07.140 but also you can pick up lots of 11:07.140 --> 11:08.430 other different tips and tricks 11:08.430 --> 11:11.280 and he'll be a good source for additional reading 11:11.280 --> 11:12.570 as you go through this course. 11:12.570 --> 11:13.403 So yeah, check it out, 11:13.403 --> 11:16.260 if you are interested in some additional reading. 11:16.260 --> 11:17.093 There we go. 11:17.093 --> 11:20.370 So that's what we need to do with the variables. 11:20.370 --> 11:23.190 And here we've got the output value. 11:23.190 --> 11:25.110 So what can our output value be? 11:25.110 --> 11:26.280 Well, we've got a couple of options. 11:26.280 --> 11:28.920 Output value can be, it can be continuous, 11:28.920 --> 11:30.120 like for instance, price. 11:30.120 --> 11:31.320 It can be binary. 11:31.320 --> 11:33.690 For instance, a person will exit or will stay. 11:33.690 --> 11:36.180 Or it can be a categorical variable. 11:36.180 --> 11:39.180 And if it's a categorical variable 11:39.180 --> 11:41.820 the important thing to remember here is that in that case 11:41.820 --> 11:43.890 your output value won't be just one 11:43.890 --> 11:45.990 it'll be several output values 11:45.990 --> 11:48.000 because these will be your dummy variables 11:48.000 --> 11:51.330 which will be representing your categories. 11:51.330 --> 11:53.798 And that's just how it works. 11:53.798 --> 11:56.490 Just important to remember that in that case 11:56.490 --> 11:59.100 that's how you're going to be getting your categories 11:59.100 --> 12:01.593 out of the artificial neural network. 12:02.820 --> 12:05.730 But let's go back to a simple case of one output value. 12:05.730 --> 12:08.010 And now let's one more point 12:08.010 --> 12:10.170 or a point that we've already made, 12:10.170 --> 12:12.273 I just wanted to reiterate this point. 12:13.380 --> 12:15.450 On the left, you've got a single observation. 12:15.450 --> 12:17.730 So one row from your dataset 12:17.730 --> 12:19.770 and on the right you have a single observation as well. 12:19.770 --> 12:22.170 And that is the same observation. 12:22.170 --> 12:25.530 So important to remember that whatever inputs 12:25.530 --> 12:26.430 you're putting in 12:26.430 --> 12:27.450 that's for one row 12:27.450 --> 12:30.060 and then the output you get that is for that same exact row. 12:30.060 --> 12:32.190 Or if you're training your neural network 12:32.190 --> 12:34.470 then you're putting the inputs in for that one row, 12:34.470 --> 12:36.390 you're putting the output in for that one row. 12:36.390 --> 12:39.330 So if you wanna simplify the complexity, 12:39.330 --> 12:42.270 think of it like a simple linear regression 12:42.270 --> 12:43.980 or a multivariate linear regression. 12:43.980 --> 12:46.710 So you're putting in your values 12:46.710 --> 12:48.343 you have your output, 12:48.343 --> 12:49.710 there's no question about it 12:49.710 --> 12:51.750 when you were talking about things like regression 12:51.750 --> 12:52.800 because we're so used to it. 12:52.800 --> 12:53.633 Same thing here. 12:53.633 --> 12:54.990 It's nothing too complex. 12:54.990 --> 12:56.760 We're just putting in values, we're getting output. 12:56.760 --> 12:57.780 But just remember 12:57.780 --> 12:59.610 that every time it's one row you're dealing with 12:59.610 --> 13:00.960 so you don't get confused 13:00.960 --> 13:02.700 and start putting in like, 13:02.700 --> 13:05.370 thinking that these are different rows 13:05.370 --> 13:06.203 that you're putting into 13:06.203 --> 13:09.150 your artificial neural network or something. 13:09.150 --> 13:11.250 This is all just values in that one row. 13:11.250 --> 13:14.010 So different observation, different characteristics 13:14.010 --> 13:15.390 or attributes relating 13:15.390 --> 13:17.703 to that one observation every single time. 13:19.170 --> 13:22.041 Okay, so next thing that we wanna talk about here 13:22.041 --> 13:23.280 are the synapses? 13:23.280 --> 13:24.840 Is the synapses. 13:24.840 --> 13:25.860 Here we've got synapses 13:25.860 --> 13:28.530 and they all actually get assigned weights. 13:28.530 --> 13:31.830 Weights, we're gonna talk more about weights further down. 13:31.830 --> 13:36.420 But in short, weights are crucial to 13:36.420 --> 13:38.640 artificial neural networks functioning 13:38.640 --> 13:41.790 because weights are how neural networks learn. 13:41.790 --> 13:44.430 By adjusting the weights, 13:44.430 --> 13:47.430 the neural network decides in every single case, 13:47.430 --> 13:49.140 what signal is important, 13:49.140 --> 13:51.150 what signal is not important to certain neuron, 13:51.150 --> 13:52.410 what signal gets passed along 13:52.410 --> 13:54.360 and what signal doesn't get passed along, 13:54.360 --> 13:55.590 or to what strength, 13:55.590 --> 13:57.750 to what extent signals get passed along. 13:57.750 --> 13:59.340 So weights are crucial. 13:59.340 --> 14:02.460 They are the things that get adjusted 14:02.460 --> 14:03.570 through the process of learning. 14:03.570 --> 14:06.180 Like when you're training an artificial neural network, 14:06.180 --> 14:08.520 you're basically adjusting all of the weights 14:08.520 --> 14:11.250 in all the synapses across this whole neural network. 14:11.250 --> 14:15.900 And that's where gradient descent and back propagation 14:15.900 --> 14:16.733 come into play. 14:16.733 --> 14:19.650 And those are concepts that we'll also discuss. 14:19.650 --> 14:21.390 So basically those are the weights. 14:21.390 --> 14:23.130 That's all we need to know for now. 14:23.130 --> 14:24.630 And here we've got the neuron. 14:24.630 --> 14:26.790 So signals go into the neuron 14:26.790 --> 14:28.410 and what happens in the neuron? 14:28.410 --> 14:30.780 So this is the interesting part, 14:30.780 --> 14:32.250 we're talking about the neuron today, 14:32.250 --> 14:33.870 What happens inside the neuron? 14:33.870 --> 14:35.520 So a few things happen. 14:35.520 --> 14:39.300 First thing and the first step is that all of these values 14:39.300 --> 14:41.340 that it's getting, get added up. 14:41.340 --> 14:46.050 So it takes the added, so the weighted sum of all 14:46.050 --> 14:48.990 of the input values that it's getting. 14:48.990 --> 14:49.890 Very simple, right? 14:49.890 --> 14:51.240 It's very, very straightforward. 14:51.240 --> 14:54.240 Just add up, multiply by the weight, add them up. 14:54.240 --> 14:57.180 And then it applies an activation function. 14:57.180 --> 14:58.050 Now we're gonna talk more 14:58.050 --> 14:59.880 about activation functions further down 14:59.880 --> 15:01.590 but it's basically a function that is assigned 15:01.590 --> 15:04.230 to this neuron or to this whole layer 15:04.230 --> 15:09.230 and it is applied to this weighted sum. 15:09.540 --> 15:12.280 And then from that, the neuron understands 15:13.740 --> 15:16.260 if it needs to pass on a signal, 15:16.260 --> 15:17.820 that's the signal that it passes on 15:17.820 --> 15:22.320 that the function applied to the weighted sum. 15:22.320 --> 15:23.970 But basically depending on the function, 15:23.970 --> 15:25.390 the neuron will either pass on a signal it 15:25.390 --> 15:27.870 or it won't pass that signal on. 15:27.870 --> 15:30.470 And that's exactly what happened here in step three, 15:31.410 --> 15:33.120 the neuron passes on that signal 15:33.120 --> 15:35.820 to the next neuron down the line. 15:35.820 --> 15:37.230 And that's what we're going to talk about 15:37.230 --> 15:39.960 in the next tutorial because it is quite an important topic. 15:39.960 --> 15:44.010 We want to delve deeper into the activation function 15:44.010 --> 15:45.270 but hopefully for now, 15:45.270 --> 15:47.610 everything should be pretty clear how, 15:47.610 --> 15:49.650 you know, you've got input values, you've got weights, 15:49.650 --> 15:50.640 you've got these synapses, 15:50.640 --> 15:52.860 you've got something, you know, happens in the neuron, 15:52.860 --> 15:54.120 you've got the weighted sum 15:54.120 --> 15:55.710 and then activation function's applied 15:55.710 --> 15:57.180 and then that is paused down the line. 15:57.180 --> 15:58.230 And that is just repeated 15:58.230 --> 15:59.940 throughout the whole neural network, 15:59.940 --> 16:02.250 on and on and on and on. 16:02.250 --> 16:04.590 You know, thousands, hundreds of thousands of times 16:04.590 --> 16:06.840 depending on how big, how many neurons you have, 16:06.840 --> 16:09.540 how many synapses you have in your neural network. 16:09.540 --> 16:10.373 So there we go. 16:10.373 --> 16:12.120 Hope you enjoyed today's tutorial. 16:12.120 --> 16:13.260 Can't wait to see you next time. 16:13.260 --> 16:15.183 And until then, enjoy deep learning.