WEBVTT 00:00.480 --> 00:02.970 -: All right, exciting tutorial ahead. 00:02.970 --> 00:04.860 Welcome back to the course on deep learning. 00:04.860 --> 00:08.070 Today, we're talking about how do neural networks work. 00:08.070 --> 00:09.810 Now, we've laid a lot of groundwork. 00:09.810 --> 00:13.020 We've talked about how neural networks are structured, 00:13.020 --> 00:16.590 what elements they consist of, and even their functionality. 00:16.590 --> 00:19.590 And today, we're going to look at a real example 00:19.590 --> 00:21.870 of how a neural network can be applied 00:21.870 --> 00:24.810 and we're actually gonna work step by step through 00:24.810 --> 00:28.320 the process of its application so we know what's going on. 00:28.320 --> 00:30.000 So let's have a look what example we're going 00:30.000 --> 00:30.833 to be talking about. 00:30.833 --> 00:33.210 We're going to be looking at property valuation. 00:33.210 --> 00:35.760 So we're going to look at a neural network 00:35.760 --> 00:39.600 that takes in some parameters of our property and values it. 00:39.600 --> 00:41.910 And the thing here, there's a small caveat 00:41.910 --> 00:44.730 for today's tutorial and that is we're not actually going 00:44.730 --> 00:45.840 to train the network. 00:45.840 --> 00:48.300 So a very important part in neural networks 00:48.300 --> 00:49.380 is training them up. 00:49.380 --> 00:52.290 And we're going to look at that in the next tutorials 00:52.290 --> 00:53.430 in this section. 00:53.430 --> 00:55.620 For now, we're going to focus on the actual application. 00:55.620 --> 00:57.540 So we're going to work with a neural network 00:57.540 --> 01:01.890 that we're going to pretend is already trained up 01:01.890 --> 01:04.860 and that will allow us to focus on the application side 01:04.860 --> 01:07.770 of things and not get bogged down in the training aspect. 01:07.770 --> 01:10.140 And then we'll cover off the training when we already know 01:10.140 --> 01:11.760 the end goal we're working towards. 01:11.760 --> 01:12.600 Sound good? 01:12.600 --> 01:15.210 All right, let's jump straight into it. 01:15.210 --> 01:19.350 So let's say we have some input parameters, right? 01:19.350 --> 01:21.450 So let's say we have four parameters about the property. 01:21.450 --> 01:24.240 We have area in squared feet. 01:24.240 --> 01:26.370 We have the number of bedrooms, the distance, 01:26.370 --> 01:28.710 the city in miles, the near city 01:28.710 --> 01:29.820 and the age of the property. 01:29.820 --> 01:33.660 And all of those four are going to comprise our input layer. 01:33.660 --> 01:37.200 Now, of course there're probably way more parameters 01:37.200 --> 01:39.960 that define the price or property, 01:39.960 --> 01:41.640 but for simplicity's sake we're going to look 01:41.640 --> 01:42.630 at just these four. 01:42.630 --> 01:46.470 Now, in its very basic form, a neural network only 01:46.470 --> 01:48.390 has an input layer and an output layer. 01:48.390 --> 01:50.040 So no hidden layers. 01:50.040 --> 01:52.290 And our output layer is the price that we're predicting. 01:52.290 --> 01:57.150 So in this form what these input variables would do 01:57.150 --> 02:01.020 is they would just be weighted up by the synapses 02:01.020 --> 02:04.080 and then the output there would be calculated 02:04.080 --> 02:05.490 or basically the price would be calculated 02:05.490 --> 02:06.323 and would get a price. 02:06.323 --> 02:10.050 And for instance, the price could be calculated as simple 02:10.050 --> 02:14.220 as the weighted sum of all of the inputs. 02:14.220 --> 02:16.590 And again, here you could use pretty much any function. 02:16.590 --> 02:19.110 You could use what we're using now, 02:19.110 --> 02:22.020 we could use any of the excavation functions 02:22.020 --> 02:26.040 we had previously, you could use a logistic regression, 02:26.040 --> 02:29.250 you could use a squared function. 02:29.250 --> 02:31.230 You could use pretty much anything here. 02:31.230 --> 02:33.480 But the point is that you get a certain output. 02:33.480 --> 02:37.290 And moreover, most of the machine learning algorithms 02:37.290 --> 02:41.010 that exist can be represented in this format. 02:41.010 --> 02:43.050 This is basically a diagrammatic representation 02:43.050 --> 02:45.630 of how you deal with the the variables, right? 02:45.630 --> 02:47.340 By changing the weights, the formulas 02:47.340 --> 02:49.350 you can accomplish quite a lot 02:49.350 --> 02:51.930 of the machine learning algorithms that we've talked 02:51.930 --> 02:55.410 about before and put them into this form. 02:55.410 --> 02:58.500 And that just stands to show how powerful neural networks 02:58.500 --> 03:00.843 are that even without the hidden layers, 03:01.890 --> 03:03.630 we already have a representation that works 03:03.630 --> 03:05.490 for most other machine learning algorithms. 03:05.490 --> 03:08.940 But in neural networks what we do have 03:08.940 --> 03:10.530 is an extra advantage that gives us lots 03:10.530 --> 03:15.530 of flexibility and power which is where that increase 03:15.570 --> 03:20.570 in accuracy comes from and that power is the hidden layers. 03:20.610 --> 03:22.920 And there we go that's our hidden layer. 03:22.920 --> 03:26.670 We've added it in, and now we're going to understand how 03:26.670 --> 03:30.270 that hidden layer gives us that extra power. 03:30.270 --> 03:32.190 And in fact, to do that we're going 03:32.190 --> 03:34.080 to walk through an example. 03:34.080 --> 03:36.510 So as we agreed this neural network has already been trained 03:36.510 --> 03:38.610 up and now we're just going to plug in, 03:38.610 --> 03:41.520 we're going to imagine that we're plugging in a property 03:41.520 --> 03:45.150 and we're going to walk step by step through how 03:45.150 --> 03:49.170 the neural network will deal with the input variables 03:49.170 --> 03:50.460 and calculate the hidden layer 03:50.460 --> 03:51.900 and then calculate the output layer. 03:51.900 --> 03:52.830 So let's go through this. 03:52.830 --> 03:55.050 This is gonna be exciting, all right. 03:55.050 --> 03:57.050 We've got all four variables on the left 03:58.110 --> 04:00.000 and we're going to first start with the top neural 04:00.000 --> 04:01.410 on the hidden layer. 04:01.410 --> 04:04.290 Now, as we previously saw in the previous tutorials, 04:04.290 --> 04:06.930 all of the neurals from the input layer, 04:06.930 --> 04:10.170 they have synopsis connecting each one 04:10.170 --> 04:13.470 of them to the top neural in the hidden layer. 04:13.470 --> 04:15.240 And those synopsis have weights. 04:15.240 --> 04:17.670 Now, let's agree that some weights will 04:17.670 --> 04:20.130 have a non-zero value, some weights will have a zero value 04:20.130 --> 04:25.003 because basically not all inputs will be important 04:26.640 --> 04:27.840 for every single neural. 04:27.840 --> 04:29.430 Sometimes inputs will not be important. 04:29.430 --> 04:31.770 And here we can see two examples that X1, X3, 04:31.770 --> 04:34.710 the area and the distance to the city in miles are important 04:34.710 --> 04:37.560 for that neural whereas bedrooms and age are not. 04:37.560 --> 04:39.240 And let's think about this for a second. 04:39.240 --> 04:40.530 How would that be the case? 04:40.530 --> 04:44.910 Like why would a certain neural be linked 04:44.910 --> 04:46.740 to the area and the distance? 04:46.740 --> 04:47.573 What could that mean? 04:47.573 --> 04:50.490 Well, that could mean that normally the further away 04:50.490 --> 04:53.430 you get from the city, the cheaper real estate becomes 04:53.430 --> 04:56.280 and therefore the space in square feet 04:56.280 --> 04:58.470 of properties becomes larger, right? 04:58.470 --> 05:00.420 So for the same price you can get a larger property 05:00.420 --> 05:02.580 the further away you go from the city, that's normal, right? 05:02.580 --> 05:03.720 That makes sense. 05:03.720 --> 05:06.175 And probably what this neural is doing is it 05:06.175 --> 05:09.900 is looking specifically, it's like like a sniper, 05:09.900 --> 05:13.877 it's looking for area properties which are not 05:15.090 --> 05:17.610 so far from the city, but have a large area. 05:17.610 --> 05:19.890 So for their distance from the city, 05:19.890 --> 05:24.890 they have an unfair square foot area, right? 05:25.110 --> 05:26.340 So something that's abnormal. 05:26.340 --> 05:27.660 It's higher than average. 05:27.660 --> 05:30.000 So they're quite close to the city, but they're still large 05:30.000 --> 05:32.673 as opposed to the other ones at the same distance. 05:34.530 --> 05:35.970 Again, we're speculating here, 05:35.970 --> 05:38.610 but that neural might be picking out laser picking 05:38.610 --> 05:41.730 out those specific properties and it will activate, 05:41.730 --> 05:43.170 hence the activation function. 05:43.170 --> 05:46.680 It'll activate, it'll fire up only when the certain criteria 05:46.680 --> 05:49.170 is met that you know the distance and the area 05:49.170 --> 05:50.310 of the property, distance to city 05:50.310 --> 05:52.020 and the area of the property. 05:52.020 --> 05:55.470 And it performs its own calculations inside itself 05:55.470 --> 05:57.900 and it combines those two and as soon 05:57.900 --> 06:00.450 as certain criteria, it fires up and that contributes 06:00.450 --> 06:02.190 to the price in the output layer. 06:02.190 --> 06:04.290 And therefore this neural doesn't really care 06:04.290 --> 06:05.910 about bedrooms and age of the property 06:05.910 --> 06:07.740 because it's focused on that specific thing. 06:07.740 --> 06:10.680 That's where the power of the neural network comes 06:10.680 --> 06:12.450 from because you have many of these neurals 06:12.450 --> 06:14.670 and we'll see just now how the other ones work. 06:14.670 --> 06:16.050 But what I wanted to agree here 06:16.050 --> 06:19.140 is that let's not even draw these lines 06:19.140 --> 06:22.620 for the synopsis that are not in play 06:22.620 --> 06:24.180 so that we don't clutter up our image. 06:24.180 --> 06:25.620 That's the only reason we're not gonna draw them. 06:25.620 --> 06:27.450 So let's just get rid of those two 06:27.450 --> 06:30.660 and that way we will know exactly, okay, so this neural 06:30.660 --> 06:34.890 is focused on area and distance to the city, all right. 06:34.890 --> 06:36.000 So as long as we agree on that, 06:36.000 --> 06:36.900 let's move on to the next one. 06:36.900 --> 06:38.880 Let's take the one in the middle. 06:38.880 --> 06:42.090 Here, we've got three parameters feeding into this neural. 06:42.090 --> 06:44.040 So we've got the area, the bedrooms 06:44.040 --> 06:45.990 and the age of the property. 06:45.990 --> 06:48.780 So what could be the reason here? 06:48.780 --> 06:52.950 Again, let's try to understand the intuition, 06:52.950 --> 06:54.690 the thinking of this neural. 06:54.690 --> 06:56.100 How is this neural thinking? 06:56.100 --> 06:57.810 Why is it picking these three parameters? 06:57.810 --> 06:58.830 What could it be? 06:58.830 --> 07:02.310 What could have hit like found in the data, right? 07:02.310 --> 07:04.620 So we've already established this is a trained up data set. 07:04.620 --> 07:06.570 The training has happened a long time ago, 07:06.570 --> 07:09.090 maybe like a day ago or somebody's already trained 07:09.090 --> 07:09.923 up this data. 07:09.923 --> 07:12.000 So now we're just applying and we know that this neural 07:12.000 --> 07:13.590 through all the thousands of examples 07:13.590 --> 07:16.950 of properties has found out that the area plus 07:16.950 --> 07:18.960 the bedrooms plus the age combination 07:18.960 --> 07:20.580 of those parameters is important. 07:20.580 --> 07:21.630 Why could that be the case? 07:21.630 --> 07:26.310 Well, for instance, maybe in that specific city 07:26.310 --> 07:29.430 in those suburbs that this neural network has been trained 07:29.430 --> 07:34.430 up in, perhaps there's a lot of families with kids, with two 07:36.540 --> 07:40.920 or more children who are looking for large properties 07:40.920 --> 07:43.620 with lots of bedrooms which are new, right? 07:43.620 --> 07:46.740 Which are not old properties. 07:46.740 --> 07:51.210 Because maybe in that area, most of the properties are kind 07:51.210 --> 07:52.950 of like big properties are usually old, 07:52.950 --> 07:55.140 but there's lots of modern families 07:55.140 --> 07:58.390 and maybe there has been a social demographic shift 07:59.460 --> 08:00.840 or maybe there's been like a lot 08:00.840 --> 08:04.830 of like some growth in terms of employment 08:04.830 --> 08:07.560 and jobs for the younger side of population. 08:07.560 --> 08:11.790 Maybe just like the population demographics have changed 08:11.790 --> 08:16.790 and now younger couples or younger families are looking 08:17.070 --> 08:20.970 for properties, but they prefer newer properties. 08:20.970 --> 08:23.760 So they want the age of the property to be lower. 08:23.760 --> 08:26.850 And hence from the training that this neural network 08:26.850 --> 08:29.280 has undergone, it knows that when there's a property 08:29.280 --> 08:32.348 with a large area and with lots of bedrooms with 08:32.348 --> 08:34.230 at least three bedrooms, for the parents, 08:34.230 --> 08:35.670 for the first child, for the second child, 08:35.670 --> 08:38.520 for at least three bedrooms, maybe a guest room, 08:38.520 --> 08:41.250 when a new property with a high area 08:41.250 --> 08:44.370 and lots of bedrooms that is valued. 08:44.370 --> 08:45.960 In that market, that is valuable. 08:45.960 --> 08:48.270 So that neural has picked that up. 08:48.270 --> 08:51.030 It knows that, okay, so this is what I'm gonna 08:51.030 --> 08:51.863 be looking for. 08:51.863 --> 08:53.280 I don't care about the distance to the city in miles, 08:53.280 --> 08:56.310 wherever it is as long as it has high area, 08:56.310 --> 08:57.143 lots of bedrooms. 08:57.143 --> 08:59.820 As soon as that criteria is met, the neural fires up. 08:59.820 --> 09:01.680 And the combination of these three parameters 09:01.680 --> 09:03.360 and this is again, this is where the power 09:03.360 --> 09:04.560 of the neural network is coming 09:04.560 --> 09:07.170 from because it combines these three parameters 09:07.170 --> 09:12.170 into a brand new attribute that helps with the evaluation 09:17.040 --> 09:17.873 of the property. 09:17.873 --> 09:19.800 It combines them into a new attribute 09:19.800 --> 09:21.840 and therefore it's more precise. 09:21.840 --> 09:24.240 So there we go, that's how that neural works. 09:24.240 --> 09:25.560 And let's look at another one. 09:25.560 --> 09:27.150 Let's look at the very bottom one. 09:27.150 --> 09:30.750 For instance, this neural could even have picked up 09:30.750 --> 09:31.890 just one parameter. 09:31.890 --> 09:33.720 It could have just picked up age 09:33.720 --> 09:35.520 and not any of the other ones. 09:35.520 --> 09:37.890 And how could that be the case? 09:37.890 --> 09:42.890 Well, this is a classic example of when age could mean like 09:43.110 --> 09:45.270 as we all know, the older property 09:45.270 --> 09:47.730 is usually it's less valuable because it's worn out, 09:47.730 --> 09:50.160 probably the building is old, probably things 09:50.160 --> 09:51.930 are falling apart, more maintenance is required. 09:51.930 --> 09:55.500 So the price drops in terms of the price of the real estate. 09:55.500 --> 09:57.750 Whereas a brand new building it would be more expensive 09:57.750 --> 09:59.010 because it's brand new. 09:59.010 --> 10:01.950 But perhaps if a property is over a certain age, 10:01.950 --> 10:03.990 that could indicate that it's a historic property. 10:03.990 --> 10:06.750 For instance, if a property is under 100 years old, 10:06.750 --> 10:11.400 then the older it is the less valuable it is, 10:11.400 --> 10:14.040 but as soon as it jumps over 100 years old, 10:14.040 --> 10:15.870 all of a sudden it becomes a historic property 10:15.870 --> 10:19.140 because this is a property where people used 10:19.140 --> 10:20.820 to live hundreds of years ago. 10:20.820 --> 10:24.090 It tells a story, it's got all this history behind it. 10:24.090 --> 10:26.250 And some people like that, some people value that. 10:26.250 --> 10:28.290 In fact, quite a lot of people would like that 10:28.290 --> 10:32.100 and would be proud to live in a property especially 10:32.100 --> 10:34.350 in the higher socioeconomic classes, 10:34.350 --> 10:37.020 they would show off to their friends or things like that. 10:37.020 --> 10:40.410 And therefore properties that are over 100 years old, 10:40.410 --> 10:42.300 could be deemed as historic. 10:42.300 --> 10:44.040 And therefore this neural as soon 10:44.040 --> 10:46.380 as it sees a property over 100 years old, 10:46.380 --> 10:49.380 it'll fire up and contribute to the overall price. 10:49.380 --> 10:51.660 And otherwise, if it's under 100 years old, 10:51.660 --> 10:52.800 then it won't even work. 10:52.800 --> 10:56.730 And this is a good example of the rectifier 10:56.730 --> 10:57.600 function being applied. 10:57.600 --> 11:02.600 So here you've got like a zero until a certain point, 11:03.480 --> 11:05.610 let's say 100 years old and then after 100 years old, 11:05.610 --> 11:08.880 the older it gets the higher the contribution 11:08.880 --> 11:11.430 of this neural to the overall price. 11:11.430 --> 11:15.720 And it's just a very simple example 11:15.720 --> 11:18.033 of this rectifier function in action. 11:18.870 --> 11:19.740 So there we go. 11:19.740 --> 11:21.060 That could be this neural. 11:21.060 --> 11:24.660 And moreover, the neural network could have even picked 11:24.660 --> 11:28.500 up things that we wouldn't have thought of ourselves, right? 11:28.500 --> 11:30.900 For instance, bedrooms plus distance to the city, 11:30.900 --> 11:34.170 maybe that's in combination somehow contributes 11:34.170 --> 11:35.003 to the price. 11:35.003 --> 11:36.810 Maybe it's not as strong as the other neurals 11:36.810 --> 11:38.250 and it contributes, but it still contributes. 11:38.250 --> 11:39.843 Or maybe it detracts from the price. 11:39.843 --> 11:42.570 That could also be the case or other things like that. 11:42.570 --> 11:45.780 And maybe a neural picked up a combination 11:45.780 --> 11:47.340 of all four of these parameters. 11:47.340 --> 11:51.030 And as you can see that these neurals, 11:51.030 --> 11:53.430 this whole hidden layer situation allows you 11:53.430 --> 11:58.430 to increase the flexibility of your neural network 11:58.560 --> 12:02.250 and allows the neural network to look 12:02.250 --> 12:06.030 for very specific things and then in combination, 12:06.030 --> 12:08.220 that's where the power comes from. 12:08.220 --> 12:09.690 It's like that example of the ants, right? 12:09.690 --> 12:12.570 Like an ant by itself cannot build in anthill, 12:12.570 --> 12:14.070 but when you have like a thousand 12:14.070 --> 12:16.020 or a hundred thousand ants, they can build 12:16.020 --> 12:17.160 an anthill together, right? 12:17.160 --> 12:18.630 And that's the situation here. 12:18.630 --> 12:20.550 Each one of these neurals by itself cannot predict 12:20.550 --> 12:24.090 the price, but together they have superpowers 12:24.090 --> 12:27.240 and they predict the price and they can do quite 12:27.240 --> 12:30.690 an accurate job if trained properly, if set up properly. 12:30.690 --> 12:32.070 And that's what this whole course 12:32.070 --> 12:35.340 is about in understanding how to utilize them. 12:35.340 --> 12:36.173 There we go. 12:36.173 --> 12:39.870 So that is a step by step example and walk through 12:39.870 --> 12:42.420 of how neural networks actually work. 12:42.420 --> 12:43.770 I hope you enjoyed today's tutorial 12:43.770 --> 12:45.570 and I can't wait to see you next time. 12:45.570 --> 12:47.523 Until then, enjoy deep learning.