WEBVTT 00:00.360 --> 00:04.140 -: Hello and welcome again to module two, Doom. 00:04.140 --> 00:05.520 We will now get us ready 00:05.520 --> 00:08.010 to start the implementation of our AI. 00:08.010 --> 00:10.290 And as usual, the first thing that we need 00:10.290 --> 00:12.060 to do is set the right folder 00:12.060 --> 00:13.590 as working directory. 00:13.590 --> 00:14.700 So, let's do this now so 00:14.700 --> 00:17.550 that we can move onto what's more interesting. 00:17.550 --> 00:20.340 So, as usual, I start on my desktop, 00:20.340 --> 00:23.430 then I go to my artificial intelligence exit folder. 00:23.430 --> 00:25.650 Then module two, now Doom. 00:25.650 --> 00:27.990 And there we go, that's the folder we have to set 00:27.990 --> 00:29.490 as working directory. 00:29.490 --> 00:30.450 So, let's do this now. 00:30.450 --> 00:32.340 We click on this tool button here, 00:32.340 --> 00:35.070 then restart kernel, and then, yes. 00:35.070 --> 00:36.780 And here we go, we now have 00:36.780 --> 00:39.450 the right folder as working directory. 00:39.450 --> 00:42.360 So, now as you can see, we have four files, 00:42.360 --> 00:44.340 well actually, three files in one folder 00:44.340 --> 00:46.230 in this working directory folder. 00:46.230 --> 00:47.610 So, let's start with the first one. 00:47.610 --> 00:50.250 The first one is ai.py. 00:50.250 --> 00:52.650 That's of course, the file that will contain 00:52.650 --> 00:54.510 our artificial intelligence, 00:54.510 --> 00:56.880 and that's nothing else than this file here. 00:56.880 --> 00:59.540 That is the ai.py file in which, 00:59.540 --> 01:02.280 we will implement everything that is related 01:02.280 --> 01:05.670 to building an AI, and especially building an AI 01:05.670 --> 01:09.150 with the deep convolutional key learning model. 01:09.150 --> 01:10.740 So, basically, that's where we'll have 01:10.740 --> 01:12.210 the big adventure. 01:12.210 --> 01:13.440 Then we have some other files. 01:13.440 --> 01:14.850 So, we have the second file here, 01:14.850 --> 01:17.430 which is experience replay.py. 01:17.430 --> 01:18.780 And so, this time, 01:18.780 --> 01:21.180 I put experience replay separately, 01:21.180 --> 01:23.640 just because we already implemented it 01:23.640 --> 01:25.950 and now, we want to focus on what's new. 01:25.950 --> 01:27.960 And trust me, we have a lot of new things 01:27.960 --> 01:31.080 to do with this new artificial intelligence 01:31.080 --> 01:33.810 because not only we want to build an AI, 01:33.810 --> 01:36.660 but we want to build an AI to beat Doom. 01:36.660 --> 01:38.310 So, you can imagine that 01:38.310 --> 01:41.580 this will require some quite advanced code. 01:41.580 --> 01:43.950 So, no worries, we have a big code waiting 01:43.950 --> 01:46.830 for us and you will learn a lot of new tricks. 01:46.830 --> 01:49.380 That's why this experience replay trick 01:49.380 --> 01:51.960 that you already know and that I remind, 01:51.960 --> 01:53.610 improves a lot the training. 01:53.610 --> 01:55.020 Well, let's put it separately 01:55.020 --> 01:57.450 in this experience replay.py file 01:57.450 --> 01:59.550 so that we can now focus on all 01:59.550 --> 02:01.320 the new concepts, techniques, and tricks 02:01.320 --> 02:03.030 that are waiting for us. 02:03.030 --> 02:04.140 All right, and then, we have 02:04.140 --> 02:06.930 the image pre-processing.py file. 02:06.930 --> 02:08.940 So, that's another Python file, 02:08.940 --> 02:11.850 which will take care of pre-processing our images 02:11.850 --> 02:14.880 because you know this time our AI will have eyes. 02:14.880 --> 02:16.890 And that's because the input states 02:16.890 --> 02:19.560 are no longer encoded by a vector, but this time, 02:19.560 --> 02:21.540 the input states are the images. 02:21.540 --> 02:24.540 So, the first layer of the big neural network 02:24.540 --> 02:26.550 that we will make will be the eyes, 02:26.550 --> 02:28.440 and that will be the convolutional layers 02:28.440 --> 02:30.450 of the convolutional neural network. 02:30.450 --> 02:33.360 But to make sure these images can be accepted 02:33.360 --> 02:35.820 as inputs of the convolutional neural network, 02:35.820 --> 02:37.890 well, we need to pre-process them. 02:37.890 --> 02:39.960 And so, this file will take care 02:39.960 --> 02:42.090 of pre-processing these images so 02:42.090 --> 02:44.190 that they can go into the neural network. 02:44.190 --> 02:45.840 And so, I separated this file 02:45.840 --> 02:48.960 because this is not directly related to AI, 02:48.960 --> 02:51.660 and again, we want to keep the maximum 02:51.660 --> 02:54.300 of our brain, and our memory, and our focus 02:54.300 --> 02:56.640 on everything that is related to AI. 02:56.640 --> 02:58.560 So, we are putting this separately 02:58.560 --> 03:00.630 so that we can pre-process the images 03:00.630 --> 03:02.820 in a flashlight, and save some energy 03:02.820 --> 03:03.870 for the rest. 03:03.870 --> 03:05.550 You can have a look at it if you want, 03:05.550 --> 03:07.710 and if you also have the deep learning course, 03:07.710 --> 03:09.090 where you can have a look 03:09.090 --> 03:10.530 at the practical tutorials; 03:10.530 --> 03:12.780 we talk about image pre-processing, 03:12.780 --> 03:15.420 but here again, we really wanna focus on AI. 03:15.420 --> 03:17.820 Trust me, we have a lot to do. 03:17.820 --> 03:19.860 And eventually, the last folder, 03:19.860 --> 03:21.720 well that's the videos folder. 03:21.720 --> 03:23.760 So, right now, this folder is empty 03:23.760 --> 03:26.910 as you can see, but when we execute the code, 03:26.910 --> 03:29.670 some videos of the AI playing Doom 03:29.670 --> 03:31.290 will be added to this folder. 03:31.290 --> 03:32.610 So, that will be very exciting 03:32.610 --> 03:35.010 because we will see on some videos 03:35.010 --> 03:36.840 how well the AI is doing. 03:36.840 --> 03:39.090 So, we will literally see the AI killing 03:39.090 --> 03:42.180 the monsters and trying to run towards the goal. 03:42.180 --> 03:44.550 So, you're gonna see this will be pretty exciting. 03:44.550 --> 03:47.190 So, of course, the first videos will be very bad 03:47.190 --> 03:50.370 because the AI will not be trained much yet, 03:50.370 --> 03:52.410 and so it will get killed very fast. 03:52.410 --> 03:54.960 But then, you will see that as the training 03:54.960 --> 03:57.420 is progressing, while the AI will get better 03:57.420 --> 04:00.000 and better, and eventually, it will manage 04:00.000 --> 04:02.400 to kill some monsters, not getting killed. 04:02.400 --> 04:04.500 And hopefully, we will be able to make 04:04.500 --> 04:06.630 it reach the goal. 04:06.630 --> 04:08.610 All right, so let's go back 04:08.610 --> 04:10.830 to our AI file, which is this one. 04:10.830 --> 04:13.410 And as you can see, I already took care 04:13.410 --> 04:15.600 of importing all the essential libraries 04:15.600 --> 04:18.030 and packages that we need to play Doom. 04:18.030 --> 04:20.160 So, let's quickly have a look at them one by one. 04:20.160 --> 04:21.600 We have of course NumPy 04:21.600 --> 04:22.950 because we'll be working 04:22.950 --> 04:25.710 with arrays, that's inevitable. 04:25.710 --> 04:27.030 Then we have Torch, of course, 04:27.030 --> 04:29.550 because we're implementing the AI with PyTorch. 04:29.550 --> 04:31.680 Then we have the torch.nn module, 04:31.680 --> 04:33.930 which contains all the tools to implement 04:33.930 --> 04:35.190 a neural network. 04:35.190 --> 04:36.690 So, for example, the nn module 04:36.690 --> 04:38.880 will contain the convolutional layers 04:38.880 --> 04:41.760 that will be part of our future neural network. 04:41.760 --> 04:44.790 Then we have the nn.functional package, 04:44.790 --> 04:46.740 which has the shortcuts F 04:46.740 --> 04:49.140 and that contains all the functions 04:49.140 --> 04:50.940 that are used in a neural network. 04:50.940 --> 04:53.730 So, typically the activation functions. 04:53.730 --> 04:54.563 We will be using 04:54.563 --> 04:57.150 some rectify activation functions, but also, 04:57.150 --> 04:58.920 some max pulling function 04:58.920 --> 05:00.960 for the convolutional neural network. 05:00.960 --> 05:02.010 And all these functions 05:02.010 --> 05:04.200 are contained in functional. 05:04.200 --> 05:06.450 Then we have Optim, which is of course, 05:06.450 --> 05:07.890 for our optimizer. 05:07.890 --> 05:10.830 I think, we'll be using an atom optimizer 05:10.830 --> 05:14.070 and this optimizer is contained in Optim. 05:14.070 --> 05:17.310 And then, the best of the best of PyTorch, 05:17.310 --> 05:20.400 the variable class from the autograph module. 05:20.400 --> 05:22.230 And that's all the power of PyTorch 05:22.230 --> 05:25.050 because that's what contains the dynamic graphs, 05:25.050 --> 05:27.330 allowing to perform very fast computations 05:27.330 --> 05:29.130 of the gradient, even the gradient 05:29.130 --> 05:30.810 of composition functions. 05:30.810 --> 05:32.850 So, we will definitely be using it, 05:32.850 --> 05:34.500 as for the self-driving car, 05:34.500 --> 05:36.810 but trust me, for Doom we will be needing 05:36.810 --> 05:38.220 it very bad. 05:38.220 --> 05:41.040 Okay, so that's all for the essential libraries. 05:41.040 --> 05:44.430 Then we need to import some packages related 05:44.430 --> 05:46.800 to open AI Gym and Doom. 05:46.800 --> 05:48.780 So, of course, we import Gym, 05:48.780 --> 05:51.330 then we import some wrappers module 05:51.330 --> 05:53.010 of the Gym library. 05:53.010 --> 05:55.380 And one of these wrappers is kip wrapper. 05:55.380 --> 05:57.480 So, that's basically to import all the tools 05:57.480 --> 05:59.190 and environments of Gym. 05:59.190 --> 06:01.320 And finally, we have this package 06:01.320 --> 06:02.820 that we need to import, and which 06:02.820 --> 06:04.500 is directly related to Doom. 06:04.500 --> 06:07.770 And that's the action space and two discretes 06:07.770 --> 06:10.710 of the Doom wrapper that basically contains 06:10.710 --> 06:13.590 the environment of Doom, and more specifically, 06:13.590 --> 06:15.330 the actions that can be played, 06:15.330 --> 06:16.530 the number of actions 06:16.530 --> 06:19.710 for the specific Doom game we're gonna play. 06:19.710 --> 06:21.720 And I remind that there are six actions. 06:21.720 --> 06:23.490 Move left, move right, turn left, 06:23.490 --> 06:27.420 turn right, move forward, and shoot, attack. 06:27.420 --> 06:29.010 All right, so that's basically 06:29.010 --> 06:30.780 what you need to import for Doom. 06:30.780 --> 06:32.850 And then finally, we of course, need 06:32.850 --> 06:36.480 to import our two internal files, 06:36.480 --> 06:39.270 experience replay.py for experience replay, 06:39.270 --> 06:42.300 and image pre-processing to pre-process 06:42.300 --> 06:44.280 the images that are nothing else 06:44.280 --> 06:46.050 than the images of the screen 06:46.050 --> 06:47.340 when playing the game. 06:47.340 --> 06:50.100 And these images will be pre-processed, 06:50.100 --> 06:52.620 converted into NumPy arrays, reshaped 06:52.620 --> 06:54.690 to a certain format, and then they will go 06:54.690 --> 06:55.740 to the neural network, 06:55.740 --> 06:58.260 the convolutional neural network. 06:58.260 --> 07:01.290 All right, so I guess, now that we are ready 07:01.290 --> 07:06.180 to start the big implementation of this AI, 07:06.180 --> 07:07.950 and now, it's very important for me 07:07.950 --> 07:09.690 to tell you that, that leads me 07:09.690 --> 07:12.450 to the very important point of this module. 07:12.450 --> 07:15.030 It's that since you know I told you we have 07:15.030 --> 07:17.460 a big implementation waiting for us, 07:17.460 --> 07:20.010 well in order to not get lost in all this, 07:20.010 --> 07:21.900 we need a good structure. 07:21.900 --> 07:25.140 And so, I already highlighted this structure. 07:25.140 --> 07:27.990 We will be implementing this in two parts. 07:27.990 --> 07:31.080 The first part will be about building the AI. 07:31.080 --> 07:33.600 So, that's where we will make the brain 07:33.600 --> 07:34.650 of the AI. 07:34.650 --> 07:36.210 And the brain, as you understood, 07:36.210 --> 07:38.340 is nothing else than the neural network. 07:38.340 --> 07:41.280 You know this big CNN composed 07:41.280 --> 07:43.590 of some convolutional layers 07:43.590 --> 07:45.270 and then, some fully connected layers 07:45.270 --> 07:46.740 to predict the outputs 07:46.740 --> 07:48.360 that are still the Q values. 07:48.360 --> 07:50.520 And then, we'll make the body of the AI. 07:50.520 --> 07:52.200 And that's a new representation 07:52.200 --> 07:54.330 I'm bringing to you, and that's again, 07:54.330 --> 07:55.590 not to get lost. 07:55.590 --> 07:57.330 You will see that the further 07:57.330 --> 07:58.920 we progress with the code, 07:58.920 --> 08:00.690 the more you will see the structure 08:00.690 --> 08:02.850 and everything will make sense in the end. 08:02.850 --> 08:04.200 And to make sure that makes sense, 08:04.200 --> 08:06.660 we need a representation of the AI, 08:06.660 --> 08:10.320 and basically, this first part, building the AI 08:10.320 --> 08:12.840 will be composed of three sections. 08:12.840 --> 08:15.750 First section will be about making the brain 08:15.750 --> 08:17.400 that is the neural network. 08:17.400 --> 08:19.500 The second section will be about making 08:19.500 --> 08:21.480 the body, and I'm calling it the body 08:21.480 --> 08:22.710 because this is the part 08:22.710 --> 08:25.170 that will tell the AI how to play the action. 08:25.170 --> 08:26.010 So, you know first you have 08:26.010 --> 08:28.650 the brain that detects the images 08:28.650 --> 08:30.660 and predicts the Q values, 08:30.660 --> 08:32.640 but then you need to specify how the AI 08:32.640 --> 08:34.050 should play the action. 08:34.050 --> 08:36.180 And that, it does it with its body, 08:36.180 --> 08:38.040 like a human body would do. 08:38.040 --> 08:40.110 So, the body will be the part 08:40.110 --> 08:42.360 where we will specify the method 08:42.360 --> 08:44.010 of playing the action. 08:44.010 --> 08:46.320 So, for example, with our self-driving car, 08:46.320 --> 08:48.750 the brain was the neural network we made 08:48.750 --> 08:51.540 and the body was how the action was played. 08:51.540 --> 08:53.670 That is with the submax method. 08:53.670 --> 08:54.720 In here, that's the same. 08:54.720 --> 08:56.310 We're gonna make a brain, 08:56.310 --> 08:57.630 and we're gonna make a buddy, 08:57.630 --> 08:59.130 which will play the action. 08:59.130 --> 09:00.510 I will let you find out, 09:00.510 --> 09:02.010 but the key point in all this 09:02.010 --> 09:04.650 is that we will have a very structured code 09:04.650 --> 09:06.900 so that not only you can take a step back, 09:06.900 --> 09:08.700 and really understand what's going on. 09:08.700 --> 09:10.800 But also, you will be able to use it 09:10.800 --> 09:12.840 as a framework whenever you want to build 09:12.840 --> 09:15.360 an AI for other purposes. 09:15.360 --> 09:18.540 All right, and after building the AI in part one, 09:18.540 --> 09:20.070 we will move onto part two, 09:20.070 --> 09:22.080 which will be about implementing 09:22.080 --> 09:24.510 the deep convolutional key learning model. 09:24.510 --> 09:26.820 And there again, we'll have different sections, 09:26.820 --> 09:28.350 and one of them will be, of course, 09:28.350 --> 09:30.180 to train the AI. 09:30.180 --> 09:32.550 So, I can't wait to dive into it now. 09:32.550 --> 09:35.250 We're gonna start with part one, obviously, 09:35.250 --> 09:37.170 and we're gonna start by making 09:37.170 --> 09:39.300 the brain of our AI. 09:39.300 --> 09:42.303 So, I can't wait, and until then, enjoy AI.