WEBVTT 00:00.180 --> 00:01.740 -: Hello my friends and welcome 00:01.740 --> 00:04.170 to this new section on the visualization 00:04.170 --> 00:07.920 of the results of the deep convolutional tutoring model. 00:07.920 --> 00:10.980 First, I really wanna say huge congratulations 00:10.980 --> 00:13.143 for having completed this whole implementation 00:13.143 --> 00:15.960 of the deep convolutional tutoring model. 00:15.960 --> 00:18.870 And now here we are at the front part where 00:18.870 --> 00:22.303 we're gonna visualize the results of the AI we created. 00:22.303 --> 00:25.590 And as I said at the end of the previous tutorial, you know 00:25.590 --> 00:27.870 the last tutorial of the implementation, 00:27.870 --> 00:29.754 I said that I had a surprise for you. 00:29.754 --> 00:32.790 And well, the surprise is exactly this. 00:32.790 --> 00:36.300 We're gonna visualize the results on Google CoLab. 00:36.300 --> 00:38.310 And why is that a good surprise? 00:38.310 --> 00:40.740 That's a good surprise because basically you 00:40.740 --> 00:44.640 will have absolutely no trouble running the code, you know 00:44.640 --> 00:46.852 running the code and visualizing the results. 00:46.852 --> 00:49.710 Indeed, this is a new feature of the course. 00:49.710 --> 00:52.530 You know, before this, we actually used to run the code 00:52.530 --> 00:57.000 on Spider and Python, but it was extremely difficult to 00:57.000 --> 00:59.880 you know, install all the packages, you know, Python torch 00:59.880 --> 01:02.613 the gym dependencies and all the different dependencies. 01:02.613 --> 01:04.740 A lot of students actually had tons 01:04.740 --> 01:07.320 of issues installing everything and at the end 01:07.320 --> 01:09.870 most of them couldn't run while the whole code 01:09.870 --> 01:12.720 and mostly they couldn't see the final results. 01:12.720 --> 01:14.792 So I absolutely wanted to fix this 01:14.792 --> 01:17.250 and really the best solution that I've come 01:17.250 --> 01:19.950 up with was to include the whole code. 01:19.950 --> 01:21.793 So you will have exactly the same code 01:21.793 --> 01:23.853 as what we've implemented in this part, 01:23.853 --> 01:25.890 but I've included the whole code 01:25.890 --> 01:29.370 on a Google CoLab notebook where the only thing 01:29.370 --> 01:31.950 you will have to do to run the code and visualize 01:31.950 --> 01:35.280 the results will be to click these play buttons here. 01:35.280 --> 01:36.630 That's the only thing you have to do 01:36.630 --> 01:39.480 because the beauty of Google CoLab is that most 01:39.480 --> 01:42.270 of the libraries and packages are already installed. 01:42.270 --> 01:44.438 For example, TensorFlow is already installed, 01:44.438 --> 01:46.200 Pie Torch is already installed, 01:46.200 --> 01:47.970 and all the torch modules functions 01:47.970 --> 01:49.950 and classes are already installed 01:49.950 --> 01:51.900 so you don't have to install them. 01:51.900 --> 01:54.750 However, we still have some little dependencies to 01:54.750 --> 01:57.210 install to make sure everything is connected well, 01:57.210 --> 02:00.540 but we won't have to install them through a terminal 02:00.540 --> 02:03.177 or you know, through the Anaconda prompt like we did 02:03.177 --> 02:04.800 in the self-driving car. 02:04.800 --> 02:07.560 We will just have to click this play button here 02:07.560 --> 02:08.550 in this first cell. 02:08.550 --> 02:10.656 Installing system dependencies for VizDoom. 02:10.656 --> 02:13.527 Now I know what some of you might be thinking. 02:13.527 --> 02:16.920 Why didn't we do the same for the self-driving car? 02:16.920 --> 02:19.560 Well, unfortunately we couldn't do the same because 02:19.560 --> 02:23.040 in the self-driving car module we had the user interactions 02:23.040 --> 02:26.130 you know, when we plot some sense on the map to, you know 02:26.130 --> 02:28.440 design the roads and put some obstacles. 02:28.440 --> 02:31.980 Well, unfortunately Google CoLab doesn't have that feature 02:31.980 --> 02:34.770 yet of allowing to have some user interactions 02:34.770 --> 02:35.790 with some map. 02:35.790 --> 02:38.400 So I couldn't do it for the self-driving car. 02:38.400 --> 02:40.917 My apologies for that, but that's why I gave you 02:40.917 --> 02:44.100 the whole chapter of my book on the AI crash course, 02:44.100 --> 02:46.920 which contains all the instructions on the self-driving car. 02:46.920 --> 02:49.230 I hope you could install everything properly. 02:49.230 --> 02:51.494 Anyway, there was not much to install compared to Doom 02:51.494 --> 02:54.090 so I hope everything went well for the self-driving car, 02:54.090 --> 02:55.273 but for Doom here, 02:55.273 --> 02:58.650 now you're covered, you won't have any trouble 02:58.650 --> 03:00.840 running the code and visualizing the results. 03:00.840 --> 03:02.070 So that's the surprise. 03:02.070 --> 03:03.240 I hope you like it. 03:03.240 --> 03:04.410 And now I'm gonna walk you 03:04.410 --> 03:06.240 through this whole implementation to show you 03:06.240 --> 03:08.909 that indeed we have exactly the same code 03:08.909 --> 03:11.518 as what we implemented in this part two. 03:11.518 --> 03:14.700 The only thing is that I merged the three codes. 03:14.700 --> 03:17.700 You know, this is the code we had in the folder, you know 03:17.700 --> 03:20.280 composed of the image, progressing that PY file 03:20.280 --> 03:22.380 then the experience replay that PY file, 03:22.380 --> 03:24.763 and then the AI that PY file with the two parts, 03:24.763 --> 03:28.165 part one building the AI and part two, training the AI. 03:28.165 --> 03:31.560 Well, what I simply did is I merged the three 03:31.560 --> 03:35.580 of them by putting first the image progressing that PY file 03:35.580 --> 03:37.230 I'll show you that in the workbook. 03:37.230 --> 03:41.550 Then second experience, replay that PY file and third 03:41.550 --> 03:43.050 the AI dot PY file. 03:43.050 --> 03:46.470 There are just one below the other in that order. 03:46.470 --> 03:48.450 Okay, so I'll show you this to you. 03:48.450 --> 03:51.300 And then I just had to change two or three things 03:51.300 --> 03:54.210 in this whole code and I'll show you exactly the changes 03:54.210 --> 03:56.910 so that you can make sure to catch the difference. 03:56.910 --> 03:59.880 Basically in that file everything is the same, 03:59.880 --> 04:02.100 in that file everything is the same as well, 04:02.100 --> 04:03.189 absolutely no change. 04:03.189 --> 04:07.440 However, in that file, well here when we import the packages 04:07.440 --> 04:10.650 for open AI and Doom, while we don't use the P package 04:10.650 --> 04:13.500 in Doom rappers action space to import to this Discreet 04:13.500 --> 04:15.090 we don't use that anymore. 04:15.090 --> 04:16.830 We'll use something else you'll see, 04:16.830 --> 04:19.410 and also I changed the dimensions 04:19.410 --> 04:21.759 of the frames here, you know, 80 by 80. 04:21.759 --> 04:25.169 I changed that to 256 by 256 04:25.169 --> 04:28.740 so that you can have better visualization results indeed 04:28.740 --> 04:31.080 because you will have bigger videos to watch. 04:31.080 --> 04:32.580 Okay, that's just what I changed. 04:32.580 --> 04:34.020 I'll show you now, let's start 04:34.020 --> 04:36.968 with image processing because indeed, as I told you 04:36.968 --> 04:39.540 I just merged the three files together. 04:39.540 --> 04:43.315 So we start with indeed image processing.py, and 04:43.315 --> 04:46.080 as you can see, this is exactly the same. 04:46.080 --> 04:48.030 We start by importing the libraries 04:48.030 --> 04:49.710 the exact same ones here. 04:49.710 --> 04:53.490 Then we preces images with the pre-process image class 04:53.490 --> 04:56.820 composed of the init method and the observation method. 04:56.820 --> 04:58.920 And as you can see, this is exactly the same 04:58.920 --> 05:02.610 pre-process image class init method and observation method. 05:02.610 --> 05:03.443 All right? 05:03.443 --> 05:06.957 Then we move on to experience replay where we have exactly 05:06.957 --> 05:10.650 the same indeed we first import the two libraries here known 05:10.650 --> 05:13.290 by named Toplin, right? 05:13.290 --> 05:16.350 Then we define one step exactly the same 05:16.350 --> 05:19.230 as what we have here, and then we make the AI progress 05:19.230 --> 05:22.450 on several end steps with the end step progress class, 05:22.450 --> 05:27.240 which is exactly the same as in the workbook, right? 05:27.240 --> 05:29.370 I just did some copy past here. 05:29.370 --> 05:32.340 We had exactly the same the init method, etu method, 05:32.340 --> 05:34.920 and the reward steps method. 05:34.920 --> 05:35.753 All right? 05:35.753 --> 05:38.460 Then of course we implement experience replay 05:38.460 --> 05:41.610 with the replay memory class, and as you can see 05:41.610 --> 05:44.220 we have exactly the same in the Google CoLab. 05:44.220 --> 05:47.100 Plus you will maybe better appreciate the structure 05:47.100 --> 05:49.620 because I added some text sales so that you know 05:49.620 --> 05:52.527 it can be even nicer to visualize on this workbook. 05:52.527 --> 05:54.420 All right, so replay memory class 05:54.420 --> 05:56.640 init method, sample, batch run steps, 05:56.640 --> 05:59.760 basically exactly the same code. All right? 05:59.760 --> 06:03.900 And then we move on to the third foul, AI for Doom 06:03.900 --> 06:05.940 AI PY containing the two parts. 06:05.940 --> 06:09.300 Part one, building AI and part two, training the AI. 06:09.300 --> 06:10.410 So let me show you 06:10.410 --> 06:12.900 that we have exactly the same on this workbook. 06:12.900 --> 06:15.360 So AI for dom, AI.py, first 06:15.360 --> 06:18.390 we import the libraries, the exact same ones, right? 06:18.390 --> 06:19.440 As you can see, 06:19.440 --> 06:22.650 then here that's the difference I already highlighted when 06:22.650 --> 06:24.877 importing the packages for open AI and Doom. 06:24.877 --> 06:27.900 Well, instead of importing gym wrappers 06:27.900 --> 06:29.070 and then skip wrapper 06:29.070 --> 06:32.040 and P packet to import to discrete, well 06:32.040 --> 06:34.500 what we do is we only have to actually import gym. 06:34.500 --> 06:35.430 That's the same. 06:35.430 --> 06:37.320 Then import (indistinct) gym, 06:37.320 --> 06:39.870 which are the new Doom environments basically 06:39.870 --> 06:42.000 which were really well improved. 06:42.000 --> 06:45.060 And finally, the rappers by the gym library. 06:45.060 --> 06:46.980 All right? So just this difference here, 06:46.980 --> 06:49.350 which will make this thing work on Google CoLab 06:49.350 --> 06:51.480 and also on Python if you want to run it. 06:51.480 --> 06:54.060 And then here, of course, I didn't have to include 06:54.060 --> 06:55.770 that code within Google CoLab simply 06:55.770 --> 06:58.290 because here we import the other files here, you know 06:58.290 --> 06:59.209 because they're separated. 06:59.209 --> 07:02.700 And since I merged the three files on Google CoLab, well 07:02.700 --> 07:04.440 I don't need to do this import. 07:04.440 --> 07:08.400 So here we move on directly to part one, building the AI. 07:08.400 --> 07:11.430 And here, as I told you, we have the exact same code 07:11.430 --> 07:13.920 except for the dimensions here, right? 07:13.920 --> 07:17.370 If we look at part one here, part one, building AI 07:17.370 --> 07:21.120 we start by making the brain with the CNN class. 07:21.120 --> 07:23.220 And as you can see, we have the exact same code 07:23.220 --> 07:25.950 except for the dimensions of the input frames here 07:25.950 --> 07:29.340 which were when we implemented the code 80 by 80, 07:29.340 --> 07:34.290 but I changed that in Google CoLab to 256 by 256 07:34.290 --> 07:36.150 so that you can have better frames 07:36.150 --> 07:38.490 and therefore better videos to watch, okay? 07:38.490 --> 07:40.590 But all the rest is exactly the same. 07:40.590 --> 07:42.660 So here we make the brain with the CNN class 07:42.660 --> 07:46.560 then we make the body with the soft max body class 07:46.560 --> 07:49.170 so exactly the same as what we have here. 07:49.170 --> 07:51.900 And finally we put the whole thing together 07:51.900 --> 07:54.960 with the AI class containing the brain and the body 07:54.960 --> 07:58.830 and also some methods to perform the actions, right? 07:58.830 --> 08:02.100 Exactly the same AI class composed of the brain 08:02.100 --> 08:04.320 and the body, and then the same method to 08:04.320 --> 08:06.270 perform the actions. All right? 08:06.270 --> 08:08.040 So exactly the same, just the dimensions 08:08.040 --> 08:09.452 of the input frames changed. 08:09.452 --> 08:12.260 And now let's move on to part two, which will be also 08:12.260 --> 08:13.800 exactly the same. 08:13.800 --> 08:16.710 First we get the Doom environment, right? 08:16.710 --> 08:18.373 Let me show you part two here as well. 08:18.373 --> 08:22.170 We get first the Doom environment, and here, remember that 08:22.170 --> 08:26.580 since we no longer use P packet to import the environment 08:26.580 --> 08:29.520 but instead VizDoom, well, the only thing that changed 08:29.520 --> 08:31.890 in that code is indeed this here and indeed we 08:31.890 --> 08:33.120 get VizDoom corridor, 08:33.120 --> 08:35.567 but this is exactly the same environment. 08:35.567 --> 08:39.300 Alright? And then of course I change these new dimensions so 08:39.300 --> 08:40.680 that you can have bigger videos here. 08:40.680 --> 08:45.403 So we no longer have 80 by 80, but this time 256 by 256 08:46.350 --> 08:48.784 and then we have executive same, we get the Doom of 08:48.784 --> 08:52.895 and the number of actions, which we will see will be seven. 08:52.895 --> 08:56.610 Then we build the AI by creating the brain object, you know, 08:56.610 --> 08:59.340 CNN then the body object, soft mac body 08:59.340 --> 09:00.780 and then the whole AI composed 09:00.780 --> 09:03.390 of the brain and the body exactly the same 09:03.390 --> 09:05.910 as what we implemented together. 09:05.910 --> 09:07.980 All right? Then we set up experience replace. 09:07.980 --> 09:09.930 So that's exactly the same here. 09:09.930 --> 09:12.600 And then we implement eligibility trays. 09:12.600 --> 09:15.600 Still executive same. And then what do we do? 09:15.600 --> 09:17.370 Well, we make the moving average 09:17.370 --> 09:20.400 on 100 steps exactly the same here. 09:20.400 --> 09:22.200 So that's the exact same code. 09:22.200 --> 09:24.360 Basically only the dimensions of frames 09:24.360 --> 09:28.110 and the way to import the environment changed. 09:28.110 --> 09:30.570 Okay? And finally, we train the AI 09:30.570 --> 09:33.300 with exactly the same code. 09:33.300 --> 09:38.115 Okay? So until here we have the whole code that you saw 09:38.115 --> 09:40.890 in the folder when you downloaded the dataset 09:40.890 --> 09:41.970 and code again. 09:41.970 --> 09:43.860 Basically we also have the whole code 09:43.860 --> 09:45.600 that we implemented together. 09:45.600 --> 09:49.020 However, then to make the visualization work 09:49.020 --> 09:52.480 on Google CoLab, well, I added this extra code, but 09:52.480 --> 09:56.970 since it is really only specific to this CoLab notebook 09:56.970 --> 10:00.187 and also only specific to visualize the AI 10:00.187 --> 10:03.540 in action in the Doom environment, well please 10:03.540 --> 10:04.650 you don't have to worry 10:04.650 --> 10:06.990 about understanding all the codes that is here 10:06.990 --> 10:09.090 because you know you will only see that once 10:09.090 --> 10:10.950 in your life when you want to build 10:10.950 --> 10:13.590 and train some AI to play in the Doom environment. 10:13.590 --> 10:15.750 What I mean is that you won't have to use this code 10:15.750 --> 10:18.240 for the other applications you'll be working 10:18.240 --> 10:19.950 on when building and training in A.I.. 10:19.950 --> 10:21.490 So don't worry too much about this code 10:21.490 --> 10:24.900 I just added that to make it work on Google CoLab. 10:24.900 --> 10:26.160 But let's see what I do. 10:26.160 --> 10:28.260 I first import libraries, of course 10:28.260 --> 10:30.630 then I added some practical codes here 10:30.630 --> 10:33.210 so that you can very quickly see the number 10:33.210 --> 10:36.660 of actions you have in the environment, as well as a display 10:36.660 --> 10:39.236 of the frame of the environment, just to see how it is like. 10:39.236 --> 10:41.910 And the reason I added that is because I'm also 10:41.910 --> 10:45.090 gonna show you how to play with other Doom environments 10:45.090 --> 10:46.980 because you actually have many of them. 10:46.980 --> 10:49.140 And so what you will simply have to do is 10:49.140 --> 10:51.090 replace the VizDoom corridor 10:51.090 --> 10:54.180 by the name of the other environment, which are here. 10:54.180 --> 10:55.290 I'm gonna give you the link 10:55.290 --> 10:57.540 of that page right after to this tutorial. 10:57.540 --> 11:00.510 But here, as you can see, you have many Doom environments 11:00.510 --> 11:01.343 that you can play with. 11:01.343 --> 11:04.410 You know, VizDoom basics, VizDoom quarter, which is the one 11:04.410 --> 11:07.260 we're gonna play with right now, VizDoom Defense Center 11:07.260 --> 11:09.960 and others. You even have some indoor mazes. 11:09.960 --> 11:11.970 And just follow the instructions here. 11:11.970 --> 11:13.832 You will find them all in this folder. 11:13.832 --> 11:16.440 You know, in this in it method, you can just 11:16.440 --> 11:19.440 find the name here, VizDoom Basics, VizDoom Corridor 11:19.440 --> 11:22.050 VizDoom Defense Center, and you have many, many 11:22.050 --> 11:22.883 of them you see. 11:22.883 --> 11:25.260 So if you wanna play with other Doom environments 11:25.260 --> 11:26.130 feel free to do that. 11:26.130 --> 11:29.910 You will just have to change the name of the environment. 11:29.910 --> 11:32.580 Anytime you see VizDoom Corridor here on this workbook 11:32.580 --> 11:34.350 you just replace that by the name. 11:34.350 --> 11:36.810 And thanks to this code and this one, while you 11:36.810 --> 11:40.230 will be quickly able to see what is the number of actions 11:40.230 --> 11:43.380 the dimensions of the input shape, and also a display 11:43.380 --> 11:45.900 of the frame to see how the environment looks like. 11:45.900 --> 11:47.670 So this will be pretty practical. 11:47.670 --> 11:49.890 Then I just made this helper function for the 11:49.890 --> 11:52.830 visualization. Then I run the AI on 11:52.830 --> 11:55.680 one episode, of course, after it is trained. 11:55.680 --> 11:59.280 So we will first train it here, as you can see, by the way 11:59.280 --> 12:00.113 I forgot to tell you 12:00.113 --> 12:02.910 that I also changed the number of epochs from 100 to 20. 12:02.910 --> 12:04.200 And that's simply because, you know 12:04.200 --> 12:06.810 I increased the dimensions of the frame. 12:06.810 --> 12:08.820 And since I increased dimensions of the frame 12:08.820 --> 12:11.220 well the training will take much longer 12:11.220 --> 12:12.960 because of course it's harder to train. 12:12.960 --> 12:15.030 And therefore I reduced the number 12:15.030 --> 12:17.820 of 100 epochs down to 20, okay? 12:17.820 --> 12:18.930 And don't worry with that 12:18.930 --> 12:21.060 we will already get well positive rewards. 12:21.060 --> 12:23.310 But of course, if you want to improve the performance 12:23.310 --> 12:24.780 and improve the results in the end 12:24.780 --> 12:26.850 feel free to increase that number of epochs. 12:26.850 --> 12:28.350 You will get even better results. 12:28.350 --> 12:30.480 But you'll see that already with 20 epochs 12:30.480 --> 12:32.460 the training will take quite some time. 12:32.460 --> 12:34.835 Okay? All right, So where was I? 12:34.835 --> 12:37.620 Yes, that helper function for the visualization. 12:37.620 --> 12:40.530 Then I run the AI on one episode, of course 12:40.530 --> 12:41.790 after it is trained. 12:41.790 --> 12:46.015 And finally, this last cell of code will output the video 12:46.015 --> 12:49.045 of the game play, you know, over this one episode. 12:49.045 --> 12:51.360 And you will see how the AI will perform 12:51.360 --> 12:54.630 over this one episode on a video which will be output 12:54.630 --> 12:58.440 and which will be able to download in files here, you know 12:58.440 --> 13:00.000 when we execute this code. 13:00.000 --> 13:03.240 Okay? All right, so now time to the execution. 13:03.240 --> 13:06.450 Time to really sit comfortably in our chair. 13:06.450 --> 13:08.640 I hope you have a good coffee or a good tea. 13:08.640 --> 13:11.490 Now we're just gonna press the different play buttons 13:11.490 --> 13:14.100 in each of the cells to run the final results. 13:14.100 --> 13:15.150 So let's do this. 13:15.150 --> 13:15.983 But first 13:15.983 --> 13:18.690 and that's very important to understand in Google Colab. 13:18.690 --> 13:21.570 So this is the link of a Google CoLab workbook 13:21.570 --> 13:24.840 which you all have access to, and therefore, of course 13:24.840 --> 13:27.150 you can't modify anything in it, because if one 13:27.150 --> 13:30.300 of you modify as well, the others will see the modification. 13:30.300 --> 13:32.580 So this is in read only mode, and therefore 13:32.580 --> 13:33.413 what you have to do 13:33.413 --> 13:35.310 in order to make some modifications, you know 13:35.310 --> 13:37.470 in case you want to change the name of the environment 13:37.470 --> 13:40.900 or increase a number of epochs or even tune your A.I., well 13:40.900 --> 13:44.340 you will have to create a copy of this workbook 13:44.340 --> 13:47.220 which will be in your own Google Drive. 13:47.220 --> 13:49.200 And that's why in order to do that 13:49.200 --> 13:51.030 you will need a Gmail address 13:51.030 --> 13:53.760 in order for you to have a Google Drive, okay? 13:53.760 --> 13:55.590 So make sure to have one. 13:55.590 --> 13:56.910 And once that's the case 13:56.910 --> 13:58.950 well you just need to click file here 13:58.950 --> 14:02.700 and then save a copy and drive, and this will create a copy 14:02.700 --> 14:05.342 of this workbook in your own Google Drive 14:05.342 --> 14:08.700 on which we'll be able to do some modifications. 14:08.700 --> 14:10.680 All right, And now here we are 14:10.680 --> 14:13.020 at the beginning of this execution. 14:13.020 --> 14:15.660 So now we're just gonna play all the cells 14:15.660 --> 14:18.903 and we're gonna get closer and closer to the final results.