WEBVTT

00:00.300 --> 00:02.340
Instructor: Hello and welcome to the level three

00:02.340 --> 00:04.770
of our self-driving car AI game.

00:04.770 --> 00:07.830
So now the challenge is going to be to still do

00:07.830 --> 00:10.650
the round trips between the airports and the downtown

00:10.650 --> 00:13.050
but this time by avoiding obstacles.

00:13.050 --> 00:15.120
And we will draw the obstacles ourselves.

00:15.120 --> 00:17.700
You can draw any difficulty of obstacles.

00:17.700 --> 00:20.880
I'll try not to be too tough with the car

00:20.880 --> 00:22.830
because I want to make it to level four,

00:22.830 --> 00:24.960
but let's keep this challenging.

00:24.960 --> 00:26.580
All right, so as usual,

00:26.580 --> 00:31.470
we're going to select all this code and execute.

00:31.470 --> 00:33.750
And there we go with level three.

00:33.750 --> 00:35.820
So let's load our brain.

00:35.820 --> 00:36.870
Here we go.

00:36.870 --> 00:39.180
Brain is loaded.

00:39.180 --> 00:42.120
The car is doing the round trips properly.

00:42.120 --> 00:44.910
And so now let's draw some obstacles.

00:44.910 --> 00:47.310
All right, so what can we do now?

00:47.310 --> 00:49.833
First, let's draw something like this.

00:52.050 --> 00:53.610
Okay.

00:53.610 --> 00:58.610
And then what we can do is draw another obstacles like this

01:00.690 --> 01:04.203
and then maybe something like this,

01:05.070 --> 01:10.070
something like this, and something like this.

01:10.170 --> 01:11.358
Let's see what it does.

01:11.358 --> 01:14.108
(exciting music)

01:45.093 --> 01:47.490
All right, so now what it's gonna do.

01:47.490 --> 01:49.770
Okay, avoiding this obstacle.

01:49.770 --> 01:52.080
Now, avoiding this one.

01:52.080 --> 01:54.000
No mistake again.

01:54.000 --> 01:55.650
So still it has to learn.

01:55.650 --> 01:59.760
It's still exploring, avoiding this obstacle.

01:59.760 --> 02:01.260
Perfect. Very good.

02:01.260 --> 02:02.760
Is it going to avoid this one?

02:02.760 --> 02:04.590
Great. It did.

02:04.590 --> 02:09.300
And going to the downtown and now going back to the airport.

02:09.300 --> 02:10.830
Avoiding this obstacle.

02:10.830 --> 02:11.700
Great.

02:11.700 --> 02:13.890
Now, is it going to avoid this one this time?

02:13.890 --> 02:16.290
And no, still mistake.

02:16.290 --> 02:17.123
It's okay.

02:17.123 --> 02:18.573
I'll get this thicker.

02:21.270 --> 02:24.660
Maybe something like this will be better.

02:24.660 --> 02:27.570
The car will understand more

02:27.570 --> 02:30.483
because right now it is being quite stubborn.

02:31.320 --> 02:33.510
Alright, still avoiding this obstacle.

02:33.510 --> 02:36.420
It seems not to have any problem with this one

02:36.420 --> 02:38.670
but kind of a problem with that one.

02:38.670 --> 02:39.720
Better now.

02:39.720 --> 02:41.460
You see I got it thicker,

02:41.460 --> 02:43.950
which means that the punishment was harder

02:43.950 --> 02:48.210
with a worse reward and now it managed to avoid it.

02:48.210 --> 02:50.490
And this one avoided as well.

02:50.490 --> 02:51.720
Perfect.

02:51.720 --> 02:55.470
So now we have a functional self-driving car as it seems.

02:55.470 --> 02:58.080
So avoiding this one again, great.

02:58.080 --> 03:00.810
And now avoiding this one.

03:00.810 --> 03:01.650
Great.

03:01.650 --> 03:06.353
Still going onto some sand, but it's okay.

03:06.353 --> 03:08.520
It will be punished for that anyway.

03:08.520 --> 03:09.990
Avoiding this one.

03:09.990 --> 03:12.187
So sorry, this one was totally unuseful

03:12.187 --> 03:17.187
but you know we can do something like this if you want,

03:17.314 --> 03:21.393
to get this even more challenging for the car.

03:22.290 --> 03:23.343
All right.

03:24.750 --> 03:29.750
And okay, we can add a big tip here so that it doesn't cheat

03:32.640 --> 03:36.960
because it's still crossing the tip of the obstacle.

03:36.960 --> 03:39.720
And we can do the same for the other tips

03:39.720 --> 03:42.210
of the other obstacles.

03:42.210 --> 03:44.220
There we go.

03:44.220 --> 03:45.603
There we go.

03:46.740 --> 03:48.123
There we go.

03:54.624 --> 03:56.041
Okay, better now.

04:01.650 --> 04:03.000
Let's see what it does now.

04:03.930 --> 04:08.730
Okay, there we go again, trying to reach the airport now.

04:08.730 --> 04:10.710
Okay, perfect.

04:10.710 --> 04:12.330
So it's going around the obstacle

04:12.330 --> 04:17.330
it's not trying to find the best path,

04:17.340 --> 04:22.340
but anyway, the goal is to reach the two goals:

04:22.770 --> 04:24.450
the downtown and the airports.

04:24.450 --> 04:28.320
Then we can add some code to try to find the best path.

04:28.320 --> 04:33.320
But we definitely already have a self-driving car.

04:33.720 --> 04:35.640
All right, great.

04:35.640 --> 04:37.050
Great job.

04:37.050 --> 04:40.663
And there we go again.

04:40.663 --> 04:42.270
It's still going on to some sand

04:42.270 --> 04:43.320
so it's still being punished.

04:43.320 --> 04:48.320
We can actually look at the score function, which is here.

04:48.540 --> 04:51.360
As you can see, it gets the punishment

04:51.360 --> 04:54.213
and the score function is actually decreasing with time.

04:57.455 --> 05:01.083
So that's because I got the obstacles thicker.

05:05.310 --> 05:06.240
And right now, as you can see,

05:06.240 --> 05:10.200
it's trying to avoid the tips of the obstacles

05:10.200 --> 05:11.160
more and more.

05:11.160 --> 05:13.650
It's doing a better and better job as you can see now.

05:13.650 --> 05:15.979
It's going around faster now.

05:15.979 --> 05:17.160
Except for this one.

05:17.160 --> 05:21.450
But if we save again and look at the score function,

05:21.450 --> 05:25.080
as you can see we see some kind of improvement here.

05:25.080 --> 05:27.840
But anyway, the car is gonna get better with time.

05:27.840 --> 05:30.210
But I think that right now we can say

05:30.210 --> 05:32.160
that level three is passed.

05:32.160 --> 05:35.520
Because anyway, it manages to do the round trips

05:35.520 --> 05:37.470
between the airports and the downtown

05:37.470 --> 05:40.770
by going around the obstacles and that's what we wanted.

05:40.770 --> 05:45.360
So maybe let's move on to level four.

05:45.360 --> 05:47.250
Level four is gonna be very challenging.

05:47.250 --> 05:51.120
I don't think we'll be able to pass level four

05:51.120 --> 05:53.340
but that will be the challenge of this module.

05:53.340 --> 05:56.040
You will have to change something in the code

05:56.040 --> 05:57.720
to pass level four.

05:57.720 --> 05:59.070
It will either be something

05:59.070 --> 06:00.780
that it has to do with the reward

06:00.780 --> 06:05.040
or something that has to do with the strategy of the game

06:05.040 --> 06:08.430
or even something with the neural network

06:08.430 --> 06:12.180
or something with the DQN algorithm.

06:12.180 --> 06:16.380
Well, you will look for it, you will do some research,

06:16.380 --> 06:20.190
and you will try to get an even better car than this one

06:20.190 --> 06:22.680
that avoids any kind of obstacle

06:22.680 --> 06:24.810
that will never go onto some sand

06:24.810 --> 06:28.170
or that will try to find the best way,

06:28.170 --> 06:31.410
the best path, the shortest path.

06:31.410 --> 06:32.970
So good luck with that.

06:32.970 --> 06:35.460
It's gonna be excellent practice for you

06:35.460 --> 06:37.650
and I can't wait to see your solutions.

06:37.650 --> 06:39.690
So now let's move on to level four.

06:39.690 --> 06:42.750
And this time it's gonna be very, very challenging.

06:42.750 --> 06:44.370
So I'll see you in the next tutorial.

06:44.370 --> 06:46.233
And until then, enjoy AI.

06:47.291 --> 06:50.041
(exciting music)