WEBVTT

00:00.810 --> 00:02.130
-: Hello and welcome back to the course

00:02.130 --> 00:04.020
on artificial intelligence.

00:04.020 --> 00:05.010
In today's section,

00:05.010 --> 00:08.550
we're going to tackle deep convolutional Q-learning.

00:08.550 --> 00:10.770
So we're taking deep Q-learning

00:10.770 --> 00:12.360
to even a further step.

00:12.360 --> 00:14.760
So we originally started with Q-learning,

00:14.760 --> 00:15.660
the simple Q-learning.

00:15.660 --> 00:18.090
Then we took that to deep Q-learning.

00:18.090 --> 00:20.880
And now we're taking it to deep convolutional Q-learning.

00:20.880 --> 00:22.440
So let's see what we're going to discuss

00:22.440 --> 00:23.910
in terms of intuition.

00:23.910 --> 00:25.915
The intuition's section gonna be quite quick.

00:25.915 --> 00:28.620
This is not much that we need to add

00:28.620 --> 00:30.600
as long as we're familiar

00:30.600 --> 00:33.570
with convolutional neural networks,

00:33.570 --> 00:35.070
and we'll touch on this towards the end

00:35.070 --> 00:36.120
of today's tutorial.

00:36.120 --> 00:38.340
So today in this section,

00:38.340 --> 00:40.620
we're going to talk about deep convolutional Q-learning,

00:40.620 --> 00:41.760
the intuition behind things

00:41.760 --> 00:43.890
and why it's so powerful,

00:43.890 --> 00:45.990
why exactly it's so important

00:45.990 --> 00:49.590
to move away from deep Q-learning

00:49.590 --> 00:52.980
and why deep Q-learning is just a basic building block

00:52.980 --> 00:55.500
where it's just a step for us on the path

00:55.500 --> 00:56.820
to deep convolutional Q-learning,

00:56.820 --> 00:58.260
and what kind of avenues

00:58.260 --> 01:01.290
deep convolutional Q-learning opens up to,

01:01.290 --> 01:04.650
what kind of avenues the knowledge opens up to,

01:04.650 --> 01:07.470
and where it can be applied.

01:07.470 --> 01:09.300
We'll have some examples of that.

01:09.300 --> 01:12.000
And then we'll talk about eligibility trace

01:12.000 --> 01:13.200
or in-step Q-learning,

01:13.200 --> 01:17.460
a very powerful addition to the whole concept

01:17.460 --> 01:19.800
of deep Q-learning.

01:19.800 --> 01:21.840
And we'll talk about the intuition behind that.

01:21.840 --> 01:23.010
It's quite a complex topic

01:23.010 --> 01:24.960
but nevertheless, we'll break down the intuition

01:24.960 --> 01:25.860
in quite simple terms

01:25.860 --> 01:29.160
and then I'll give you some additional references

01:29.160 --> 01:32.070
where you can read up about eligibility traits

01:32.070 --> 01:33.600
if you'd like to go into more detail.

01:33.600 --> 01:36.390
But it is important for us to get the intuition down pat

01:36.390 --> 01:37.800
because we're going to be using that

01:37.800 --> 01:38.970
in the practical terms

01:38.970 --> 01:43.080
because we're delving into much more complex topics now

01:43.080 --> 01:47.560
that we need to add these extra elements

01:48.720 --> 01:52.500
to our agents or to our Q-learning algorithms

01:52.500 --> 01:56.190
so that they can actually handle these complex environments

01:56.190 --> 01:58.470
and navigate them successfully.

01:58.470 --> 02:01.260
And of course, in this section,

02:01.260 --> 02:04.710
because we're talking about convolutional neural networks,

02:04.710 --> 02:07.440
it is highly advisable that you check out annex number two,

02:07.440 --> 02:09.270
convolutional neural networks.

02:09.270 --> 02:13.140
Once again, if you've done the Deep Learning A to Z course,

02:13.140 --> 02:16.470
then you're already familiar with this information.

02:16.470 --> 02:18.770
So you can safely proceed with these tutorials

02:19.680 --> 02:21.570
on the deep convolutional Q-learning.

02:21.570 --> 02:24.480
if you haven't done the Deep Learning A to Z course,

02:24.480 --> 02:26.490
then it's a great idea

02:26.490 --> 02:28.500
to look at convolutional neural networks

02:28.500 --> 02:30.000
and look at those intuition tutorials there

02:30.000 --> 02:31.800
so you understand better

02:31.800 --> 02:35.700
how images are processed by neural networks

02:35.700 --> 02:38.130
in order to look for features,

02:38.130 --> 02:41.100
and what's the whole convolutional layers

02:41.100 --> 02:43.140
or about the pooling layers,

02:43.140 --> 02:44.310
the flattening layers

02:44.310 --> 02:46.230
and how all that works in order to come up

02:46.230 --> 02:51.210
with some parameters about that describe the environment

02:51.210 --> 02:52.770
or that describe that image.

02:52.770 --> 02:54.540
And therefore, we're gonna be using those

02:54.540 --> 02:56.730
as inputs into our neural network

02:56.730 --> 02:58.290
instead of that vector, which we're talking about.

02:58.290 --> 03:00.030
But more on that in the next tutorial.

03:00.030 --> 03:02.476
So if you haven't seen those tutorials yet,

03:02.476 --> 03:05.520
we highly advise you to check them out

03:05.520 --> 03:06.600
to get up to speed with

03:06.600 --> 03:10.500
or refresh your knowledge on convolutional neural networks.

03:10.500 --> 03:12.270
All in all, we've got an exciting section

03:12.270 --> 03:14.280
and as you can see quite

03:14.280 --> 03:15.780
not that many intuition tutorials,

03:15.780 --> 03:17.580
meaning that you'll be able

03:17.580 --> 03:22.170
to jump into the practical side of things very quickly.

03:22.170 --> 03:23.520
So on that note, I can't wait

03:23.520 --> 03:24.420
to see you on the first tutorial,

03:24.420 --> 03:26.913
and until then, enjoy AI.