WEBVTT 00:01.110 --> 00:02.310 -: Hello and welcome back to the course 00:02.310 --> 00:03.990 on artificial intelligence. 00:03.990 --> 00:06.120 In today's tutorial, we're going to discuss 00:06.120 --> 00:08.790 the plan of attack for this amazing section 00:08.790 --> 00:13.050 on the most powerful algorithm in artificial intelligence 00:13.050 --> 00:15.540 which is A3C at the moment. 00:15.540 --> 00:16.920 So let's have a look. 00:16.920 --> 00:19.380 Today, we'll talk about the following things. 00:19.380 --> 00:22.590 We'll start with the three A's of A3C, 00:22.590 --> 00:25.200 so we'll talk about what the abbreviation stands for 00:25.200 --> 00:28.770 and how we're going to deal with the three A's. 00:28.770 --> 00:31.320 And also, we'll look at the actual article 00:31.320 --> 00:35.670 or the actual research paper that gave birth to A3C, 00:35.670 --> 00:37.620 the Google DeepMind research paper. 00:37.620 --> 00:39.840 And we'll have a quick look at it 00:39.840 --> 00:41.700 so that you're more comfortable navigating it 00:41.700 --> 00:42.780 in your free time. 00:42.780 --> 00:47.670 Plus, we will see exactly why the A3C is the cutting edge, 00:47.670 --> 00:50.310 most advanced algorithm on the planet 00:50.310 --> 00:52.110 for artificial intelligence at the moment 00:52.110 --> 00:56.790 and why it's so important to have this in your arsenal. 00:56.790 --> 00:59.730 Next, we're going to talk about the first A of A3C, 00:59.730 --> 01:00.870 which is actor-critic. 01:00.870 --> 01:03.060 Well, A3C actually stands for 01:03.060 --> 01:07.080 Asynchronous Advantage Actor-Critic algorithm. 01:07.080 --> 01:09.870 And so we're going through the A's not in order, 01:09.870 --> 01:11.850 but we're going through them in an order 01:11.850 --> 01:12.990 that makes sense for us, 01:12.990 --> 01:16.020 that will help us in our intuitive understanding. 01:16.020 --> 01:19.080 Then we'll talk about the asynchronous element 01:19.080 --> 01:21.480 in the A3C algorithm, 01:21.480 --> 01:22.770 and then we'll talk about the advantage 01:22.770 --> 01:24.960 and that's the tutorial where we'll put everything together 01:24.960 --> 01:25.920 and it will all start making sense. 01:25.920 --> 01:28.020 So a quick word of caution 01:28.020 --> 01:29.310 that as you're going through these, 01:29.310 --> 01:32.790 you might notice that after the actor-critic 01:32.790 --> 01:35.130 and even after the asynchronous tutorial, 01:35.130 --> 01:39.270 you will still not exactly have the full picture 01:39.270 --> 01:42.810 or full understanding of how this all ties in together. 01:42.810 --> 01:43.740 And that's totally okay 01:43.740 --> 01:47.940 because all these three elements they indeed work together 01:47.940 --> 01:50.100 and they require each other to function. 01:50.100 --> 01:54.000 So just power through it and get to the third tutorial. 01:54.000 --> 01:56.400 And in the third tutorial, the advantage tutorial, 01:56.400 --> 01:58.050 that's when it will all come together. 01:58.050 --> 01:59.100 And if at that point 01:59.100 --> 02:02.640 you still don't quite understand something, 02:02.640 --> 02:04.740 then you can just revise that specific tutorial, 02:04.740 --> 02:06.300 whether it's actor-critic or asynchronous 02:06.300 --> 02:08.340 or the advantage tutorial, to put it all together. 02:08.340 --> 02:11.280 But basically, just don't get thrown off 02:11.280 --> 02:13.380 if after one of those two tutorials, 02:13.380 --> 02:15.630 you still feel that you need some information, 02:15.630 --> 02:17.480 that's because they all work together 02:19.643 --> 02:21.390 to one functioning A3C algorithm. 02:21.390 --> 02:22.950 That's in the advantage tutorial, 02:22.950 --> 02:25.830 that all ties in together very nicely. 02:25.830 --> 02:29.640 And finally, we'll talk about some additions 02:29.640 --> 02:33.510 that you can add or some additional modules and elements 02:33.510 --> 02:34.980 that you can add to the A3C algorithm. 02:34.980 --> 02:37.350 The thing is that there's different versions 02:37.350 --> 02:39.420 of the A3C algorithms, different modifications 02:39.420 --> 02:42.618 and already through the first four tutorials in the section, 02:42.618 --> 02:45.450 we will see some modifications. 02:45.450 --> 02:47.820 And at the end, we will add an extra one, 02:47.820 --> 02:51.300 the long short-term memory, LSTM. 02:51.300 --> 02:55.440 Now, here we won't go into a lot of detail on LSTM. 02:55.440 --> 02:56.760 That's a topic for another course, 02:56.760 --> 03:00.840 but nevertheless, you will get a overview of what LSTM is 03:00.840 --> 03:02.850 and how those structures 03:02.850 --> 03:04.920 or how those layers and neural networks work 03:04.920 --> 03:06.630 so that you have a better understanding 03:06.630 --> 03:08.130 in preparation for the practical tutorials. 03:08.130 --> 03:10.590 But very important also, 03:10.590 --> 03:15.240 this will showcase how the A3C algorithm can be modified 03:15.240 --> 03:17.310 to suit certain purposes. 03:17.310 --> 03:19.410 So there we go. That's the plan for this section. 03:19.410 --> 03:23.670 Definitely it's going to be an exciting algorithm to look at 03:23.670 --> 03:27.540 because it's the top algorithm at the moment in the world 03:27.540 --> 03:29.820 and we're going to be developing an intuitive understanding. 03:29.820 --> 03:33.000 And then together after this, you're going to be coding it, 03:33.000 --> 03:34.650 which is very, very exciting. 03:34.650 --> 03:37.230 And I can't wait to see you on the first tutorial. 03:37.230 --> 03:39.153 And until then, enjoy AI.