WEBVTT 00:04.200 --> 00:08.960 Now that we have the docker compose ready with the required changes, it's just a question of spinning 00:08.960 --> 00:10.040 up the containers. 00:10.560 --> 00:16.280 We have already went over some Docker commands briefly in the previous lecture, but here we'll cover 00:16.280 --> 00:17.280 them in more depth. 00:17.480 --> 00:20.240 Before you can do this, make sure that Docker is running. 00:20.240 --> 00:25.520 You can check this by going on Docker desktop and at the bottom left make sure that the engine is running. 00:26.440 --> 00:34.360 Once you know that it is running, run the docker compose up minus d command where the minus d flag 00:34.400 --> 00:39.600 means that the containers will run and detached mode, meaning they will run in the background. 00:39.640 --> 00:45.560 Let's press enter and the first time that you run it, since you probably won't have the airflow, Postgres 00:45.600 --> 00:51.920 and Redis images, it might take some time to run the docker command until the images are all installed. 00:51.960 --> 00:57.120 Ultimately, what we want is to get to the point where the containers we need are running and healthy. 00:57.960 --> 01:00.280 We will continue when the containers are running. 01:07.600 --> 01:09.880 So it looks like the containers are now running. 01:09.920 --> 01:13.230 We can confirm this by running the docker PS command. 01:17.390 --> 01:20.350 By default, this command will only show the running containers. 01:20.350 --> 01:28.230 And here you will see things like the image that is being used, the status ports and even the container 01:28.230 --> 01:28.750 names. 01:29.270 --> 01:33.870 A powerful command that you can use to access the containers is Docker exec commands. 01:34.230 --> 01:41.910 So if, for example, you want to access the scheduler container, we would use the command docker exec 01:42.470 --> 01:46.390 with the minus ID flag, which I will shortly explain. 01:47.230 --> 01:52.950 We will be using the scheduler container and then we use the bash command. 01:53.310 --> 01:59.550 So before we move further, the minus eight flags essentially give you an interactive shell session 01:59.550 --> 02:00.750 inside the container. 02:01.310 --> 02:04.830 And the bash command at the end invokes the bash shell. 02:05.590 --> 02:09.510 Now we are inside the airflow scheduler container. 02:09.670 --> 02:15.830 We can do a simple check while we are inside this container to see whether the JSON data file that is 02:15.830 --> 02:24.180 inside the data directory on our local machine is also inside the Docker container we have here. 02:24.220 --> 02:28.100 Let's check this right now by running CD command into the data directory. 02:28.660 --> 02:30.700 So CD into data. 02:32.140 --> 02:35.220 And then we'll run a simple LS command to see what we have. 02:36.260 --> 02:42.340 As you can see, the JSON file is also present inside the Docker container. 02:42.380 --> 02:49.700 Now in terms of airflow commands, if you write the following command airflow dash dash help, you will 02:49.700 --> 02:53.140 get a list of commands that you can use inside airflow. 02:53.380 --> 02:55.780 You can go through them in your own time. 02:57.420 --> 03:04.420 Another command that you can use is airflow info to get information about the airflow environment. 03:04.420 --> 03:12.020 So here, for example, we know that we're using version 2.9.2 with this particular executor and so 03:12.020 --> 03:12.380 on. 03:12.580 --> 03:17.220 As you can see there is a lot to explore and I invite you to do so in your own time. 03:17.220 --> 03:24.660 One final command that I would like to share with you with regards to airflow is the airflow cheat sheet 03:25.460 --> 03:27.140 and variables mode. 03:27.780 --> 03:33.800 And here you will get a list of airflow commands that you can use in your own day to day. 03:35.880 --> 03:44.200 I will leave you with two final docker compose commands, the first one being docker compose up minus 03:44.200 --> 03:45.680 d dash dash build. 03:46.200 --> 03:51.880 This is a useful command when you want to rebuild your services with, for example, updates in the 03:51.880 --> 03:52.600 docker file. 03:53.000 --> 03:59.240 It's a quick way to apply new builds without stopping all services, keeping your container orchestration 03:59.240 --> 04:00.280 smooth and running. 04:00.440 --> 04:06.880 If then you want to completely stop your Docker containers, you would use the command docker compose 04:07.960 --> 04:08.520 down. 04:08.760 --> 04:13.720 So this is a command that you should run if you want to apply bigger changes in your docker compose, 04:13.920 --> 04:17.960 like adding or removing services or changes to named volumes. 04:18.400 --> 04:23.800 So you would run this docker compose down command and then to spin up the containers again you would 04:23.800 --> 04:27.880 run docker compose up with the minus d flag. 04:28.680 --> 04:30.520 So let me run docker compose down. 04:33.160 --> 04:37.360 And here you are seeing Docker containers are stopped. 04:40.150 --> 04:43.350 Something that I want to mention for the sake of completeness is. 04:43.510 --> 04:50.030 In a previous lecture, I had said that the volume would be the volume that you defined here in the 04:50.030 --> 04:50.950 Docker compose. 04:51.390 --> 04:57.110 As you can see, this is roughly similar, but you also have the YouTube underscore at the beginning. 04:57.310 --> 05:03.990 This is there because the repository that we have is named YouTube underscore ELT. 05:04.430 --> 05:12.150 So the actual named volume by default will show as the name of the repository and then the volume defined 05:12.190 --> 05:12.630 here. 05:12.670 --> 05:19.990 One last piece of advice that I can give you is to always run the docker compose down command when you 05:19.990 --> 05:24.550 want to shut down your laptop, to prevent potential issues such as conflicts. 05:24.670 --> 05:29.430 The next slide will address the situation of this potential issue, but is not something that you should 05:29.430 --> 05:34.790 worry about if you properly shut down your Docker services after you have finished using them. 05:34.830 --> 05:40.150 I will see you in the next section of this course, where we will go over the Python scripts we built 05:40.150 --> 05:44.430 in the first section and apply the changes to create the airflow DAGs.