WEBVTT 00:04.360 --> 00:09.920 In order to create the CI CD process, we need to first understand a bit more with a GitHub workflow 00:09.920 --> 00:11.880 is in its simplest form. 00:11.920 --> 00:18.840 A GitHub workflow is an automated process made up of one or multiple YAML scripts, which contain a 00:18.840 --> 00:20.880 set of steps to execute. 00:21.800 --> 00:28.080 You define these YAML files in the GitHub workflows directory, so let's create it right now. 00:29.200 --> 00:31.760 Let's create a folder dot GitHub. 00:32.880 --> 00:38.280 And then here we will create a subfolder called workflows. 00:39.000 --> 00:46.640 And let's also go ahead and in our case create one YAML file that will contain all the ci CD workflow. 00:46.680 --> 00:50.200 Let's name it something like ci CD for YouTube. 00:50.360 --> 00:58.920 So new file ci CD, YouTube, elt dot YAML. 00:59.120 --> 01:01.000 Don't worry if it's showing in red. 01:01.040 --> 01:04.840 Once we create and save, this will no longer be red. 01:04.880 --> 01:07.920 Also, this icon is different from what you might have. 01:08.080 --> 01:14.440 Since I have the GitHub actions extension, each GitHub workflow YAML file will vary, but there are 01:14.440 --> 01:17.640 certain elements which are common in almost all of them. 01:17.680 --> 01:21.760 We will now go through the YAML file and customize it for our needs. 01:22.080 --> 01:25.880 The first line of YAML file will contain the name of the workflow. 01:26.240 --> 01:29.960 This name will also be displayed in the GitHub actions interface. 01:30.360 --> 01:31.360 Let's give it a name. 01:31.800 --> 01:35.640 For CI CD pipeline. 01:38.000 --> 01:42.080 Next we need to define what events trigger the workflow. 01:42.120 --> 01:47.960 The key word en is used to trigger a workflow, and in this case we want to trigger on three different 01:47.960 --> 01:48.640 scenarios. 01:48.680 --> 01:51.800 It will either be a push on the main branch. 01:58.360 --> 02:04.870 It could also be ideal because of a pull request on again the main branch. 02:08.430 --> 02:14.150 And also, for those who don't know what a pull request is, it is basically a proposal to merge a set 02:14.150 --> 02:19.630 of changes from a feature branch into a target branch, which in this case is main. 02:20.150 --> 02:26.150 This would be a good idea to include workflow dispatch, which is useful when you want to allow manual 02:26.150 --> 02:29.950 triggering of the workflow for debugging or special cases. 02:35.470 --> 02:41.430 This manual triggering can be done on the GitHub actions UI, and we will see it when we come to the 02:41.430 --> 02:43.750 manual triggering part of this section. 02:44.030 --> 02:50.390 Now in our case we only worked with the main branch, but while developing you will use some feature 02:50.390 --> 02:56.470 branches to test out code before pushing to the main and before pushing to main, we would want that 02:56.470 --> 02:58.630 our CI CD pipeline is tested. 02:59.150 --> 03:05.750 So we can do one small change here and include the feature branches. 03:05.750 --> 03:07.310 And this will be written as follows. 03:08.910 --> 03:15.710 So this feature here, what it signifies is that the pipeline will be triggered on any push to a branch 03:15.750 --> 03:18.670 starting with feature slash. 03:18.710 --> 03:24.110 Now that we define the triggers, we can move on to the jobs which are effectively the tasks to run. 03:24.550 --> 03:27.430 And each job contains a sequence of steps. 03:27.470 --> 03:28.430 That's right jobs. 03:29.630 --> 03:37.310 And let's start by defining the continuous integration jobs with the first CI job being building and 03:37.310 --> 03:38.670 pushing the Docker image. 03:38.790 --> 03:43.230 If either the docker file or requirements.txt change. 03:43.550 --> 03:48.230 So we can define the name of the job and which runner will be used to run the job. 03:48.230 --> 03:51.430 In our case, this will be using the latest ubuntu image. 03:51.550 --> 03:53.070 This will look as follows. 03:54.590 --> 04:04.550 Here I am naming the job And I am also saying the runner, which will be the ubuntu latest image. 04:04.550 --> 04:10.950 When we talk about runner, we are referring to an environment where the job is executed and GitHub 04:10.950 --> 04:13.190 will provide a hosted virtual machine. 04:13.190 --> 04:17.470 So coming back to our code, as I said, a job consists of steps. 04:17.470 --> 04:21.750 And here we declare the steps that will be performed in the specific job. 04:22.190 --> 04:23.590 So we use the step scheme. 04:23.710 --> 04:27.870 The first thing that we need to do is to get the code from the repository we have on GitHub. 04:27.870 --> 04:33.630 GitHub actions provides a built in action called the Checkout Actions. 04:33.670 --> 04:38.510 It allows you to have access to your code base and the recording of this video. 04:38.550 --> 04:41.510 We are at v4 of this action, so we will use that. 04:41.550 --> 04:47.310 I will leave a link in the appendix of this section to the checkout action which you see over here. 04:48.310 --> 04:50.710 So let's place the checkout action in our code. 04:51.830 --> 04:57.020 And now that we have access to our repo based, we need a way to check if the Dockerfile file or the 04:57.020 --> 05:02.580 requirements.txt have changed in the current commit compared to the base branch. 05:02.820 --> 05:09.140 Also, we are checking only these two files, since the docker image is based on solely these two files, 05:09.340 --> 05:14.860 and to do this we can use another inbuilt action which is called changed files. 05:15.100 --> 05:20.940 This would also be in the appendix of this section, and is the action that you see in the screen here. 05:22.420 --> 05:26.700 So writing this out we have the name which is the description of the step. 05:30.740 --> 05:38.100 We specify the ID of this step so its output can be referenced in subsequent steps. 05:39.060 --> 05:40.700 This is a very important step. 05:43.620 --> 05:50.860 And now we can specify the actual action which we are invoking which is currently at version 45. 05:52.140 --> 05:57.180 What is left then is to specify the files we want to monitor, which we can do as follows. 05:58.180 --> 06:04.260 Now that we have a way of checking if the docker file or the requirements.txt has changed, we can start 06:04.260 --> 06:06.100 the process of rebuilding Docker image. 06:06.300 --> 06:12.780 We will do this by first configuring Docker to use the build tool for building Docker images. 06:13.260 --> 06:19.220 For this we will use the setup build action, which is another action of GitHub actions currently at 06:19.220 --> 06:24.460 version three, which is what you see on the screen right now. 06:25.420 --> 06:32.420 Again, all of this will happen if and only if the docker file or requirements.txt have changed since 06:32.420 --> 06:33.340 the last commit. 06:33.540 --> 06:37.700 So we can combine all of this together and we can therefore write the following. 06:39.180 --> 06:46.020 So what I'm doing right now is simply formatting, as I'm seeing that on some lines I'm getting errors. 06:46.020 --> 06:52.100 And this is because for example name and uses should be indented to the same indentation. 06:52.780 --> 06:59.420 Same goes for name id AD and also with they all have to be in the same indentation. 07:02.740 --> 07:06.540 So with the build next step, we have only written the setup of the configuration. 07:06.820 --> 07:11.460 But to actually build and push the Docker image, we need to still write the code for it. 07:11.460 --> 07:17.580 And even before we can do this, we need to ensure that we are first logged in to Docker Hub where our 07:17.580 --> 07:18.660 image resides. 07:18.940 --> 07:25.140 Since we said we need to log in, that implies that we need to pass sensitive information like Docker 07:25.140 --> 07:29.980 Hub password in order to avoid hard coding this or any other variable for that matter. 07:30.020 --> 07:35.340 We can use GitHub actions, secrets and variables which you can access by doing the following. 07:35.780 --> 07:37.660 So first Google in your GitHub repo. 07:38.220 --> 07:39.580 In my case this is this one. 07:39.580 --> 07:43.420 Over here press on settings on the left hand side. 07:43.460 --> 07:46.580 Down here under security you will see secrets and variables. 07:46.900 --> 07:49.300 Click on it and press actions. 07:50.180 --> 07:57.130 Here you will see two options either the secrets or the variables which, as you might have already 07:57.130 --> 07:57.650 guessed. 07:57.730 --> 08:02.770 Secrets is to store sensitive data and variables is for the non-sensitive data. 08:02.770 --> 08:08.730 Inside both secrets and variables, you have the option to define both the environment and repository 08:08.770 --> 08:10.010 secrets and variables. 08:10.290 --> 08:15.170 This is a great thing to have to distinguish between your development and production environments, 08:15.170 --> 08:20.570 where your secrets and variables will be different if you are not working with environments, which 08:20.570 --> 08:27.010 is our current use case, you can use the Repository Secrets, which will define your secrets variables 08:27.050 --> 08:28.530 across environments. 08:28.530 --> 08:29.970 So let's use that one. 08:30.130 --> 08:35.770 We can now define the Docker Hub namespace and Docker Hub username in the variables section. 08:35.770 --> 08:36.770 So let's go here. 08:37.010 --> 08:44.250 Click New Repository variable and define the key under the name and the value for the specific value 08:44.290 --> 08:45.010 it will have. 08:45.050 --> 08:48.410 As a side note, regarding the namespace and the username. 08:48.410 --> 08:53.530 By default these will be the same, but it is good that we make a distinction between the two as the 08:53.530 --> 08:59.450 namespace is essentially a container for your repositories, and the user name is the unique identifier 08:59.450 --> 09:00.770 for your Docker Hub account. 09:00.930 --> 09:03.970 So let's define the Docker Hub namespace and username. 09:05.410 --> 09:09.090 And in my case these will have the value as follows. 09:11.930 --> 09:13.650 I will also define the username. 09:13.850 --> 09:19.290 And while we are here we can also specify the Docker Hub repository variable, which will be the repository 09:19.290 --> 09:21.250 name we pushed to Docker Hub. 09:21.970 --> 09:24.690 In my case, the name is as follows. 09:26.010 --> 09:34.770 So the key will be Docker Hub repository and the actual value is the YouTube underscore API underscore 09:34.810 --> 09:35.250 ELT. 09:36.930 --> 09:40.410 What is left to define is the Docker Hub password. 09:40.450 --> 09:44.970 Obviously we will specify this in the secrets of GitHub actions. 09:47.130 --> 09:49.010 I will define this password here. 09:49.050 --> 09:50.370 Make sure that's from your end. 09:50.570 --> 09:54.880 You define your Docker Hub passwords under the secrets. 09:55.600 --> 10:01.040 And as you can see, the actual contents of the password will not be shown, unlike the variables. 10:01.560 --> 10:06.520 Here the variables you will have the value present, but for the secrets this will be hidden. 10:06.960 --> 10:10.440 Having done these steps, we can go back to the YAML file. 10:11.200 --> 10:15.480 So first we need to add this step where we are logging in to Docker Hub. 10:15.920 --> 10:17.160 This will look as follows. 10:17.960 --> 10:23.760 And to login we will use the login action which as of latest is at version three. 10:26.000 --> 10:29.160 Here you will see the documentation for the login action. 10:29.760 --> 10:31.880 The login action will look as follows. 10:32.040 --> 10:37.720 Notice how here we are calling the Docker Hub username and password, which we just defined in GitHub. 10:37.720 --> 10:39.440 Actions, secrets and variables. 10:39.480 --> 10:46.360 Since password was defined under the secrets, we use this secrets dot and the variable key. 10:46.680 --> 10:52.160 And since the username was defined as a variable, we use the dot Syntax. 10:52.200 --> 10:58.240 Now that we are logged in, all that is left is to build and push the Docker image to our own personal 10:58.240 --> 10:59.320 Docker Hub account. 10:59.520 --> 11:05.720 Again, this is only needed when there is a change in the requirements of the XD or the docker file, 11:06.120 --> 11:07.800 so we can write the following. 11:08.600 --> 11:14.800 From here we will touch upon the topic of Docker image tags and we will give the image two tags. 11:15.080 --> 11:21.240 We will have the latest tag to indicate the image being pushed is the most recent stable image. 11:21.440 --> 11:23.040 This will look as follows. 11:24.600 --> 11:30.320 Again, we note that here we are making use of the GitHub actions variables that we just used a few 11:30.320 --> 11:34.040 minutes ago to define the Docker Hub namespace and the repository. 11:34.280 --> 11:41.720 Coming back to the tags, we also have a second tag based on the commit hash, which is GitHub actions 11:41.720 --> 11:47.080 variable that uniquely identifies the image for the specific commit. 11:47.120 --> 11:48.680 This will look as follows. 11:50.040 --> 11:53.200 The commit hash being this path over here. 11:54.760 --> 11:58.560 At this point, you might ask, why do we have the second tag with the commit hash? 11:58.880 --> 12:04.680 This is important as it acts as a unique identifier, ensuring that we have traceability to the exact 12:04.680 --> 12:06.560 code that produced the image. 12:06.960 --> 12:09.760 This is equivalent to what we were doing with the EMV. 12:09.880 --> 12:18.160 Specifying the image tag variable and setting it to, for example, at the start 1.0.0, and then when 12:18.160 --> 12:22.840 a change was done to the image, we set it to 1.0.1. 12:23.080 --> 12:29.520 The very last thing which we can forget is that we need to include the build context for Docker, which 12:29.560 --> 12:31.480 in our case is the current directory. 12:31.880 --> 12:37.960 So we specify this dot over here before we can conclude the first job of our CI pipeline. 12:38.240 --> 12:41.280 We need to perform some changes in Docker compose. 12:41.480 --> 12:46.760 Currently in Docker Compose we are using the Docker image with this image tag. 12:46.960 --> 12:48.160 Environment variable. 12:48.200 --> 12:55.070 In the docker compose we are referencing the environment variables which we get from the EMV in this 12:55.110 --> 12:57.950 ubuntu environment offered by GitHub actions. 12:57.990 --> 13:04.630 We will not have access to the env as it was being versioned by git, so we will need to specify all 13:04.630 --> 13:09.670 the environment variables you see here as GitHub actions, variables or secrets. 13:09.950 --> 13:17.470 For now, the only change we need to do is to change the image tag variable to latest, which will be 13:17.470 --> 13:21.750 the same as we specified in the tag command in the workflow YAML file. 13:26.710 --> 13:30.910 We will do further changes to the docker compose, but these will come later on. 13:30.910 --> 13:34.070 So with that we finished the first CI job. 13:34.110 --> 13:37.430 Now we can move on to the second and final job. 13:37.430 --> 13:44.590 This will join both the CI where we run the unit and integration tests, and also the CD where we run 13:44.590 --> 13:46.070 the end to end tests.