WEBVTT 1 00:00:00.000 --> 00:00:01.370 Hi guys, and welcome back! 2 00:00:01.370 --> 00:00:03.020 In this video we're going to talk about 3 00:00:03.020 --> 00:00:06.300 how to store sensitive data in your code. 4 00:00:06.300 --> 00:00:09.220 Or how not to store sensitive data in your code. 5 00:00:09.220 --> 00:00:11.310 So, what is sensitive data? 6 00:00:11.310 --> 00:00:12.930 Well, now we no longer 7 00:00:12.930 --> 00:00:15.970 will connect to the data.db file. 8 00:00:15.970 --> 00:00:18.440 Therefore, we need to put our ElephantSQL 9 00:00:18.440 --> 00:00:21.230 username and password, really the whole thing, 10 00:00:21.230 --> 00:00:23.210 including the host name and everything else, 11 00:00:23.210 --> 00:00:24.860 somewhere in our code so that our code 12 00:00:24.860 --> 00:00:26.900 can actually connect to it. 13 00:00:26.900 --> 00:00:28.770 But, often when we're working with code, 14 00:00:28.770 --> 00:00:30.410 we're sharing it with other people, 15 00:00:30.410 --> 00:00:32.600 or with everyone on sites like GitHub, 16 00:00:32.600 --> 00:00:34.960 so we don't want to open up our entire database 17 00:00:34.960 --> 00:00:36.840 by putting our username and password in there, 18 00:00:36.840 --> 00:00:38.510 because people can then go and you know, 19 00:00:38.510 --> 00:00:41.300 delete tables or drop the entire thing, or you know, 20 00:00:41.300 --> 00:00:44.170 find our data in there and read peoples passwords 21 00:00:44.170 --> 00:00:45.120 and all that stuff. 22 00:00:45.120 --> 00:00:48.060 So we need to be a bit careful with sensitive data. 23 00:00:48.060 --> 00:00:50.720 What we're gonna do instead is in our programme, 24 00:00:50.720 --> 00:00:53.470 we're going to write some code that expects 25 00:00:53.470 --> 00:00:55.990 a connection string to be defined. 26 00:00:55.990 --> 00:00:58.310 And then we're simply gonna tell other developers 27 00:00:58.310 --> 00:01:00.310 that want to look at our code that, 28 00:01:00.310 --> 00:01:01.810 yeah, they can look at everything, 29 00:01:01.810 --> 00:01:03.930 except the connection string, 30 00:01:03.930 --> 00:01:05.250 and that they need to define 31 00:01:05.250 --> 00:01:06.460 their own connection string to 32 00:01:06.460 --> 00:01:09.200 connect to their own database if so they want. 33 00:01:09.200 --> 00:01:12.320 The way we do this is normally with environment variables. 34 00:01:12.320 --> 00:01:14.680 Environment variables are context-dependent names 35 00:01:14.680 --> 00:01:15.840 that have values, 36 00:01:15.840 --> 00:01:17.610 and the context can be for example, 37 00:01:17.610 --> 00:01:19.870 an environment variable that effects every programme 38 00:01:19.870 --> 00:01:20.703 in a computer, 39 00:01:20.703 --> 00:01:23.180 so every programme has access to it, 40 00:01:23.180 --> 00:01:24.910 or it can be for specific programmes, 41 00:01:24.910 --> 00:01:28.070 so only specific programmes have access to the variable, 42 00:01:28.070 --> 00:01:29.960 or it can be programmes started by this user, 43 00:01:29.960 --> 00:01:32.230 you know there's a whole bunch of different contexts, 44 00:01:32.230 --> 00:01:33.420 but what we're gonna do, 45 00:01:33.420 --> 00:01:35.840 is we're going to define an environment variable 46 00:01:35.840 --> 00:01:37.800 that contains the connection string 47 00:01:37.800 --> 00:01:39.750 and we're gonna use that from our code. 48 00:01:39.750 --> 00:01:41.910 When other coders want to use our code, 49 00:01:41.910 --> 00:01:43.540 they will have to define their own 50 00:01:43.540 --> 00:01:45.480 environment variable in their computers 51 00:01:45.480 --> 00:01:47.940 with their own connection string. 52 00:01:47.940 --> 00:01:49.610 Okay, so I just got ahead of myself there, 53 00:01:49.610 --> 00:01:51.380 but that essentially what they have to do, 54 00:01:51.380 --> 00:01:53.910 our programme will be coded to expect the variable, 55 00:01:53.910 --> 00:01:55.680 and then it's gonna read it and connect to 56 00:01:55.680 --> 00:01:58.100 whatever that connection string is. 57 00:01:58.100 --> 00:02:00.130 Of course we're gonna assume the connection string 58 00:02:00.130 --> 00:02:01.650 is a valid connection string, 59 00:02:01.650 --> 00:02:03.650 and we're not gonna be checking that ourselves. 60 00:02:03.650 --> 00:02:05.230 To create environment variables, 61 00:02:05.230 --> 00:02:07.370 well every operating system has a different 62 00:02:07.370 --> 00:02:09.210 way of creating them manually. 63 00:02:09.210 --> 00:02:11.240 In Windows, for example, you have to go into your settings 64 00:02:11.240 --> 00:02:12.660 and change some things there, 65 00:02:12.660 --> 00:02:16.490 in macOS you have to export a new variable and so forth, 66 00:02:16.490 --> 00:02:17.680 but in coding, 67 00:02:17.680 --> 00:02:20.210 we normally create a text file called .env, 68 00:02:21.120 --> 00:02:23.900 and there we put our environment variables. 69 00:02:23.900 --> 00:02:25.800 Then, when we run the programme, 70 00:02:25.800 --> 00:02:27.340 we tell the operating system 71 00:02:27.340 --> 00:02:29.570 to turn the contents of that file 72 00:02:29.570 --> 00:02:34.570 into environment variables valid for this operating system. 73 00:02:34.670 --> 00:02:35.503 And fortunately, 74 00:02:35.503 --> 00:02:38.710 there is a library called python-dotenv 75 00:02:38.710 --> 00:02:41.000 that does this for us. 76 00:02:41.000 --> 00:02:44.350 So all we have to do is create the .env file, 77 00:02:44.350 --> 00:02:46.540 and then run python-dotenv, 78 00:02:46.540 --> 00:02:48.487 and that is going to turn the contents of the 79 00:02:48.487 --> 00:02:50.960 .env file into environment variables, 80 00:02:50.960 --> 00:02:53.750 and we don't have to worry about creating them ourselves, 81 00:02:53.750 --> 00:02:55.610 which as I said, can be different depending on 82 00:02:55.610 --> 00:02:57.470 the operating system that you're using. 83 00:02:57.470 --> 00:03:01.200 So, our .env file is going to look something like this. 84 00:03:01.200 --> 00:03:02.570 This is the .env file, 85 00:03:02.570 --> 00:03:05.610 and the contents are "DATABASE_URL" 86 00:03:05.610 --> 00:03:07.600 that's the name of our environment variable, 87 00:03:07.600 --> 00:03:08.470 equal, 88 00:03:08.470 --> 00:03:09.970 and then come the connection string. 89 00:03:09.970 --> 00:03:13.450 So postgres:// username, password, et cetera. 90 00:03:13.450 --> 00:03:16.700 We are not going to share this file with anybody, 91 00:03:16.700 --> 00:03:18.670 so this file will remain private. 92 00:03:18.670 --> 00:03:21.450 Even if we were putting our code up on GitHub, 93 00:03:21.450 --> 00:03:24.420 we would still not share this file. 94 00:03:24.420 --> 00:03:25.360 Instead what we would do 95 00:03:25.360 --> 00:03:30.360 is we would create a new file called .env.example, 96 00:03:30.470 --> 00:03:32.753 and that would contain "DATABASE_URL=" 97 00:03:34.270 --> 00:03:35.760 and then nothing. 98 00:03:35.760 --> 00:03:38.460 That way we can share this file with other coders, 99 00:03:38.460 --> 00:03:41.920 and they'll know that they have to create a .env file 100 00:03:41.920 --> 00:03:46.660 with this value in it, and a valid database URL. 101 00:03:46.660 --> 00:03:50.170 So again, we would not share our .env file, 102 00:03:50.170 --> 00:03:52.780 we would share .env.example. 103 00:03:52.780 --> 00:03:55.140 Similarly, if you are zipping your code up 104 00:03:55.140 --> 00:03:57.730 and sending it to someone else or something like that, 105 00:03:57.730 --> 00:04:00.280 make sure to not include your .env file in there, 106 00:04:00.280 --> 00:04:03.290 since that would give them access to your database. 107 00:04:03.290 --> 00:04:04.700 So, let's do this in our code. 108 00:04:04.700 --> 00:04:07.620 We'll create the .env and .env.example files, 109 00:04:07.620 --> 00:04:10.030 we will instal python-dotenv 110 00:04:10.030 --> 00:04:11.440 and we'll load the environment variables 111 00:04:11.440 --> 00:04:12.760 when we start the app, 112 00:04:12.760 --> 00:04:14.970 and we will use the environment variable value 113 00:04:14.970 --> 00:04:16.730 as our connection string. 114 00:04:16.730 --> 00:04:18.930 Note that when you're installing python-dotenv, 115 00:04:18.930 --> 00:04:22.030 you may also see a couple other libraries, such as .env, 116 00:04:22.030 --> 00:04:23.750 and make sure to not instal those. 117 00:04:23.750 --> 00:04:25.263 Instal python-dotenv. 118 00:04:26.130 --> 00:04:28.620 I'll guide you through how to do that just now. 119 00:04:28.620 --> 00:04:30.830 All right, so here we are in PyCharm, 120 00:04:30.830 --> 00:04:31.890 and as I mentioned, 121 00:04:31.890 --> 00:04:35.010 I've copied everything from our movie watch list project 122 00:04:35.010 --> 00:04:37.030 in the last section over here. 123 00:04:37.030 --> 00:04:40.630 Now the first thing to do is to instal python-dotenv. 124 00:04:40.630 --> 00:04:43.070 So, we're gonna go over to file, settings, 125 00:04:43.070 --> 00:04:45.220 or preferences on the Mac, 126 00:04:45.220 --> 00:04:47.370 and find the project interpreter section, 127 00:04:47.370 --> 00:04:48.820 click the plus icon, 128 00:04:48.820 --> 00:04:51.410 and type python-dotenv, 129 00:04:51.410 --> 00:04:54.080 make sure to not instal just .env, 130 00:04:54.080 --> 00:04:56.440 and we're gonna instal this package. 131 00:04:56.440 --> 00:04:59.850 This is just really a helper to load the .env file 132 00:04:59.850 --> 00:05:02.120 and make it into environment variables 133 00:05:02.120 --> 00:05:03.750 that our computer understand. 134 00:05:03.750 --> 00:05:05.230 So now that we've got that, 135 00:05:05.230 --> 00:05:07.990 we are going to go ahead and make use of it. 136 00:05:07.990 --> 00:05:10.910 But first, we have to create our .env file. 137 00:05:10.910 --> 00:05:12.190 So, we're gonna create a new file, 138 00:05:12.190 --> 00:05:13.800 just an empty file here, 139 00:05:13.800 --> 00:05:15.470 and call it .env. 140 00:05:15.470 --> 00:05:16.970 Notice that PyCharm is gonna ask you, 141 00:05:16.970 --> 00:05:19.850 you know, what sort of file is this? 142 00:05:19.850 --> 00:05:22.040 And we can just say text, don't worry about it, 143 00:05:22.040 --> 00:05:23.700 it's gonna be a very simple file. 144 00:05:23.700 --> 00:05:26.470 And you may see the plugins supported, 145 00:05:26.470 --> 00:05:27.650 and you can instal plugins, 146 00:05:27.650 --> 00:05:29.920 and that's gonna give you a bit of syntax highlighting, 147 00:05:29.920 --> 00:05:31.100 I wouldn't worry about it. 148 00:05:31.100 --> 00:05:33.097 Here we're gonna put our DATABASE_URL, 149 00:05:34.230 --> 00:05:35.340 and then an equals sign, 150 00:05:35.340 --> 00:05:39.220 and then the value that you copy from ElephantSQL. 151 00:05:39.220 --> 00:05:41.090 So I've put it in here, 152 00:05:41.090 --> 00:05:42.640 then we're also gonna right click again, 153 00:05:42.640 --> 00:05:46.200 create a new file and call it .env.example, 154 00:05:46.200 --> 00:05:47.550 again the same thing up here, 155 00:05:47.550 --> 00:05:49.000 but we can just click okay, 156 00:05:49.000 --> 00:05:53.050 and then here we're gonna put DATABASE_URL=, nothing. 157 00:05:53.050 --> 00:05:54.210 So these are our two files, 158 00:05:54.210 --> 00:05:56.000 remember the .env file, 159 00:05:56.000 --> 00:05:58.140 we're never going to share with other people. 160 00:05:58.140 --> 00:06:00.360 So if you were using Git for this project, 161 00:06:00.360 --> 00:06:02.440 you would now ignore the .env file 162 00:06:02.440 --> 00:06:05.380 using the .gitignore config file. 163 00:06:05.380 --> 00:06:06.610 But, we're not doing that at the moment, 164 00:06:06.610 --> 00:06:08.420 so I won't bore you with all those details. 165 00:06:08.420 --> 00:06:10.370 So, here we've got the rest of our code. 166 00:06:10.370 --> 00:06:12.880 Now, we have to load the .env file 167 00:06:12.880 --> 00:06:14.170 when we run this code. 168 00:06:14.170 --> 00:06:16.790 So, let's go over to our database.py file, 169 00:06:16.790 --> 00:06:18.920 and we need to import a couple of things, 170 00:06:18.920 --> 00:06:22.210 we're gonna do import os, 171 00:06:22.210 --> 00:06:23.820 we're gonna stop importing sqlite3, 172 00:06:23.820 --> 00:06:26.820 and instead we're gonna import psycopg2, 173 00:06:26.820 --> 00:06:31.690 and then we're gonna do from dotenv, import load_dotenv, 174 00:06:31.690 --> 00:06:34.960 and then immediately after, we're going to run it. 175 00:06:34.960 --> 00:06:36.920 load_dotenv can take a few arguments, 176 00:06:36.920 --> 00:06:39.170 as PyCharm will kindly tell you 177 00:06:39.170 --> 00:06:41.300 when you stop typing in there, 178 00:06:41.300 --> 00:06:43.870 but if you leave it empty, it's going to try to load 179 00:06:43.870 --> 00:06:47.140 the dotenv file in your root folder, 180 00:06:47.140 --> 00:06:48.060 which is what we've got, 181 00:06:48.060 --> 00:06:49.540 so we don't really have to do anything 182 00:06:49.540 --> 00:06:51.230 other than call this function. 183 00:06:51.230 --> 00:06:55.210 When that happens, any code after this one 184 00:06:56.060 --> 00:06:59.030 will be able to use those environment variables. 185 00:06:59.030 --> 00:07:01.720 Any code that runs before this runs 186 00:07:01.720 --> 00:07:04.530 will not be able to use the environment variables. 187 00:07:04.530 --> 00:07:06.080 So, that's why we do this at the top, 188 00:07:06.080 --> 00:07:08.830 and then when we connect to our database down here, 189 00:07:08.830 --> 00:07:10.090 we're going to be able to use 190 00:07:10.090 --> 00:07:12.910 the environment variable to connect. 191 00:07:12.910 --> 00:07:16.560 So, just to recap, python-dotenv is going to load 192 00:07:16.560 --> 00:07:18.870 our dotenv file and make it available to us, 193 00:07:18.870 --> 00:07:21.560 after we run load_dotenv. 194 00:07:21.560 --> 00:07:23.100 Then, down here, we're going to use 195 00:07:23.100 --> 00:07:26.770 the environment variable to load up the connection string, 196 00:07:26.770 --> 00:07:29.750 and then give it to psycopg2 to connect. 197 00:07:29.750 --> 00:07:31.860 So, how do we connect with psycopg2? 198 00:07:31.860 --> 00:07:34.300 Well, we just do psycopg2.connect, 199 00:07:34.300 --> 00:07:37.410 and then we pass in the connection string here. 200 00:07:37.410 --> 00:07:40.790 The connection string we can access with the os library. 201 00:07:40.790 --> 00:07:42.160 This is part of the standard library, 202 00:07:42.160 --> 00:07:46.800 so nothing to instal there, and we can do os.environ, 203 00:07:46.800 --> 00:07:50.030 for environment variable, then inside square brackets, 204 00:07:50.030 --> 00:07:54.370 because this is a dictionary, we put in DATABASE_URL. 205 00:07:54.370 --> 00:07:56.960 And that is gonna get the DATABASE_URL key 206 00:07:56.960 --> 00:07:59.740 from the environ dictionary in the os package, 207 00:07:59.740 --> 00:08:02.440 and give it to us so that we can connect to it. 208 00:08:02.440 --> 00:08:04.950 All right, so I mentioned earlier on in the course, 209 00:08:04.950 --> 00:08:07.500 that whenever we use psycopg2, 210 00:08:07.500 --> 00:08:09.110 we need to create cursors, 211 00:08:09.110 --> 00:08:12.488 and we can't do just connection.execute 212 00:08:12.488 --> 00:08:14.970 as we have been doing, so we're gonna have to 213 00:08:14.970 --> 00:08:16.340 change a few things in here, 214 00:08:16.340 --> 00:08:18.310 but let's do that in the next video. 215 00:08:18.310 --> 00:08:20.760 Thank you guys for joining me, I'll see you soon.