WEBVTT 00:00.110 --> 00:06.350 Now we are actually going to go and make a project where we are going to utilize the API that we just 00:06.350 --> 00:07.070 found. 00:07.730 --> 00:10.580 So I'm going to make a directory. 00:11.210 --> 00:15.620 I will call it Nordstrom scraper. 00:17.350 --> 00:24.250 And go inside the directory CD Nordstrom scraper and then initialize npm. 00:24.610 --> 00:25.720 NPM init. 00:25.750 --> 00:26.080 Dash. 00:26.080 --> 00:26.980 Dash js. 00:28.930 --> 00:36.070 Once we initialize the project or the NPM, I'm going to open the folder inside of Visual Studio code. 00:45.950 --> 00:48.620 Now inside Visual Studio code. 00:48.620 --> 00:52.640 I'm going to make a folder here and we'll call it the back end. 00:53.960 --> 00:59.060 And we're going to put an index.js file in here. 01:00.050 --> 01:01.850 Then we're going to add some packages. 01:01.850 --> 01:08.630 Now we're going to add our trusty old request and request promise. 01:10.420 --> 01:12.580 Let me just enhance that. 01:14.340 --> 01:16.140 So you can see the text. 01:16.260 --> 01:21.750 So request and need to add yarn, add yarn. 01:21.750 --> 01:23.520 Add request Package. 01:23.550 --> 01:24.810 Request Promise. 01:25.260 --> 01:29.010 And we also need to add the express package. 01:29.010 --> 01:35.430 So express is a way to make rest APIs inside of NodeJS. 01:37.500 --> 01:38.850 So we add those. 01:40.320 --> 01:44.760 And now we go ahead and write our back end. 01:45.120 --> 01:48.390 So the first thing we have is we have express. 01:48.600 --> 01:51.600 So we say require express. 01:54.040 --> 02:00.010 And then we make what's called we call it app where we initialize express. 02:01.360 --> 02:08.140 And what we do then is we say app, listen, what port we want to run on. 02:08.140 --> 02:10.930 So we say we run it on port 4000. 02:11.740 --> 02:18.430 Then we have a function that runs once the NodeJS API is running. 02:18.430 --> 02:26.890 So we write console.log server running on port 4000 just like that. 02:26.890 --> 02:32.110 And then I'm going to run the API in the terminal here. 02:32.290 --> 02:42.940 So I'll do CD back end to the folder here containing the index.js and then I'll run Node man Index.js. 02:43.240 --> 02:51.650 If you don't have Node Man installed already, go ahead and install it using NPM install dash g a node 02:51.680 --> 02:52.280 one. 02:52.610 --> 02:59.810 So the dash G makes it a global install, which means that you can use it anywhere inside of your terminal. 03:00.110 --> 03:06.350 And Norman is great because when we make changes inside Index.js. 03:07.850 --> 03:11.330 It's going to automatically reload the changes. 03:11.330 --> 03:13.100 So if we change this to. 03:15.080 --> 03:20.660 Something like that, it's going to automatically start the script again so you don't have to type node 03:20.660 --> 03:23.510 dot node index.js all the time. 03:23.690 --> 03:26.690 You can just have this Node one running instead. 03:27.570 --> 03:34.410 So now we have our NodeJS Express server running, but we're actually not able to use it so much because 03:34.410 --> 03:39.900 we don't have any endpoints we're listening to and that's what we're going to do now. 03:40.050 --> 03:52.050 So let's say app dot get and we'll say slash Nordstrom and then we can have a request. 03:52.050 --> 03:55.350 So this is the request parameters we get in. 03:55.350 --> 04:00.870 I'll get into that longer and next time then response and then next. 04:02.200 --> 04:07.330 And so this one is basically how you're calling the API. 04:07.360 --> 04:10.150 What parameters are you sending as you're calling the API? 04:10.180 --> 04:11.050 We'll get into that. 04:11.080 --> 04:17.320 Next rest is what you use to send a response back from the API to the user. 04:17.380 --> 04:19.840 And next is something you run. 04:20.380 --> 04:24.130 You can run next like middleware stuff. 04:24.310 --> 04:26.890 We're not going to be using that in this section. 04:28.360 --> 04:34.870 Now, I want to make a little disclaimer here and say you could make a course about this, about NodeJS 04:34.870 --> 04:35.380 Express. 04:35.380 --> 04:39.310 And there's a lot of courses out there about NodeJS Express. 04:39.310 --> 04:46.690 I'm not going to go into detail and teach you how everything there is to know about NodeJS Express. 04:47.080 --> 04:52.630 This is just to get you up and running, having a scraping API running. 04:53.020 --> 04:59.320 And so if you want to know more about NodeJS Express, there's a lot of great courses out there you 04:59.320 --> 04:59.860 can take. 04:59.860 --> 05:05.800 So please don't be discouraged if some of this just doesn't make sense to you. 05:06.550 --> 05:13.060 It's a basically a whole course in itself to learn, but this is going to get you up and running, running 05:13.060 --> 05:14.680 a web scraping API. 05:14.800 --> 05:19.480 So if you follow me step by step, don't worry, we are going to get through this and you will have 05:19.480 --> 05:21.460 a web scraping API in no time. 05:22.600 --> 05:28.690 Next up is we want to make a call to this API that we just discovered and. 05:29.840 --> 05:32.810 And so we had this. 05:33.260 --> 05:35.360 We had it inside of Postman. 05:36.770 --> 05:39.800 Let me see if I can get postman up and running here. 05:42.520 --> 05:51.460 So here we have the URL that we came to that we found out was working with the parameters here, the 05:51.460 --> 05:54.040 query parameters and the headers. 05:54.040 --> 05:57.940 So let's just copy the URL here and. 06:00.250 --> 06:00.940 You know what? 06:00.940 --> 06:03.790 Let's first try and just get this running. 06:03.790 --> 06:08.950 Let's try and say a hello world just so you can see how it is running. 06:09.800 --> 06:17.990 So now it is running on this endpoint where we do a get request on this endpoint here, Nordstrom and 06:17.990 --> 06:20.270 it's going to return a Hello world. 06:20.270 --> 06:25.850 So if we do inside of Postman if you type in. 06:26.880 --> 06:33.600 A Http and you type in localhost and you say 4000 and you say Nordstrom. 06:34.230 --> 06:35.220 Then. 06:36.140 --> 06:37.970 It's going to say Hello world. 06:38.780 --> 06:41.180 And that's just for pure text. 06:41.210 --> 06:49.550 It's not Json, but basically you now have an API running on NodeJS on your own local computer, on 06:49.550 --> 07:00.410 Port 4000 on the Norstrom endpoint, but we obviously want to return the Nordstrom API results instead 07:00.770 --> 07:03.770 so we can use them inside of our React client. 07:05.520 --> 07:06.720 Let me show you how we do this. 07:06.720 --> 07:09.000 So we have the URL. 07:09.030 --> 07:14.970 I copied that from the postman request we did before in the previous section. 07:15.840 --> 07:17.700 And so let me see. 07:17.700 --> 07:23.280 So I copied that in here and I say I copied into a variable const URL. 07:24.980 --> 07:31.070 And now we're going to try and make a request inside of our API. 07:31.580 --> 07:35.780 So remember in the previous lecture we imported request also. 07:35.780 --> 07:38.270 Well, we going to start using that now. 07:39.050 --> 07:42.050 So let's import request here. 07:43.010 --> 07:43.880 Require. 07:44.710 --> 07:45.630 My quest. 07:45.640 --> 07:46.750 Promise. 07:49.270 --> 07:57.940 What I want to show you now is remember in the Postman API, we found out we needed these two headers. 07:57.940 --> 08:06.190 We needed the authorization header with the API key and we also needed the API version in the header. 08:06.310 --> 08:10.060 So we need to have those inside the request. 08:12.470 --> 08:13.730 Import as well. 08:14.180 --> 08:17.390 We're just going to set it by default to have that in here. 08:17.390 --> 08:20.030 So we say dot defaults. 08:21.720 --> 08:24.300 And then we put in our headers. 08:24.300 --> 08:25.710 We can put that in here. 08:27.260 --> 08:28.910 And then remake one hitter. 08:28.910 --> 08:37.760 That's called authorization and we paste in the API key we have here that we made inside of our postman. 08:39.160 --> 08:40.360 Paste it in here. 08:41.050 --> 08:44.290 Then the newer API version. 08:45.850 --> 08:47.230 And we set that to one. 08:48.160 --> 08:55.060 So now we have the same headers that we had inside of Postman that we found out was working to get to 08:55.060 --> 08:57.880 the the results inside of the API. 08:59.330 --> 09:04.510 Now all that is left now is to do a little request and return it. 09:04.520 --> 09:05.580 So let's see. 09:05.600 --> 09:10.970 So we have this is going to return the Json so we say request. 09:11.920 --> 09:15.250 Actually let me use a weight also because it's async request. 09:15.250 --> 09:16.960 So we want to use await. 09:17.320 --> 09:19.390 So request dot get. 09:20.740 --> 09:23.650 Earl and. 09:25.120 --> 09:26.290 Then we say rest. 09:27.190 --> 09:29.830 So rest set, Jason. 09:30.100 --> 09:36.700 So rest is the object you can use inside to send Jason. 09:37.330 --> 09:44.380 And now, since we're using Await inside here, I'm going to have async in front of the function here. 09:46.700 --> 09:52.520 Okay, now let's see what is going on when we try to make a request to the Nordstrom endpoint. 09:52.550 --> 09:55.190 Now inside of our API. 09:55.220 --> 09:56.930 So before it was just Hello world. 09:56.930 --> 09:58.310 Let's see what it says now. 09:59.560 --> 10:00.900 I'd still say subtle world. 10:00.910 --> 10:01.570 Okay. 10:01.620 --> 10:02.130 Okay. 10:02.230 --> 10:05.920 That's because we're doing a recent Hello world up here. 10:06.040 --> 10:12.520 So it's not we're never getting down to this part, so we'll just remove the recent hello world here 10:12.730 --> 10:14.290 and let's try again. 10:17.690 --> 10:18.680 And there we go. 10:18.680 --> 10:19.520 So we return. 10:19.520 --> 10:20.330 Oops. 10:21.290 --> 10:21.960 We return. 10:22.010 --> 10:29.930 We get lots of Actually, it is Jason, but it's like formatted like text in here. 10:30.290 --> 10:38.180 So we need to you can see the headers of the response here is saying content-type text HTML. 10:38.330 --> 10:44.240 So we need to also set the header to be a content type of Json. 10:44.240 --> 10:45.740 So that's pretty easy to do. 10:45.770 --> 10:48.320 We just do rest dot header. 10:49.250 --> 10:54.380 Oh no restart set hitter and we say content type. 10:55.770 --> 11:03.720 I'm passing application slash Json and that is going to make Postman and other applications recognize 11:03.720 --> 11:06.300 that this is Jason that we are receiving. 11:06.570 --> 11:08.250 Now let's try and do it again. 11:09.090 --> 11:14.070 And now it gets a prettified it gets formatted as Json. 11:14.070 --> 11:14.940 So here we go. 11:14.940 --> 11:23.010 So this is the exact same response that we would get inside of the the result we get from the API here. 11:23.790 --> 11:25.200 It's exactly the same. 11:25.200 --> 11:32.070 We're just passing it through our own API instead so we can use it in our front end applications like 11:32.100 --> 11:32.940 our React app. 11:32.970 --> 11:33.960 We're going to make. 11:36.000 --> 11:36.510 Okay. 11:36.510 --> 11:37.380 So. 11:39.550 --> 11:40.000 Let's see. 11:40.030 --> 11:40.390 Let's see. 11:40.390 --> 11:48.100 In the next section we are going to pass in query parameters also so we can decide how many results 11:48.100 --> 11:48.550 we want. 11:48.580 --> 11:53.890 This top parameter here and we are also going to pass in the keyword parameter.