WEBVTT 00:00.080 --> 00:06.530 Before we move on to the job description, let's copy paste the code or the selector we have inside 00:06.530 --> 00:11.450 of console for our neighborhood and make a variable. 00:11.450 --> 00:12.500 Call it hood. 00:12.890 --> 00:19.430 Paste the selector inside here and then add it to our result object. 00:22.420 --> 00:30.400 Now the next thing we need to do is we need to go through each of these jobs URL and get their job description. 00:32.900 --> 00:39.140 You can also get their address inside of that page and maybe their compensation. 00:39.740 --> 00:46.220 So first, let's just rename our main function from scrape Craigslist to scrape job header. 00:50.780 --> 00:51.620 Like so. 00:52.580 --> 00:59.690 And then let's make another main function, which we will call scrape Craigslist. 01:02.080 --> 01:12.580 And in here we will make a call for the scrape job header instead and we get the array back from it. 01:12.610 --> 01:19.030 The job array with the headers, with the title and links and so on. 01:24.750 --> 01:32.970 We'll say await and then want to return the scrape results array from here, which contains the title 01:33.000 --> 01:33.510 date. 01:33.510 --> 01:35.130 It was posted and so on. 01:40.950 --> 01:46.830 And then let's make a new function in here, which we call scrape description, where we will be going 01:46.830 --> 01:55.740 through each of the jobs inside of this scrape results array and get the description for the jobs. 01:56.010 --> 01:59.550 So we pass in the array jobs with headers. 02:01.990 --> 02:10.900 And then inside here, we need to do a little magic using Promise.all to iterate through each of these 02:10.900 --> 02:13.390 jobs and make a request. 02:15.160 --> 02:19.300 Let me just show you what we will get from this function. 02:19.300 --> 02:24.820 We will get the jobs with the full data and we pass in the jobs with headers. 02:26.890 --> 02:30.910 Now we say await first and then promise dot all. 02:30.940 --> 02:41.110 So this takes in an array of promises and it's only going to resolve once all of the promises have resolved. 02:41.710 --> 02:48.780 And we use this JavaScript headers dot map to iterate over the array of jobs with headers. 02:48.790 --> 02:54.940 This is basically JavaScript ES6 that I'm showing you here, or Es7. 02:58.050 --> 03:04.610 We iterate over the chops with headers array using map, and then we can go through each chop and make 03:04.640 --> 03:10.070 a request to the chops URL that we got earlier from the main page. 03:13.980 --> 03:16.620 And I'm also throwing in async here as well. 03:16.620 --> 03:20.940 So we can use the await keyword inside of the map function. 03:22.310 --> 03:28.130 So all of this can seem confusing if you don't know ES6 or seven first, but. 03:30.520 --> 03:37.330 If you don't know exactly what I'm doing here, maybe brush up a bit on Es7 or ES6, but hopefully you 03:37.330 --> 03:39.100 will get the gist of how this works. 03:39.100 --> 03:41.470 If you don't know Es7 already. 03:41.920 --> 03:45.940 Then let's make the chario variable just like we did before. 03:47.520 --> 03:49.320 In the job titles. 03:49.800 --> 03:52.620 Now we're just doing it for every job instead. 03:54.170 --> 04:02.240 And then let's get the posting body ID, which contains all of the, um, the job description. 04:04.650 --> 04:08.100 And now let's see how posting body text looks like. 04:09.140 --> 04:17.150 We can see we get all of the job description here, but I don't like the QR code link to this post so 04:17.150 --> 04:20.180 much in the top, so let's try and get rid of that. 04:26.480 --> 04:34.040 Now since the QR code thing is inside of posting body, I think I want to remove this element first 04:34.040 --> 04:39.320 and then we can do the dot text on posting body to get the job description. 04:40.160 --> 04:41.810 Let me show you how that works. 04:42.530 --> 04:47.810 So we can use the print QR code container class to select it. 04:50.660 --> 04:54.140 And then we'll just simply remove it using jQuery. 04:55.840 --> 04:58.390 And now the element has been removed. 04:58.390 --> 05:07.360 And when I use text on posting body again, I can see that there's no longer the QR code link text up 05:07.360 --> 05:14.340 on the top and now we can simply do the exact same thing inside of the NodeJS project. 05:14.350 --> 05:22.300 We will remove the element first to not have the QR code link text in the job description and then use 05:22.300 --> 05:26.650 the text to create our own description variable. 05:27.570 --> 05:28.530 Like so. 05:33.740 --> 05:37.610 And now just let me make a property on the job object. 05:37.610 --> 05:41.900 And we have the job description, basically the job object. 05:43.180 --> 05:46.540 And let's also take the address while we're at it. 05:48.680 --> 05:49.970 Find out where the address is. 05:49.970 --> 05:53.450 So select the element and select the address. 05:54.860 --> 06:03.230 And we can see there's a div with a class map address, so why not try and select this? 06:04.690 --> 06:08.650 You can also use right click copy selector inside of Chrome tools. 06:08.650 --> 06:17.230 But I don't think the selector is always so short or efficient as when you're just making it up yourself. 06:18.110 --> 06:20.760 Let's try and see how this selector looks like. 06:20.780 --> 06:26.510 You can see it's very long because it wants to make sure that we're selecting the exact element. 06:26.660 --> 06:32.630 But in this case, I think we can just use the div and the map address class. 06:34.100 --> 06:36.440 And let's try and look how the text looks for that. 06:36.440 --> 06:37.870 And that's perfect. 06:37.880 --> 06:46.070 We get the address of the job, so let's just copy that and make that as our job dot address.