WEBVTT 00:01.760 --> 00:09.110 One of the reasons I created this mock proxy site is so students of this course can avoid getting stuck 00:09.110 --> 00:16.250 by getting banned on real websites and to provide sort of a safe playground for the students. 00:16.340 --> 00:23.510 So let's try and see how if we do a console log after the request here, let's try and see how fast 00:23.510 --> 00:25.460 the requests are getting fired off. 00:25.460 --> 00:33.110 So if we do a console log just with the job URL here and let's say request fired. 00:36.150 --> 00:37.440 Last job URL. 00:37.530 --> 00:40.860 Let's try and see how fast the requests are being fired off here. 00:42.130 --> 00:45.370 So you can see they get fired off really fast here. 00:45.370 --> 00:54.190 I did also print the job descriptions, but definitely if you did this on a Craigslist website or other 00:54.190 --> 01:00.820 websites, you might risk getting banned because this is way too many, uh, requests to fire off compared 01:00.820 --> 01:07.870 to someone who is just browsing a website clicking around, they wouldn't be making this many requests 01:07.870 --> 01:08.860 on a website. 01:09.940 --> 01:17.080 So the simplest way to get around this is to rewrite our map loop into a good old for loop instead. 01:17.080 --> 01:24.070 So everything is done sequentially, and we can wait for each promise to resolve sequentially. 01:24.790 --> 01:27.370 So that would look something like this instead. 01:27.370 --> 01:28.570 So we do a for. 01:30.140 --> 01:32.840 And we say const job of all. 01:33.700 --> 01:34.930 Chops. 01:39.850 --> 01:48.550 And then we can do simply the same thing here where we say job description page and we get the job description 01:48.550 --> 01:49.360 as well. 01:50.350 --> 01:57.490 Exactly the same, although in this case we will be creating a its empty array instead and then put 01:57.490 --> 01:59.140 the job inside there. 01:59.140 --> 02:02.800 So let's call it all jobs with descriptions. 02:04.780 --> 02:06.610 Let's initialize this empty array. 02:07.570 --> 02:15.760 And then at the end of this for loop here, let's push this, uh, modified job with the description 02:15.760 --> 02:17.470 inside of our array. 02:18.730 --> 02:22.090 So push chop inside here. 02:22.090 --> 02:31.720 And then let's delete all this map loop we have here, because it does fight off all the requests at 02:31.720 --> 02:32.110 once. 02:32.110 --> 02:35.890 At most websites are not going to like this. 02:36.730 --> 02:38.590 So now let's try and see. 02:38.590 --> 02:41.350 Well let's try and do the console log also here. 02:41.350 --> 02:42.220 Let's see. 02:42.220 --> 02:43.780 Request fired. 02:44.510 --> 02:47.390 With the chop, Earl. 02:48.290 --> 02:53.570 Just so you can see how different it looks like when we use this for loop instead. 02:54.350 --> 02:56.780 Let's do the node index.js again. 02:58.830 --> 03:02.430 And it says here all job descriptions is not defined. 03:02.460 --> 03:06.570 It's because I called it all jobs with descriptions instead. 03:07.580 --> 03:09.560 So let's try and run this again. 03:11.500 --> 03:13.600 And now you can see it fires up. 03:13.600 --> 03:18.970 One request goes on to the next one, and it awaits also for the promise to resolve. 03:18.970 --> 03:25.990 So it actually waits for the request to finish and getting the HTML before it moves on to the next one. 03:26.290 --> 03:27.550 So that's slower. 03:27.550 --> 03:34.870 But there is another trick to make it even slower, because imagine if you were someone surfing the 03:34.870 --> 03:36.340 Grayslake website. 03:36.550 --> 03:39.400 This would actually still be quite fast. 03:39.460 --> 03:44.440 I mean, if you're just clicking around on Craigslist browsing things, you wouldn't really go this 03:44.440 --> 03:44.890 fast. 03:44.890 --> 03:50.320 I think this is kind of fast to click around on Craigslist and, uh, look at things. 03:50.320 --> 03:54.910 So you might still risk getting blocked even with this for loop instead. 03:56.410 --> 04:03.100 Now, in the next section, I'm going to show you a trick to avoid even further to get banned from real 04:03.100 --> 04:04.180 websites.