WEBVTT 00:00.050 --> 00:05.360 So now we looked inside of the network tab in Chrome Developer Tools. 00:05.390 --> 00:14.120 We figured out which request in this log had the login request and we also saw the request headers and 00:14.120 --> 00:17.060 the form data for this request to log in. 00:17.320 --> 00:24.440 Now let's see if we can replicate this request inside of Postman instead of inside of the browser. 00:25.010 --> 00:32.030 So Postman is sort of the next step before we move on to using request inside of NodeJS. 00:32.330 --> 00:38.780 It makes it easier for us to debug how to actually pass this log in request and get ourselves logged 00:38.780 --> 00:39.320 in. 00:39.970 --> 00:42.610 So let me show you how we do this. 00:43.120 --> 00:48.160 First, let's go and take the request URL from the login request. 00:48.190 --> 00:50.380 We have this post request. 00:50.410 --> 00:51.790 Let's copy that. 00:52.480 --> 00:54.100 And paste it in here. 00:54.550 --> 00:57.370 Now make sure to put the request method. 00:58.310 --> 01:02.060 This one to be the same as we have in here. 01:02.780 --> 01:08.990 So make sure to say it as post, which is the same as we had inside chrome dev tab. 01:10.010 --> 01:15.350 And now we need to pass in some kind of data or form data. 01:16.380 --> 01:17.490 To do that. 01:17.550 --> 01:29.520 Go into the body tab here and usually the forms data is in CSV form, Urlencoded that is basically as 01:29.520 --> 01:31.980 a web form or HTML form. 01:32.260 --> 01:36.720 There's also form data in raw binary. 01:37.470 --> 01:39.690 Most of the forms is in this. 01:40.370 --> 01:47.120 Some, for example, for APIs can also be in this data, but it's not so often that you see it. 01:48.030 --> 01:51.980 So make sure to select this one if you want to be really sure. 01:51.990 --> 01:58.050 What kind of data is being passed to this endpoint, you can check it inside of the Chrome developer 01:58.050 --> 01:58.800 tools. 01:59.160 --> 02:04.260 So if you go inside of the let's see, here we have. 02:04.890 --> 02:06.840 The request headers. 02:06.840 --> 02:11.370 So this is the request that you send to the server. 02:11.580 --> 02:14.160 And here we have the response headers. 02:14.160 --> 02:17.010 So that's the response back from the server. 02:17.310 --> 02:26.880 And if we look inside of the request headers, we can see a content type of application form URL encoded. 02:27.510 --> 02:32.400 So we can, in good confidence, set this data to be the same. 02:33.240 --> 02:35.400 Now we need to pass in some data. 02:36.750 --> 02:43.950 And the first data I usually try is the most obvious one, which is the email and the password. 02:44.760 --> 02:51.150 I'm going to show you sort of my process as I try to figure out how to actually get this login working. 02:51.510 --> 02:57.360 I already know how we can get it working, but I'm going to show you how my process is for getting it 02:57.360 --> 02:58.080 working. 02:58.350 --> 03:03.030 So by that I mean basically we copy this form data here. 03:03.330 --> 03:05.760 For example, input email handle. 03:06.570 --> 03:13.980 And we pass it in here as a key in the body tab and then write in the value we have also. 03:13.980 --> 03:15.480 So this is my email. 03:16.260 --> 03:19.200 And then we also need a password here. 03:19.200 --> 03:21.030 It's called input password. 03:22.210 --> 03:28.990 And notice that these values can be different based basically on based on what side you are on. 03:28.990 --> 03:34.540 So maybe Amazon just calls it email or they just call it password. 03:34.540 --> 03:37.000 It depends on how they made this site up. 03:37.390 --> 03:42.820 But for Craigslist, they just call it input, email handle and input password. 03:43.680 --> 03:51.960 So copy password or input password key as well, and then copy the value as well we have here. 03:53.540 --> 03:54.740 Paste it in. 03:54.890 --> 04:00.080 And now let's try and do the request and see what response we get from Craigslist. 04:02.190 --> 04:11.640 So you can see here we get a 404 not found, which means that our oops, which means that our request 04:11.640 --> 04:13.020 was not successful. 04:13.020 --> 04:14.820 It says Page not found. 04:15.060 --> 04:17.430 So something went wrong. 04:17.430 --> 04:21.660 We didn't give Craigslist all the data that they wanted. 04:21.690 --> 04:27.000 They don't they are not just happy with just the email and password, it seems. 04:27.090 --> 04:33.900 So then what I would do if this doesn't work at first, then I would try other values, maybe inside 04:33.900 --> 04:40.560 of the form data, maybe we can try the browser info data we have here. 04:40.590 --> 04:46.290 Let's try and paste it in inside the value then say browser. 04:47.910 --> 04:48.930 Info. 04:50.010 --> 04:50.910 Just like that. 04:50.910 --> 04:54.030 And then I would try again to see does it work now? 04:55.010 --> 04:57.200 And we can see it still doesn't work. 04:57.230 --> 04:59.840 It still says four four not found. 05:00.380 --> 05:03.710 Then I could try other ones like this one. 05:04.010 --> 05:05.090 It's called TI. 05:05.120 --> 05:08.910 Maybe that's something they really want to have with the form. 05:08.930 --> 05:10.880 So I tried that one also. 05:11.770 --> 05:13.510 And it still doesn't work. 05:13.690 --> 05:18.680 And now maybe I try all of these data as well. 05:18.700 --> 05:19.630 Also this one. 05:19.630 --> 05:27.400 Maybe I try them all step by step and see is there any point where I suddenly can be enabled to log 05:27.400 --> 05:27.880 in? 05:29.070 --> 05:30.650 Can see it still doesn't work. 05:30.920 --> 05:38.210 And then if I try all of the form data and try to basically copy all of that inside of Postman and it 05:38.210 --> 05:39.530 still doesn't work. 05:39.650 --> 05:43.810 Then the next step maybe is to look at your request headers. 05:43.820 --> 05:50.120 Make sure that you set your request headers inside postman the same as we have in here. 05:50.510 --> 05:53.540 So I can see there's a content type. 05:53.540 --> 05:56.980 We already have that because we selected this one here. 05:56.990 --> 05:59.990 So Postman is already setting that for us. 06:00.710 --> 06:03.830 Then I can see there's one with cookie. 06:03.860 --> 06:08.570 There's a cookie being set when we are logged into Craigslist just without logging in. 06:08.570 --> 06:12.470 I mean, just as you visit the site, Craigslist is going to set a cookie. 06:12.560 --> 06:14.000 Maybe it could be that. 06:14.000 --> 06:15.740 But let's save that for later. 06:15.770 --> 06:17.840 There's also an accept header. 06:17.840 --> 06:19.400 Maybe we can try that. 06:19.400 --> 06:21.650 So I'm going to copy the value here. 06:22.130 --> 06:26.420 And then inside the headers tab, I will write accept. 06:27.280 --> 06:29.080 And I paste in the value. 06:29.230 --> 06:31.090 Let's see if that works now. 06:31.390 --> 06:33.340 And it still doesn't work. 06:33.550 --> 06:37.060 So I just keep on trying all of these things. 06:37.060 --> 06:43.540 I mean, some things are more sensible or reasonable to make a difference. 06:43.540 --> 06:49.030 It's something you learn by practice basically on different sites. 06:49.630 --> 06:54.670 Maybe the origin site, the origin header information could work. 06:55.880 --> 06:57.080 So let's try that. 06:58.870 --> 07:03.940 Origin and then set the value in here from the Chrome developer tab. 07:05.070 --> 07:06.510 And then let's try again. 07:07.530 --> 07:09.940 It still says four for one not found. 07:09.960 --> 07:17.340 So that's pretty much the process as I tried to find out how to how can I log in? 07:17.460 --> 07:21.510 User agent is also a really good thing. 07:21.510 --> 07:23.520 You could try to see if that works. 07:23.520 --> 07:25.890 So let's try and set the user agent. 07:28.250 --> 07:29.660 Then paste that in. 07:31.450 --> 07:33.020 Well, it still doesn't work. 07:33.040 --> 07:38.770 And then finally, at some point, you arrive at something that just works. 07:38.770 --> 07:39.950 So let's try. 07:39.980 --> 07:41.110 Refer. 07:41.500 --> 07:42.460 Refer means. 07:42.460 --> 07:48.040 Well, what side have been are we getting referred from? 07:48.040 --> 07:56.200 I mean, you are on this side and then you get a refer from that side and you send that to this endpoint 07:56.200 --> 07:58.870 so it knows where does this request come from? 07:59.380 --> 08:00.820 And let's try again. 08:01.330 --> 08:03.640 And now it says 200. 08:03.640 --> 08:06.610 Okay, so now I've actually logged in. 08:08.470 --> 08:10.930 And let's see, what does the site say? 08:10.930 --> 08:13.510 It says here, Home of Stephan Hulshof. 08:13.780 --> 08:14.830 That's my name. 08:14.830 --> 08:16.780 Which means that I have logged in. 08:16.810 --> 08:18.610 It also says log out here. 08:18.850 --> 08:23.080 So now I have actually logged in using Postman. 08:23.530 --> 08:30.550 You can see in here in the cookies also, we have a session cookie being set with this value here and 08:30.550 --> 08:36.430 it has a expiry date which is really far out in the future, 2068. 08:36.430 --> 08:41.440 So if you are logged in to Craigslist, you are really logged in for a long time. 08:42.070 --> 08:44.320 But that's how it works. 08:44.350 --> 08:51.030 We use simply try a lot of things like the form data and the headers. 08:51.040 --> 08:58.090 Just try a lot of things until it simply works, until you find out what is actually needed to make 08:58.120 --> 08:59.740 to make yourself logged in. 08:59.980 --> 09:07.390 And then once I finally find something that actually makes me logged in, I try to disable other values. 09:07.390 --> 09:13.130 I've set until now, so I don't have to set a lot of values when I'm using a request. 09:13.280 --> 09:20.270 For example, let's try and just take off all of these extra form data we tried before. 09:20.870 --> 09:25.490 Let's see if our request still works and I can see it still works. 09:25.490 --> 09:28.520 I still get a 200 okay response. 09:29.000 --> 09:31.910 So I am still logged in. 09:32.610 --> 09:36.270 Now let's try and remove some of the header values we have here. 09:36.270 --> 09:41.640 Let's try and remove the exact origin and user agent. 09:45.280 --> 09:48.090 And now I can see I still get a 200. 09:48.100 --> 09:48.790 Okay. 09:49.670 --> 09:52.010 Now let's try and remove the referrer again. 09:53.740 --> 09:56.500 And here we can see I get a 404 not found. 09:56.650 --> 10:03.580 So the referrer header was apparently the key to getting this request work. 10:03.820 --> 10:09.580 And this can really be different between what sort of. 10:10.720 --> 10:14.020 So here you can see it says four four not found. 10:14.630 --> 10:17.340 Sometimes it does say 4 or 4 not found. 10:17.360 --> 10:23.600 I found out when I'm debugging this and you just have to try again, it seems, and then it logging 10:23.600 --> 10:24.890 is logging you in. 10:25.760 --> 10:28.310 But what I was trying to say is that. 10:29.600 --> 10:38.330 Finding out what is important when you log in and stuff like this is very different from site to site 10:38.360 --> 10:43.760 based on how they programmed or created the login functionality. 10:43.790 --> 10:49.700 They make different steps necessary, so to say different values. 10:49.700 --> 10:56.200 Sometimes you have to pass, you have to put all of the form data in as it is from Google Chrome. 10:56.270 --> 10:59.900 Sometimes you can get away with just the email and password. 11:00.050 --> 11:03.020 Sometimes you don't have to set any headers at all. 11:03.020 --> 11:07.160 Sometimes you have to set lots of headers and weird hitters also. 11:07.580 --> 11:15.920 So you have to look inside of your chrome developer developer tab, look at all the values and try them 11:15.920 --> 11:21.890 one by one inside of Postman until you get something that just clicks and works. 11:22.010 --> 11:28.340 And then once you get something that clicks and works, you tick off all of the boxes until you find 11:28.340 --> 11:34.350 the minimum required to make it work, which is going to make your life a lot easier as we in the next 11:34.350 --> 11:37.620 section make the request inside of NodeJS. 11:37.770 --> 11:42.510 So I'll see you in the next section where we are going to use request in NodeJS.