WEBVTT

00:01.500 --> 00:07.830
Today, we're going to be looking at a website where you have to log in in order to be able to web scrape

00:07.830 --> 00:09.510
the content that we want to get.

00:09.510 --> 00:16.740
And the website is also expecting you to have some kind of cookie set and also pass some data from these

00:16.740 --> 00:19.020
cookies to the login form.

00:19.020 --> 00:23.970
So it's a little trickier example of how to do login with web scraping.

00:23.970 --> 00:26.880
But I'm going to show you step by step how to do this.

00:27.610 --> 00:29.890
So first we have the website.

00:29.890 --> 00:32.830
It's a internship website for Indian students.

00:32.830 --> 00:37.690
I think I had one of my students ask me about how to scrape this website.

00:38.050 --> 00:41.680
So we go on to, Well, first you need to register.

00:41.680 --> 00:48.280
If you don't have a user inside, just use a temporary email like temp email or something like that.

00:50.070 --> 00:55.740
If you don't want to use your own email, you can use like a disposable email to register inside there.

00:58.090 --> 01:00.500
So go ahead, make a user inside there.

01:00.520 --> 01:03.190
Then we go to the login form.

01:03.490 --> 01:06.130
Let me show you what we're doing here.

01:06.160 --> 01:08.770
I open up my chrome developer tools.

01:10.370 --> 01:14.150
And we go to the login form and let's watch on the network tab here.

01:15.860 --> 01:16.630
So there we go.

01:16.640 --> 01:17.810
Now log in.

01:19.090 --> 01:21.670
And then it shows the internship postings.

01:21.670 --> 01:26.740
These are the postings we have the objective of trying to scrape in our web scraper.

01:28.860 --> 01:33.270
So how do we log in using NodeJS requests on this website?

01:33.300 --> 01:35.910
Well, I'm going to show you that now step by step.