WEBVTT

00:00.230 --> 00:01.100
Hello, everyone.

00:01.100 --> 00:06.650
In this section, I'm going to teach you how to use puppeteer to scrape websites.

00:06.770 --> 00:13.130
Specifically, we're going to scrape the software developer jobs from Craigslist in the San Francisco

00:13.130 --> 00:14.090
Bay area.

00:14.300 --> 00:18.140
We're going to go through each jobs job listings, as you can see.

00:18.170 --> 00:25.370
Puppeteer is doing right now and scrape the content from the job description and the compensation also.

00:26.060 --> 00:30.140
So right now you can see Puppeteer is controlling the chromium browser.

00:30.170 --> 00:36.590
I'm also going to show you how to put a limit on how often it should scrape a site so that you don't

00:36.590 --> 00:39.380
risk getting blocked when you're scraping websites.

00:40.340 --> 00:45.590
And after we have been scraping all of the data using puppeteer and cheerio.

00:45.920 --> 00:54.560
I'm going to show you as well how to save our data onto a MongoDB database and how to easily and fast

00:54.560 --> 00:58.130
set up a MongoDB database using Mlab.

00:58.790 --> 01:01.370
So M-lab is free of charge to use.

01:01.400 --> 01:06.020
It's really fast and easy, so don't worry about any cost or anything like that.

01:06.110 --> 01:12.620
And I'm going to show you step by step how to do this project, and I hope you will have lots of fun

01:12.620 --> 01:14.930
and learn lots of things when doing it.