So remember how in the previous lectures we found out the CSS selector for item prop URL and we found
out that the content attribute actually contained the URL for the home we wanted to scrape.
Now let's try and get the HTML for each page, for each page, and then from the HTML we can use.
Cheerio.
We could also execute um, vanilla JavaScript inside of the page, but I like to use Cheerio instead
to scrape my pages.
Also, I'm going to add a try catch clause inside here.
Just to catch any errors.
Like so.
Now let's get the HTML from the page.
So const HTML await page dot evaluate.
Now if you're familiar with something like Nightmare JS, for example, this is really similar to Nightmare
JS where we basically execute JavaScript inside of the browser.
So this is just similar to going inside of your console and writing code inside here.
So it's going to execute inside of this chromium browser.
And what we want to do is we want to return the HTML of the side so we can use the inner HTML.
Now this is HTML we want to pass into Chario.
And then we can use the selector on it.
So let's try and get some homes.
We say homes and we select our
item.
Excuse me.
Item prop all the elements with the item prop attribute.
That says you're also.
Excuse me.
I need to make this a double quote so we can have the single quotes in here.
So all the elements that has the attribute origin prop with the URL inside.
And let's just run a map over this one.
So I element and for each of these we just return the elements attribute content.
So in case you forgot from the previous section, each of these homes in this page, they have a.
Let me show you.
They have a item prop or a meta element containing the item prop URL.
This is what we're selecting on and it has the content with the URL for the home or the room we're looking
for in this page.
Let's go into the code again.
So this is the property we're looking for.
I'm missing a square bracket here.
Remember to put that in if you didn't get it also.
So let's try and run this and see if we get a array of homes.
So I will run console log homes.
And this should contain an array of the different URLs from the different homes.
Let's try and run it.
So now the browser is running.
Let's see.
Okay.
Uh, I think I did the old one again.
Um, if you've seen the nightmares section, I did this also.
We need to use dot get.
Excuse me.
Dot, get in order to actually get the values.
If not, we're getting the chario objects instead.
So we're dot get it should be fine.
We should get the URLs like we want.
Now do mind if you want to.
You can also just execute some JavaScript in here which is returning a array and just get the array
directly like that.
However, I prefer to just take the whole instead of the page and pass it into Cheerio, and then I
can select as much as I want after that.
But that's just something to keep in mind, that that's also an opportunity if you don't like to use
Cheerio so much.
Now let's try and run this again and see if we get the array that we want.
Okay, so it looks like we are getting the array.
Now all that's left now is to go through each of these and basically get the descriptions and the rooms,
prices and so on for each of these rooms and homes.