So first thing we're going to do is we're going to import puppeteer.
So puppeteer require puppeteer.
Then we'll write our main function.
So async function main.
And we'll create the browser instance.
So const browser await property and launch.
With the headless set to true to false.
Let me see.
So you can see.
All of it like that.
So const browser await property launch and we set this to false so we can see the browser when it's
launched and then we'll create a page inside of the browser.
So await browser new page.
And then inside of this page, we're going to set a specific viewport for it because, well, we're
going to be using something with the width and height.
So it's pretty useful to set an exact height and width.
And with that, we're going to go to the site with the demo page.
So the URL for this infinite scroll demo, I'm going to link the put the link in the resources as well
if you don't want to type it out and.
Then we're going to set a target amount of the items we want to get.
So target item count, we set to 100.
So this means we get 100 of these different boxes.
So once we get 100, we're done.
So it's going to scroll down and down until we get 100 items.
Then we're going to say const items.
Await scrape.
Infinite.
Infinite scroll items and pass in the page to another function we're going to write here, which is
going to extract items and then the target item count.
And
then we can console log the items.
So this function is what we're going to look at first, which is returning the items of these boxes
on the page.
So let's go ahead and write that function extract items.
And we're going to just.
Get a extracted extracted items array.
So this is where we call array from document query selector all.
Access to books.
So let me just explain a bit about what is going on here.
So this document, query selector all is a function.
We are running inside of the browser.
So if we paste that inside here, you can see a node list of 40 boxes.
So there's 40 boxes in the Dom right now, these ones.
And that's a node list.
And unfortunately, that is not an array.
We want to make it into an array so we can use dot map on it.
And so let me show you.
So we do a items and then we do extract it.
Items dot map.
And we say element return the elements Inner text.
Like so.
And then we return the items.
So.
The it's going to iterate over every of these node list items and get their inner text.
So infinite scroll backs, one, infinite scroll box two and so on.
And so the node list is transformed into an array, and then we map over the array and return each elements
in our text.
And then we finally return this array of all these text objects.
Now.
Um.
So let's see if that works.
Just for now.
Let's see.
So I comment out this.
Then we can run extract items.
Actually you can run, await, page, evaluate, and inside evaluate.
You can either pass in a string, you can pass in a string like this.
Oops.
And then we put single quotes here so we don't, um, escape the double quotes.
So like this.
You can pass in a string, which is basically a console command for the browser.
You can also pass in a function.
So let's try and see.
What we get here.
So if I just do console.log result here for this one.
And let's see.
Let me open up the terminal.
So I do note.
Index.js.
So I'm not getting anything because I'm not running the function.
Of course the main.
So down here below type main.
So we run the main function.
So now the browser is running.
Let's see what it says in here.
And here we get this list.
There's no text in it, but that's okay because it's, um.
It's a node list item you can say.
Right.
But you can also go and say extract items.
So you pass in the function here, extract items.
So now we're just passing.
Passing in this function here.
And let's see how that looks.
Now it's running the site again.
And now we get these, um, text boxes inside.
So all the text in here.
Um, so yeah, you can see it works fine.
It says something about items is not defined.
Yeah, it's this.
It's this console log here because I haven't defined items.
But anyway, you can see now that we are passing in a function to the page evaluate and it runs the
function in here.
And so I hope that it's clear for you now what exactly is going on in this section.
But in the next lecture, we're going to flesh out this function here and we're going to see how we're
scrolling the page.