To scroll down until you can’t anymore with JavaScript Puppeteer, we use a while loop.
For instance, we write
await page.evaluate(async () => {
let scrollPosition = 0;
let documentHeight = document.body.scrollHeight;
while (documentHeight > scrollPosition) {
window.scrollBy(0, documentHeight);
await new Promise((resolve) => {
setTimeout(resolve, 1000);
});
scrollPosition = documentHeight;
documentHeight = document.body.scrollHeight;
}
});
to call page.evaluate with a callback that uses a while loop to scroll down until the documentHeight is bigger than scrollPosition.
We call scrollBy to do the scrolling down the page.
When documentHeight is bigger than scrollPosition than we can’t scroll anymore.
