How to use Python Requests with JavaScript pages?

Sometimes, we want to use Python Requests with JavaScript pages.

In this article, we’ll look at how to use Python Requests with JavaScript pages.

How to use Python Requests with JavaScript pages?

To use Python Requests with JavaScript pages, we can create an HTMLSession with requests_html.

To install it, we run

pip install requests-html

Then, we write

from requests_html import HTMLSession

session = HTMLSession()
r = session.get('http://www.example.com')
r.html.render() 

to create an HTMLSession.

Then we call get with the URL we want to make a request to.

And then we call html_render to render the page with JavaScript.

Conclusion

To use Python Requests with JavaScript pages, we can create an HTMLSession with requests_html.