Sometimes, we want to do web scraping with Python.
In this article, we’ll look at how to do web scraping with Python.
How to do web scraping with Python?
To do web scraping with Python, we can use BeautifulSoup.
To install it, we run
pip install beautifulsoup4
Then we use it by writing
import urllib2
from bs4 import BeautifulSoup
soup = BeautifulSoup(urllib2.urlopen('http://example.com').read())
for row in soup('table', {'class': 'spad'})[0].tbody('tr'):
tds = row('td')
print(tds[0].string, tds[1].string)
to open the page at the URL with urlopen.
And then we call read to convert the response into a HTML string.
Next, we use the BeautifulSoup class with the string to create the soup object.
And then we get the table element with soup and then we get the tr element in the table with tbody.
Then we get the td’s in the tr element with row.
And then we get the text of the td’s with string.
Conclusion
To do web scraping with Python, we can use BeautifulSoup.
