Ways to Scrape Table

There are a lot of different ways to scrap tables. The method depends on the structure of the table.

You can apply string methods. As you remember, the function find_all() looks through a tag’s descendants and retrieves all descendants that match the parameter. If we apply it to the tag <tr> the result will be a list of contents of each <tr> tag.

You can also do it using XPath:

rows = html.xpath('//tr')

Then we can clean the data and convert it to the DataFrame. This method can be useful if the table has a complex and confusing structure.