Find/Find_all

BeautifulSoup offers methods for going through HTML tags. One of them is the function .find(). It returns the first tag which matches the parameter or None if there are no matches:

print(soup.find("p"))

print(soup.find("h9"))


              12
            
print(soup.find("p"))
print(soup.find("h9"))

We will get the same result by accessing tags directly from the BeautifulSoup object: print(soup.p).

To receive the list of all occurrences of the particular tag, we can use the built-in function of the BeautifulSoup object .find_all():

python
print(soup.find_all("p"))

It returns the list of instances of the tag object provided by BeautifulSoup. Tag objects offer a comfortable interface to work with their contents.

One of the most important functions of BeautifulSoup is the ability to find the specific types of tags using their attributes:

print(soup.find_all("p", id = "id2"))

print(soup.find_all(attrs = {"class":"afterbanner", "id": "id1"}))


              12
            
print(soup.find_all("p", id = "id2"))
print(soup.find_all(attrs = {"class":"afterbanner", "id": "id1"}))

The functions .find() and .find_all() are more convenient in usage as they can work in combination with attributes and regexes.

Using scraping, you are always interested in a specific part of the website, and unique attributes can help to identify them.

Tehtävä

Swipe to start coding

In this task, you will work with the following page.

Create the BeautifulSoup object using as parameters html and "html.parser".
Print the first div tag using the function .find() of the object soup.
Print the p tag where the id equal to "id0" using the function .find_all() of the soup oblect.

Ratkaisu

# Import libraries

from bs4 import BeautifulSoup

from urllib.request import urlopen

# Open the page

url = "https://codefinity-content-media.s3.eu-west-1.amazonaws.com/18a4e428-1a0f-44c2-a8ad-244cd9c7985e/mother.html"

page = urlopen(url)

html = page.read().decode("utf-8")

# Create the BeautifulSoup object

soup = BeautifulSoup(html, "html.parser")

# Print the first div tag

print(soup.find("div"))

# Print the tag p with the correct id

print(soup.find_all("p", id = "id0"))

Oliko kaikki selvää?

Kiitos palautteestasi!

Osio 2. Luku 3