Notice: This page requires JavaScript to function properly.
Please enable JavaScript in your browser settings or update your browser.
Impara Clean and Convert | Fake News
Identifying Fake News

book
Clean and Convert

We have decided to create a complete chapter on the topic of text cleaning and preprocessing. As you may imagine, complete texts cannot be directly fed into an ML model. For this reason, we will apply specific preprocessing techniques.

The first step will be to remove punctuation from our column to reduce noise in our data. We will do this using regex (regular expression matching operations).

Then we will vectorize our text. Refer to the picture below for more information. Essentially, we will represent words, sentences, or even larger units of text as vectors.

Compito

Swipe to start coding

  1. Remove punctuaction with regex by using the appropriate method to replace the given pattern with an empty string.

  2. Vectorize the texts of the articles.

Soluzione

from sklearn.feature_extraction.text import TfidfVectorizer

# Remove punctuation with regex
news['text'] = news['text'].str.replace('[^\w\s]', '', regex=True)

vectorization = TfidfVectorizer()

# Vectorize the texts of the articles
texts_vectorized = vectorization.fit_transform(news['text'])

Mark tasks as Completed
Tutto è chiaro?

Come possiamo migliorarlo?

Grazie per i tuoi commenti!

Sezione 1. Capitolo 4

Chieda ad AI

expand
ChatGPT

Chieda pure quello che desidera o provi una delle domande suggerite per iniziare la nostra conversazione

some-alt