Introduction to NLP

1. Text Preprocessing Fundamentals

We will start our journey with learning and implementing the most common text preprocessing techniques used in NLP to convert the initial raw text into a clean, standardized form.

NLP and Its Applications

Understanding Text Preprocessing

Tokenization

Tokenize a Sentence

Tokenization Using Regular Expressions

Tokenize Using Regex

Removing Stop Words

Applying Text Preprocessing in Practice

2. Stemming and Lemmatization

Without further ado, let's explore stemming and lemmatization. These techniques may enhance efficiency and effectiveness of some NLP tasks, especially when working with large text corpora and treating different words forms as the same word.

Stemming

Stem the Tokens

Lemmatization

Part of Speech Tagging

Lemmatization with POS Tagging

3. Basic Text Models

Preprocessed text should then be transformed into a numerical representation to be used in machine learning or deep learning models for various tasks such as prediction, classification, or clustering. Here, we will learn to implement the most basic yet popular text models that transform text data into numbers.

Overview of Vector Space Models

Key Types of Vector Space Models

Bag of Words

Customizing Bag of Words

Create a Bag of Words

TF-IDF

Implementing TF-IDF

4. Word Embeddings

Time to unlock the power of word embeddings and delve into advanced techniques for capturing semantic relationships between words. We will explore various embedding models such as Word2Vec, GloVe, and FastText, with a particular focus on the Word2Vec model and its implementation.

Word Embeddings Basics

CBoW and Skip-gram Models (Optional)

Implementing Word2Vec

Create Word Embeddings

Create a Bag of Words

Task

Your task is to display the vector for the 'graphic design' bigram in a BoW model:

Import the CountVectorizer class to create a BoW model.
Instantiate the CountVectorizer class as count_vectorizer, configuring it for a frequency-based model that includes both unigrams and bigrams.
Utilize the appropriate method of count_vectorizer to generate a BoW matrix from the 'Document' column in the corpus.
Convert bow_matrix to a dense array and create a DataFrame from it, setting the unique features (unigrams and bigrams) as its columns. Assign this to the variable bow_df.
Display the vector for 'graphic design' as an array, rather than as a pandas Series.

Everything was clear?

Section 3. Chapter 5

Course Content

Introduction to NLP

Create a Bag of Words

Task

Your task is to display the vector for the 'graphic design' bigram in a BoW model:

Import the CountVectorizer class to create a BoW model.
Instantiate the CountVectorizer class as count_vectorizer, configuring it for a frequency-based model that includes both unigrams and bigrams.
Utilize the appropriate method of count_vectorizer to generate a BoW matrix from the 'Document' column in the corpus.
Convert bow_matrix to a dense array and create a DataFrame from it, setting the unique features (unigrams and bigrams) as its columns. Assign this to the variable bow_df.
Display the vector for 'graphic design' as an array, rather than as a pandas Series.

Everything was clear?

Section 3. Chapter 5