Notice: This page requires JavaScript to function properly.
Please enable JavaScript in your browser settings or update your browser.
TF Score | Text Summarization with TF-ISF
Text Summarization with TF-ISF
course content

Course Content

Text Summarization with TF-ISF

bookTF Score

Term Frequency (TF) is a measure that quantifies the importance of a word within a specific sentence or document, relative to the sentence or document's length. In essence, it's a way to highlight how frequently a word appears, adjusted for the size of the text to ensure fairness across texts of different lengths.

TF is calculated using a logarithmic scale to dampen the effect of very high frequencies, which helps maintain a balanced importance across all words. The formula used here is log(1 + (frequency of the word in the sentence) / (total number of words in the sentence)). This adjustment accounts for the intuition that the significance of a word to a sentence does not increase linearly with its frequency.

For each sentence in our list of tokenized sentences (tokenized_sentences), we calculate the TF score for every unique word. This is achieved by iterating through each word in a sentence, calculating its frequency relative to the sentence length, and applying the logarithmic formula. The result is a dictionary for each sentence, mapping words to their respective TF scores.

Task

Calculate the term frequency (TF) of each word in each sentence.

Switch to desktopSwitch to desktop for real-world practiceContinue from where you are using one of the options below
Everything was clear?

How can we improve it?

Thanks for your feedback!

Term Frequency (TF) is a measure that quantifies the importance of a word within a specific sentence or document, relative to the sentence or document's length. In essence, it's a way to highlight how frequently a word appears, adjusted for the size of the text to ensure fairness across texts of different lengths.

TF is calculated using a logarithmic scale to dampen the effect of very high frequencies, which helps maintain a balanced importance across all words. The formula used here is log(1 + (frequency of the word in the sentence) / (total number of words in the sentence)). This adjustment accounts for the intuition that the significance of a word to a sentence does not increase linearly with its frequency.

For each sentence in our list of tokenized sentences (tokenized_sentences), we calculate the TF score for every unique word. This is achieved by iterating through each word in a sentence, calculating its frequency relative to the sentence length, and applying the logarithmic formula. The result is a dictionary for each sentence, mapping words to their respective TF scores.

Task

Calculate the term frequency (TF) of each word in each sentence.

Switch to desktopSwitch to desktop for real-world practiceContinue from where you are using one of the options below
Section 1. Chapter 7
some-alt