Project

Simple Text Summarization

Challenging

31 completions

~ 28 hours

4.6

You will learn how to use the TF-IDF metric for extractive text summarization, see how XML files are structured and how to parse them. You will also get familiar with a Python machine learning library scikit-learn and a library BeautifulSoup.

Provided by

JetBrains Academy

About

The modern world is rapid and dynamic, and every day people read a lot of news about politics, science, entertainment industry, programming, and so on. When there is so much information to digest, it gets really hard to deal with all of it! In this project, you will get familiar with something that might help: simple text summarization technique. This method extracts the most important sentences from the given text and is a great baseline for further experiments with other summarization approaches.

Graduate project

This project covers the core topics of the Introduction to Natural Language Processing course, making it sufficiently challenging to be a proud addition to your portfolio.

At least one graduate project is required to complete the course.

What you'll learn

Once you choose a project, we'll provide you with a study plan that includes all the necessary topics from your course to get it built. Here’s what awaits you:

Extract the first √N sentences of the text assuming that they represent the most important information in the summarized text.

Implement the tokenization and lemmatization of each news article.

Create and fit a TfIidfVectorizer object on preprocessed texts. Find the mean TF-IDF for each sentence and pick √N sentences with the highest scores.

Use your trained TfIidfVectorizer object and add extra weights for words occurring in the headline.

Reviews

Mauro Gonçalves

4 months ago

5.0

In this project, I learned how to create a simple text summarization program using Python and NLP techniques. I practiced reading XML files, extracting news headers and texts, tokenizing sentences and words, removing stopwords and punctuation, and applying lemmatization.

Raman But-Husaim

2 years ago

4.3

A really nice project that helps to grasp several NLP topics, including TF-IDF algorithm. Really enjoyed the process.

Christopher Christian

3 years ago

5.0

had a lot of fun learning more about web scraping and numpy operations. very informative.

4.6

Learners who completed this project within the Introduction to Natural Language Processing course rated it as follows:

Usefulness

4.8

Fun

4.5

Clarity

4.3