Project

Text Generator

Hard

741 completions

~ 19 hours

4.4

You will create a program that can predict the next word in a pseudo-sentence based on the previous words in the sequence and the data that is used to create a statistical model. You will get a deeper understanding of natural language processing and preprocessing with the NLTK library, string operations, and the application of statistics in your code.

Provided by

JetBrains Academy

About

Machine learning is getting crazy smart these days. You've probably read texts or scripts written by machine learning algorithms, some of them even mimicking styles of certain people! If you ever wondered how it works and what are the secrets behind machine learning, this project is for you! You will have a chance to understand and implement a simple text generator using Markov chains.

Graduate project

This project covers the core topics of the Introduction to Natural Language Processing course, making it sufficiently challenging to be a proud addition to your portfolio.

At least one graduate project is required to complete the course.

What you'll learn

Once you choose a project, we'll provide you with a study plan that includes all the necessary topics from your course to get it built. Here’s what awaits you:

Open the given text corpus, break the text into separate words, and obtain some properties of the corpus.

Bigrams are sequences of two consecutive words from the dataset. Transform the preprocessed corpus into a list of bigrams.

Create a Markov chain model that shows the probability of certain words appearing after a given chain of words.

Use the Markov model to generate a text starting with a user-specified word and handle exceptions.

Modify the algorithm so that sentences always start with capital letters and end with punctuation marks.

Extend the program to create a Markov model based on trigrams in order to generate more sensible sentences.

Reviews

Bojan Gjokjevski

3 months ago

5.0

Great project! Learned and understood how bigrams and trigrams work, ad how to construct sentences in NLP.

User 59778810

1 year ago

4.3

It was a really good project, I learned about text processing, some linguistic basics and how to use nltk library

User 618540273

1 year ago

4.7

It was pretty good. I learned a new concept, its implementation, and it was quite challenging from programming perspective.

4.4

Learners who completed this project within the Introduction to Natural Language Processing course rated it as follows:

Usefulness

4.5

Fun

4.5

Clarity

4.2