Project

Text Generator

Hard

737 completions

~ 19 hours

4.4

You will create a program that can predict the next word in a pseudo-sentence based on the previous words in the sequence and the data that is used to create a statistical model. You will get a deeper understanding of natural language processing and preprocessing with the NLTK library, string operations, and the application of statistics in your code.

Provided by

JetBrains Academy

About

Machine learning is getting crazy smart these days. You've probably read texts or scripts written by machine learning algorithms, some of them even mimicking styles of certain people! If you ever wondered how it works and what are the secrets behind machine learning, this project is for you! You will have a chance to understand and implement a simple text generator using Markov chains.

Training project

This project allows you to practice and strengthen your coding skills, helping you get ready for more advanced tasks ahead.

What you'll learn

Once you choose a project, we'll provide you with a study plan that includes all the necessary topics from your course to get it built. Here’s what awaits you:

Open the given text corpus, break the text into separate words, and obtain some properties of the corpus.

Bigrams are sequences of two consecutive words from the dataset. Transform the preprocessed corpus into a list of bigrams.

Create a Markov chain model that shows the probability of certain words appearing after a given chain of words.

Use the Markov model to generate a text starting with a user-specified word and handle exceptions.

Modify the algorithm so that sentences always start with capital letters and end with punctuation marks.

Extend the program to create a Markov model based on trigrams in order to generate more sensible sentences.

Reviews

User 59778810

5 months ago

4.3

It was a really good project, I learned about text processing, some linguistic basics and how to use nltk library

User 618540273

7 months ago

4.7

It was pretty good. I learned a new concept, its implementation, and it was quite challenging from programming perspective.

Jerome Schmutz

7 months ago

3.3

The project is fun, and it's a good practice to understand better the theory.

4.4

Learners who completed this project within the Chatbots with Python course rated it as follows:

Usefulness

4.5

Fun

4.5

Clarity

4.2