Computer scienceData scienceNLPText processingText normalization

Tokenization implementation

Theory

Regular expressions and tokens

Report a typo

There is a sentence in the input. Use the pattern "[A-z'-]+" to print the tokens.

Tip: Do not forget to import the module you need

Sample Input 1: 

The more you read, the more you learn.

Sample Output 1: 

['The', 'more', 'you', 'read', 'the', 'more', 'you', 'learn']
Write a program in Python 3





___

Create a free account to access the full topic