Computer scienceData scienceNLPText processingText normalization

Tokenization implementation

Theory

TreebankWordTokenizer

Report a typo

You have a sentence in the input. Tokenize it with TreebankWordTokenizer.

Sample Input 1: 

Alpha Centauri is listed in the 2nd-century star catalog of Ptolemy

Sample Output 1: 

['Alpha', 'Centauri', 'is', 'listed', 'in', 'the', '2nd-century', 'star', 'catalog', 'of', 'Ptolemy']
Write a program in Python 3
# put your python code here
from nltk import TreebankWordTokenizer as TWT

text = input()
___

Create a free account to access the full topic