Computer scienceData scienceNLPNLP metricsClassic NLP metrics

BLEU

BLEU's clipped n-gram count

Report a typo

Suppose you have a following set of two references and a system output:

Reference 1

He hastily returned to the station
Reference 2 He fastly went back to the station
Output He fastly returned back to the station

Calculate the sum of the clipped count column for bigrams of the output.

Hint

Carefully review the example walk-through in the Modified precision section of the theory.

Remember: In the clipped count, we go through each bigram and count the maximum number of times it appeared in any reference. So, for the output in this task, the bigrams and their counts are given as

Output bigram Bigram count
he fastly 1
fastly returned 1
returned back 1
back to 1
to the 1
the station 1

Now, you just go through all bigrams and check if they appear in Reference 1, Reference 2, or both. If a bigram is present only once in either Reference 1 or Reference 2, it's clipped count is 1. If the bigram is present in both Reference 1 and Reference 2 (there are 2 or more occurrences of it in the references), it's clipped count is also 1. In case the output bigram is not present in either Reference 1 or Reference 2, it's clipped count is 0.

Enter a number

Create a free account to access the full topic