Binomial distribution

Binomial distributions help calculate the likelihood of success in repeated experiments. They're useful in real-life situations, such as predicting a hockey team's wins or goals scored in a season.

Imagine that you're into hockey, and you would like to calculate how many games your favorite team can win or the number of goals scored in a season. How to do it? You should apply the binomial distribution!

When to apply the binomial distribution?

Let's dive deeper into the mathematics behind binomial distribution.

You can apply this distribution in a series of experiments if:

1. The outcome of every experiment is either a success or a failure
2. The probability of success is exactly the same from one trial to another

The first condition means that you can apply this distribution only to situations like the following:

• winning a match: it's either won or lost
• getting a head when tossing a coin: you either get it or not
• buying a lottery ticket: you either win or not

The second condition means that the binomial distribution only works if the probability of success stays the same from one trial to the next. Card games like UNO don't fit this requirement because the probability of winning changes with the randomly dealt cards and players' skills, which can vary from game to game.

The second condition may also be not satisfied if the outcome of one observation affects the outcome of another, that is, the observations are dependent. For example, if you measure the height of a growing plant, the height on one day is dependent on the height of the plant on the previous day, so the observations are dependent, and the probability of success changes from one trial to another.

These conditions remind you of Bernoulli distribution, don't they? It shouldn't be something surprising because the Bernoulli distribution is the binomial distribution with only one trial. How to choose between them?

Probability mass function of the binomial distribution

Let's discuss this distribution a little bit more formally. The likelihood of seeing $x$ successful outcomes in a total of $n$ observations, where the probability of success in each individual attempt is $p$ is represented by the following probability mass function (PMF):

$$\begin{align*} f(x,p,n) &= {n \choose x} \cdot p^x \cdot(1-p)^{n-x}, \\ {n \choose x} &= \frac{n!}{x!(n-x)!}, \; x = 0,\dots , n \end{align*}$$

Why does this formula look like this? The ${n \choose x}$ is called the binomial coefficient and read as $n$ choose $x$. It is equal to the number of ways, disregarding order, that $x$ objects can be chosen from among $n$ objects. $p^x$ is equal to getting exactly $x$ successes and $(1-p)^{n-x}$ is equal to getting exactly $n - x$ failures, since the probability of each failure is $1-p$.

Let's apply this PMF to model the following situation. Your favorite hockey team is playing $5$ matches. This season's team is not that good, so the probability of winning is equal to $0.5$ in every match. They need to win $3$ matches to get into the playoffs. So, what is the probability of winning exactly $3$ matches? In order to model this situation, use the formula of the PMF with $n = 5$, $x = 3$ and $p = 0.5$.

$${5 \choose 3} = \frac{5!}{3!(5-3)!}= \frac{5\cdot4\cdot3\cdot2\cdot1}{3\cdot2\cdot1\cdot2\cdot1} = 10$$$$f(3,0.5,5) = 10 \cdot 0.5^3 \cdot(1-0.5)^{(5-3)} = 0.3125$$

Cumulative distribution function of the binomial distribution

So, now you know how to calculate the probability of getting the exact number of successes in a series of trials. But what should you do to calculate the probability of getting a number of successes less than or equal to a certain value? This situation may occur if you would like to calculate the probability that the team from the previous paragraph won't play in the playoffs because they'll win $2$ or fewer matches. The cumulative distribution function (CDF) of the binomial distribution help in situations like this one. It is denoted using the following formula:

$$F(k,p,n) =\sum_{i=0}^k {n \choose i} \cdot p^i \cdot(1-p)^{n-i}, \; k=0,\dots, n$$This function may seem a little bit complicated, let's dive into it together. In order to calculate the value of the CDF for any $k$ you should just calculate values of ${n \choose i} \cdot p^i \cdot(1-p)^{(n-i)}$ for every $i, 0\leq i\leq k$ and sum these values. We can derive this formula if we integrate the PMF of the binomial distribution, but it might be really hard, so we won't do it. In fact, the CDF of the binomial distribution is included in many packages for different computer programming languages or can be found using many online calculators like MedCalc.

Coming back to our hockey team, how should you apply this formula? So, if the team wins $2$ or fewer matches, they can win $0,1$ or $2$ matches:

$$F(2,0.5,5) =\sum_{i=0}^2 {5 \choose i} \cdot 0.5^i \cdot(1-0.5)^{5-i} = {5 \choose 0} \cdot 0.5^0 \cdot(1-0.5)^{5-0}+ \\{5 \choose 1} \cdot 0.5^1 \cdot(1-0.5)^{5-1}+{5 \choose 2} \cdot 0.5^2 \cdot(1-0.5)^{5-2}=\\ =0.03125+0.15625+0.3125=0.5$$As you see, unfortunately, your favorite team is more likely not to play in the playoffs.

Now you can calculate the probability of winning the exact number of matches or the probability of winning the number of matches less than or equal to a certain value. Imagine that you would like to compare your favorite team with your friends James and Mary's favorite teams. In this case, you'll have to solve a more general problem about hockey teams. Let's consider that all teams play $100$ games a season. The chance of winning each game is equal to $0.2, 0.5, 0.9$ in the case of James', yours, and Mary's favorite teams respectively. Let's plot the cumulative distribution function for each case!

To solve this problem, you have to calculate the values of the CDF for several points in the range from $0$ to $100$ for every value of the chance of winning each game. You see that the graph of the CDF moves to the right if the value of $p$ increases, and it means that the bigger the value of $p$ is, the more matches are likely to be won. So, it seems that Mary's team is going to be the champion this season! You also can see, that Mary's team will definitely win more than $80$ matches, while James' will definitely win more than $10$.

Moments of the binomial distribution

Now you know what the binomial distribution is and how to calculate its PMF and CDF. What else can you calculate using it? Coming back to our hockey examples, you might be interested to know how many matches your favorite team will win this season. The expected value of the binomial distribution will help answer your question!

The expected value of the binomial distribution with $n$ trials represents the average outcome you can expect and can be calculated using the following formula:

$$\mu = n \cdot p$$And for your favorite team $$\mu = 100 \cdot0.5=50$$

Considering the expected value, you can contemplate the degree to which the number of matches won can deviate from it. The variance and standard deviation of the binomial distribution will help you answer this question. They can be found using the following formulas:

$$\sigma ^2 = n\cdot p \cdot(1-p) \\ \sigma = \sqrt {n\cdot p \cdot(1-p)}$$In fact, the standard deviation is more useful than the variance, because confidence intervals are usually built using it. These intervals estimate a range for a random variable described by a distribution. But the variance and standard deviation are highly related to each other, so we shouldn't omit them.

As we've mentioned above, the Bernoulli distribution is the binomial distribution with only one trial, so its expected value is equal to $p$, variance is equal to $p\cdot(1-p)$ and standard deviation is equal to $\sqrt{ p\cdot(1-p)}$. You don't have to remember all formulas for the binomial distribution if you remember the formulas for the Bernoulli distribution. Just multiply them by $n$ in case of the expected value or the variance, or by $\sqrt n$ in case of the standard deviation.

Let's look once more at the plots in the previous section and apply our knowledge of the moments of the binomial distribution.

The most probable number of successes is near the expected value of the distribution, so you see an increase in the graph, and the bigger the value of $p$ is, the more the graph shifts to the right. As for other parts of the graph, if the values of $p$ are not close to $0$ and $1$, it's quite unlikely to observe a small number of successes or a number close to $100$, so these parts are nearly horizontal.

And here's one more example about hockey. In general $15\%$ of all people are left-handed, and $85\%$ are right-handed. A hockey team has $25$ players. Let's calculate the expected value of left-handed players, its variance, and standard deviation. In this case, a "trial" is every player, and a "success" is the fact that the person is left-handed. The probability of "success" is equal to $0.15$.

$$\mu = n \cdot p = 25 \cdot 0.15=3.75 \\ \sigma ^2 = n\cdot p \cdot(1-p) = 25 \cdot 0.15\cdot (1-0.15)=3.1875 \\ \sigma = \sqrt {n\cdot p \cdot(1-p)} = \sqrt{3.1875} \approx 1.79$$

Conclusion

Let's sum up all the most essential facts about binomial distribution.

• The binomial distribution can be applied if the outcome of every experiment is either a success or a failure and the probability of success is exactly the same from one trial to another
• Bernoulli distribution is a binomial distribution with only $1$ trial.
• PMF of the binomial distribution:$$\text{PMF} = {n \choose x} \cdot p^x \cdot(1-p)^{n-x}, \\ {n \choose x} = \frac{n!}{x!(n-x)!}, \; x = 0,\dots, n$$
• CDF of the binomial distribution:$$\text{CDF} =\sum_{i=0}^k {n \choose i} \cdot p^i \cdot(1-p)^{(n-i)}, \; k=0,\dots ,n$$
• The expected value of the binomial distribution:$$\mu = n \cdot p$$
• Variance and standard deviation of the binomial distribution: $$\sigma ^2 = n\cdot p \cdot(1-p) \\ \sigma = \sqrt {n\cdot p \cdot(1-p)}$$