Bernoulli distribution

Previously, we covered topics like probability mass function and cumulative mass function. In this topic, we will see how we can apply such concepts to a particular discrete probability distribution, the Bernoulli distribution.

The Bernoulli distribution can be applied when there is a single experiment with only two possible outcomes, such as success/failure, red/blue, heads/tails, or true/false. Such an experiment is called a Bernoulli trial. Researchers will often denote the two possibilities as success vs. failure, with success referring to the expected or desired result. The Bernoulli distribution has applications in medicine, marketing, finance, etc.

As an example, suppose that you are in a group of four friends and you are deciding when to watch a movie together. Three out of four people are free this Saturday, but one person is not.

The "experiment" of randomly discovering whether one of these people is free on a chosen day only has two outcomes: a person will either be free or not free on that day. As "available" and "not available" are the only two outcomes, the Bernoulli distribution can be applied to this scenario.

PMF and CDF of Bernoulli Distribution

In general, Bernoulli trials must follow certain conditions:

• Bernoulli trials must be independent of each other
• Their probabilities must remain the same across the trials

A Bernoulli trial has two possible outcomes: success/failure. For success, the random variable takes value $1$ with probability $p$. For failure, it takes value $0$ with probability $1-p$. Note that if the probability of one outcome is $p$, then the opposite is $1-p$.

Looking back at the example in the introduction, if we consider availability to watch a movie as a success, then $p=0.75$ and $1-p=0.25$. Since $1/4$ is $0.25$, the probability of outcome with value $0$ is $0.25$, and the other one is $1-0.25 = 0.75$.

The probability mass function (PMF) gives the probability that a discrete random variable equals a specific value. For the Bernoulli distribution, the PMF, denoted $f$, of a discrete random variable $X$ with a parameter $p$ and evaluated at $x$ , is as follows:

$$\begin{equation*} f(x, p) = \left\{ \begin{array}{ll} p & \quad x = 1 \\ 1-p & \quad x = 0 \end{array} \right. \end{equation*}$$This means that $x$ will equal $1$ with a probability $p$ and it will equal $0$ with a probability $1-p$. It is common to use $0$ to denote an unfavorable outcome, but strictly speaking, nothing prevents us from doing otherwise.

The histogram below is the PMF based on the example described in the introduction.

As a side note, if we were flipping an ideal coin we would see the bars of the same height since the probability of getting heads and tails is equal to $0.5$ and $1-0.5$ is still $0.5$.

The cumulative distribution function (CDF) gives the probability that a real-valued random variable is less than or equal to a specific value. For the Bernoulli distribution, the CDF, $F$, of a random variable $X$ with a parameter $p$ and evaluated at $x$, is as follows:

$$\begin{equation*} F(x, p) = \left\{ \begin{array}{ll} 0 & \quad x < 0 \\ 1-p & \quad 0 \leq x < 1 \\ 1 & \quad x \geq 1 \end{array} \right. \end{equation*}$$This means that there the probability of $x < 0$ is $0$, that of $0 \leq x < 1$ is $1-p$ and that of $x \geq 1$ is $p$.

The step function below represents the CDF for the example from the introduction.

Mean and variance

Two important values in the field of probability and statistics are mean and variance. The mean is essentially a weighted average. The variance is a measure of variability in the data; that is, it tells you how spread out the data is.

The mean or expected value of the distribution is calculated as follows:

$$μ = (1-p)\cdot0+p\cdot1=p$$ The variance or the expected squared distance of a value from the mean is calculated as follows:

$$σ^2=(1-p)\cdot(0-p)^2 + p\cdot(1-p)^2 = (1-p)\cdot p^2 + p\cdot (1-2p+p^2) = p\cdot(1-p)$$As you probably noticed, the formulas in the mean and variance calculations are pretty simple. This is due to the fact that one of the outcomes is $0$ and so some terms are nixed.

Let's revisit the problem from the introduction. The mean of the experiment is:

$$μ = p = 0.75$$Consequently, the variance of the experiment is:

$$σ^2=p \cdot (1-p) = 0.75\cdot0.25 = 0.1875$$

Therefore, the expected value of the experiment amounts to $0.75$. The variance, $0.1875$, gives an indication of how far the data points are spread out from the expected value.

Symmetry of notation

In the example we used in this topic, we considered success to be when a person is free on a given date, and failure to be when they are not free. It is important to note that success and failure have nothing to do with the actual "SUCCESS, MONEY, AND FAME". 'Success' and 'failure' are just the labels we use.

So, what would happen if we assigned $0$ and $1$ differently? Let's say that success is now when a person is not free. Now, $p=0.25$ and $1-p=0.75$.

This is what the PMF would look like.

And this is what the CDF would look like.

The mean:

$$μ = p = 0.25$$The variance:

$$σ^2=p \cdot (1-p) = 0.75\cdot0.25 = 0.1875$$Thus, while the real-life situation and experiment do not change, our model changes slightly. Our diagrams and mean value change, but the variance does not.

In general, noticing symmetry in such problems is a nice skill to have since sometimes such changes might lead to easier calculations.

Conclusion

• The Bernoulli distribution has two possible outcomes:
  • Success: value $1$, probability $p$
  • Failure: value $0$, probability $1-p$
• $$\begin{equation*} \text{Probability mass function} = \left\{ \begin{array}{ll} p & \quad x = 1 \\ 1-p & \quad x = 0 \end{array} \right. \end{equation*}$$
• $$\begin{equation*} \text{Cumulative distribution function} = \left\{ \begin{array}{ll} 0 & \quad x < 0 \\ 1-p & \quad 0 \leq x < 1 \\ 1 & \quad x \geq 1 \end{array} \right. \end{equation*}$$
• Expected value: $μ = p$
• Variance: $σ^2=p \cdot (1-p)$