Probability mass function

Before diving into theory: if two six-sided fair dice are tossed, on which sum will you bet? Does each sum value has an equal probability or are their probabilities distributed differently?

We know that usually some of the results of a probabilistic experiment are more likely to happen. Similarly, some values of our discrete random variable can occur with a higher probability. To distribute each possible value of the discrete random variable according to their likelihood, we will use Probability Mass Function, or in short PMF. We will be able to draw a plot of the values of discrete random variables with respect to their PMF.

What is PMF and where do we need it?

Let's imagine that you were given two six-sided fair dice to toss. You were asked to calculate and plot the sum of those dice values with regard to their probability of occurrence.

The plot will help us see which sum is more likely to happen. We will also see how all the sum values are distributed according to their probability. In order to plot this graph, let's reimagine the problem by using a discrete random variable. We will assign a PMF to each value of that discrete random variable.

PMF shows the probability of different possible values of our discrete random variable. The PMF sometimes is also called ''probability law'' or ''probability distribution'' of a discrete random variable. The PMF is associated only with discrete random variables. We can't utilize PMF in the case of continuous random variables.

Let's specify the notation that we will use. Uppercase letters such as $X$, $Y$, $Z$, serve to represent discrete random variables. Lowercase letters such as $x$, $y$, $z$, represent values of discrete random variables.

The formula for the PMF is the following:

Here

• $p_X(x)$ is probability mass function (PMF) notation;

• $P(X=x)$ is the probability of the event that discrete random variable $X$ takes particular values (for the fixed $x$ value, $X=x$ is an event).

Formally, PMF is the probability of all outcomes ($\omega \in \Omega$), for which the numerical value of our discrete random variable is $x$. An $x$ is the argument of PMF. It ranges over the possible values of the discrete random variable $X$. In this sense, we can clearly state that PMF is a function. That is the reason it is denoted as $p_X(x)$.

The required PMF properties are:

• $0 \leq p_X(x) \leq 1$: PMF is a probability, it must be greater than or equal to 0 for all $x$ values in a discrete random variable $X$. (Nonnegativity axiom – all probabilities are non-negative)

• $\sum_{x} p_X(x) = 1$: PMF values of all $x$ in a discrete random variable $X$ sum up to one. (Normalization axiom – the probability of a sample space is equal to one)

PMF calculation steps

Let's break down PMF calculation into several small steps.

We start by representing our experiment in terms of discrete random variables.

Before starting to calculate the PMF, we have to:

• specify the sample space ($\Omega$) of our experiment;

• specify the probability law for our experiment.

Only after that, we will be able to associate a PMF to a discrete random variable.

To calculate PMF for some arbitrary discrete random variable $X$ means to calculate $p_X(x)$ for all possible values of $x$ in the range of the discrete random variable $X$. Simply put, we need to repeat PMF calculation for all possible $x$ in a given discrete random variable $X$.

The next step is to make sure that our calculations meet the properties that are required for PMF.

Finally, we could produce a PMF plot as $p_X(x)$'s for all possible $x$ values and hence distribute the probabilities for each possible $x$.

PMF calculations for the sums of two fair dice in a single toss

Let's go back to our main task. We have two six-sided fair dice to toss and need to plot the sum of those dice with regard to their occurrence probability. In other words, we need to plot probability distribution (PMF) values of the sum of two six-sided fair dice on a graph. We will follow the calculation steps explained above.

Step 1: represent our experiment in terms of discrete random variables.

We will represent this problem in terms of discrete random variables as:

• $X$ will be a discrete random variable for the first die with values $x\in\{1,2,3,4,5,6\}$;

• $Y$ will be a discrete random variable for the second die with values $y\in\{1,2,3,4,5,6\}$;

• $Z=X+Y$, where $Z$ will be the sum of the first and the second die, also a discrete random variable with values $z\in\{2,3,4,5,6,7,8,9,10,11,12\}$.

To see a clearer picture, let's illustrate it as a table:

Step 2: specify the sample space ($\Omega$) of our experiment.

From the table above, we can see that all possible choices consist of 36 values (6 x 6) total, which are $\{2,3,4,5,6,7,3,4,5,6,7,8,4,5,6,7,8,9,5,6,7,8,9,10,6,7,8,9,10,11,7,8,9,10,11,12\}$Based on those choices, our Sample Space ($\Omega$) for the sum of two dice will be the following:

$\Omega=\{(1, 1), (1, 2), (1, 3), (1, 4), (1, 5), (1, 6) ,\\ (2, 1), (2, 2), (2, 3), (2, 4), (2, 5), (2, 6),\\ (3, 1) , (3, 2), (3, 3), (3, 4), (3, 5), (3, 6), \\(4, 1), (4, 2), (4, 3), (4, 4), (4, 5), (4, 6),\\ (5, 1),(5, 2), (5, 3), (5, 4), (5, 5), (5, 6), \\(6, 1), (6, 2), (6, 3), (6, 4), (6, 5), (6, 6)\}$

Step 3: specify the probability law for our experiment.

Since we were told that both dice are fair, we will use the Discrete Uniform Law. The Discrete Uniform Law states that each element in a finite sample space is equally likely to happen. It means each element in our sample space has a probability of $\frac{1}{36}$, since our sample space consists of 36 elements.

Step 4: associate a PMF to a discrete random variable.

The probability of each element in our Sample Space ($\Omega$) is $\frac{1}{36}$. Now, let's apply the formula for the PMF for each value $z\in\{2,3,4,5,6,7,8,9,10,11,12\}$ in our discrete random variable $Z$ as:

$p_Z(z) = P(Z=z) = P(\{\omega \in \Omega\ such\ that Z(\omega)=z\})$

Step 5: repeat PMF calculation.

We will repeat the PMF calculation for all possible $z$ values as:

$p_Z(2) = P(Z=2)=\frac{1}{36} \hspace{0.2cm} when \hspace{0.2cm} z=2$

$p_Z(3) = P(Z=3)=\frac{2}{36} \hspace{0.2cm} when \hspace{0.2cm} z=3$

$p_Z(4) = P(Z=4)=\frac{3}{36} \hspace{0.2cm} when \hspace{0.2cm} z=4$

$p_Z(5) = P(Z=5)=\frac{4}{36} \hspace{0.2cm} when \hspace{0.2cm} z=5$

$p_Z(6) = P(Z=6)=\frac{5}{36} \hspace{0.2cm} when \hspace{0.2cm} z=6$

$p_Z(7) = P(Z=7)=\frac{6}{36} \hspace{0.2cm} when \hspace{0.2cm} z=7$

$p_Z(8) = P(Z=8)=\frac{5}{36} \hspace{0.2cm} when \hspace{0.2cm} z=8$

$p_Z(9) = P(Z=9)=\frac{4}{36} \hspace{0.2cm} when \hspace{0.2cm} z=9$

$p_Z(10) = P(Z=10)=\frac{3}{36} \hspace{0.2cm} when \hspace{0.2cm} z=10$

$p_Z(11) = P(Z=11)=\frac{2}{36} \hspace{0.2cm} when \hspace{0.2cm} z=11$

$p_Z(12) = P(Z=12)=\frac{1}{36} \hspace{0.2cm} when \hspace{0.2cm} z=12$

Step 6: make sure that our calculations match properties that are required for PMF.

Let's check if our calculations match PMF's required properties:

• All of $p_Z(z) \geq 0$. We can see that all our calculations of PMF ($p_Z(z)$) values, where $z\in\{2,3,4,5,6,7,8,9,10,11,12\}$ are greater than zero. That means we meet this requirement.

• Probability of all values z in our discrete random variable $Z$ sum up to one as: $\sum_z p_Z(z)= 1$

Let's check it:

And calculate:

Once again, we meet the required property for the PMF.

Step 7: produce a PMF plot.

Now we can plot a graph. Values, $z\in\{2,3,4,5,6,7,8,9,10,11,12\}$, of our discrete random variable $Z$, are plotted based on their PMF values.

Conclusion

Let's summarize the concepts we have learned in this topic.

• Probability Mass Function (PMF) is also called ''probability law'' or ''probability distribution''. It gives us the probability of different possible values of our discrete random variable.

• Mathematically speaking, the PMF is the probability of all outcomes for which the numerical value of our discrete random variable is $x$, where $x$ is the argument of the PMF, and it ranges over the possible values of the random variable $X$.

• Two required properties of PMF: since PMF is a probability, it must be greater than or equal to 0 for all $x$ values in $X$. PMF values of all $x$ in a discrete random variable $X$ sum up to one.

• PMF values are used to plot a graph showing the distribution of probabilities of each value in a discrete random variable.

• PMF can be used only with discrete random variables and can't be used with continuous random variables.