Rules of sum, product, and complement

Let's start this topic with the definition of dependent and independent events.

Dependence and independence of two events

Events $A$ and $B$ are independent if $\mathbb{P}(A) = \mathbb{P}(A|B)$. In other words, the occurrence of one does not affect the probability of the occurrence of the other.

If we unfold $\mathbb{P}(A|B)$ in the definition, we can rewrite the definition of independent events: $\mathbb{P}(A|B) = \frac{\mathbb{P}(A \cap B)}{\mathbb{P}(B)}$ , and we can get the following equality $\mathbb{P}(A \cap B) = \mathbb{P}(A) \ \mathbb{P}(B)$. This equality can be used as the definition of independent events.

The following events are independent:

• $A$ $-$ rolling 4 on a single 6-sided die, $B$ $-$ rolling 1 on a second roll of the die;

• $2$ dice were rolled. $A$ $-$ the number of points on the first one is greater than $3$, $B$ $-$ number of points on the second one is greater than $3$;

• Whole sample space $\Omega$ and event $A$ with zero probability.

We will see some interesting properties of independent events in the next sections.

The pairwise and mutual independence

Now we know what the phrase "events $A$ and $B$ are independent" means. But what does the phrase "events $A_1, A_2, \dots, A_n$ are independent" mean? With many events, it's impossible to understand the type of independence.

Events $A_1, A_2, \dots, A_n$ are pairwise independent if every pair of events is independent: $\mathbb{P}(A_i \cap A_j) = \mathbb{P}(A_i) \ \mathbb{P}(A_j)$ for all $i \neq j$.

Events $A_1, A_2, \dots, A_n$ are mutually independent if for every $k \leq n$ and for any subset of events of size $k$ the following equality is true: $\mathbb{P}(A_{i_1} \cap A_{i_2} \cap {\dots} \cap A_{i_k}) = \mathbb{P}(A_{i_1}) \ \mathbb{P}(A_{i_2}) \ \dots \ \mathbb{P}(A_{i_k})$.

It's easy to understand that if events $A_1, A_2 \dots, A_n$ are mutually independent, then they are pairwise independent too (put $k = 2$), but the converse is not necessarily true.

Let's look at Bernstein's example, which shows the difference between these two definitions.

Suppose we have a regular tetrahedron which has one side colored red, the second side $-$ green, the third one $-$ blue and the last contains all three colors. The tetrahedron is dropped on the desk. There are three events: $R$ $-$ the lower side contains red, $B$ $-$ it contains blue, and $G$ $-$ it contains green.

It's clear that $\mathbb{P}(R) = \mathbb{P}(B) = \mathbb{P}(G) = \frac{1}{2}$ ($2$ of $4$ sides are suitable) and $\mathbb{P}(R \cap G) = \frac{1}{4}$. So events $R$ and $G$ are independent because $\mathbb{P}(R \cap G) = \mathbb{P}(R) \ \mathbb{P}(G) = \frac{1}{2} \cdot \frac{1}{2} = \frac{1}{4}$. For other pairs, similarly. So, events $R, G, B$ are pairwise independent.

Let's check the definition of mutual independence: $\mathbb{P}(R \cap G \cap B) = \frac{1}{4}$, but $\mathbb{P}(R)\mathbb{P}(B)\mathbb{P}(G) = \frac{1}{2} \cdot \frac{1}{2} \cdot \frac{1}{2} = \frac{1}{8} \neq \frac{1}{4}$. So these events are not mutually independent.

Rules of sum, product, and complement

Rule of sum

The rule of the sum is the following equality: $\mathbb{P}(A \cup B) = \mathbb{P}(A) + \mathbb{P}(B) - \mathbb{P}(A \cap B)$ . We can understand this rule intuitively just by looking at the picture:

We need to subtract $\mathbb{P}(A \cap B)$ because we added it twice in $\mathbb{P}(A) + \mathbb{P}(B)$.

But if the events $A$ and $B$ are mutually disjoint, which means $A \cap B = \varnothing$, so the rule of sum looks simpler: $\mathbb{P}(A \cup B) = \mathbb{P}(A) + \mathbb{P}(B)$.

Let's look at an example. A fair $6$-side die is rolled. What is the probability that the roll is an even number (event $A$) or a prime (event $B$) or both? It's clear that $\mathbb{P}(A) = \mathbb{P}(B) = \frac{1}{2}$ and $\mathbb{P}(A \cap B) = \frac{1}{6}$ ($2$ points). So, the answer $\mathbb{P}(A \cup B) = \frac{1}{2} + \frac{1}{2} - \frac{1}{6} = \frac{5}{6}$.

Summarizing, the rule of sum helps us calculate the probability that an event $A$ or an event $B$ or both would happen.

Rule of product

Actually, we saw this rule in the first paragraph, but let's write it one more time.

We know that $\mathbb{P}(A \cap B) = \mathbb{P}(A |B) \ \mathbb{P}(B)$ in general case. But if $A$ and $B$ are independent, then $\mathbb{P}(A \cap B) = \mathbb{P}(A) \ \mathbb{P}(B)$. The last equality is called the rule of product. We can calculate the probability that event $A$ and event $B$ will happen using this rule.

If you read other topics related to events and probabilities, you have already seen an example of rolling $2$ dice. And before you had to count all pairs, but now it's not necessary.

For example, what is the probability that each die will have a number of points greater than $3$? For each die, this probability is $\frac{1}{2}$, so the answer is $\frac{1}{2} \cdot \frac{1}{2} = \frac{1}{4}$.

Notice once again, we could use this rule only when $A$ and $B$ are independent!

Complement rule

Suppose that we have some event $A$. We can consider the set $A^C := \Omega \setminus A$ of all outcomes not contained in $A$. This set is called the complement of $A$. From the definition, it immediately follows that $A^C$ doesn't intersect with $A$ and $A \cup A^C = \Omega$. We can use the rule of the sum to get the complement rule: $1 = \mathbb{P}(\Omega) = \mathbb{P}(A \cup A^C) = \mathbb{P}(A) + \mathbb{P}(A^C)$.

This equality is called the complement rule: $\mathbb{P}(A) + \mathbb{P}(A^C) = 1$.

By this rule, we can calculate the probability that event $A$ won't happen. For example, if it's known that the probability of rain today is $0.2$, then the probability that it won't rain is $1 - 0.2 = 0.8$.