Measures of location: expected value

In the topic Random variables, we saw that random variables can be well described by a set of their realizations. For example, to understand the height of a random person, one can take a set of heights of a large number of people and construct a histogram. However, in real life, describing random phenomena using a large set of realizations or bar charts/histograms is not always convenient. Often, we want to know just one number that would describe where the bulk of the values is located. In probability theory and statistics, such characteristics of random variables are called measures of location or measures of central tendency. In this topic, we will introduce several of them: the expected value and the law of large numbers.

Expected value

Suppose that we own a restaurant, and we are interested in the profit that the next 100 customers will bring us. Clearly, we cannot determine this value exactly, but we can easily estimate it:

1. Look at the profit that each of the previous customers brought us.
2. Calculate the average profit: we sum up the profits of the previous customers and divide by their number.
3. Estimate the future profit of 100 customers by multiplying the calculated average profit by 100.

In Python, it might look something like this:

past_customers_profits = [10, 35, 12, 14, 54, 32, 45, 11, 41, 7]
average_profit = sum(past_customers_profits) / len(past_customers_profits)
future_customers = 100
print(f'Future profit: ≈{future_customers * average_profit}')

Future profit: ≈2610.0

Alternatively, you can use the numpy.mean method to calculate the average profit.

import numpy

print(numpy.mean(past_customers_profits))

2610.0

Law of large numbers

The average profit that we calculated in step 2 depends on the specific profit that each previous customer brought us: for example, if we own not one restaurant but a chain of 10 restaurants, their average profits will differ since their customers differ. Let's see how this looks in the following example. Suppose that the profit of a random customer in each of 10 restaurants follows an exponential distribution with parameter $\lambda = 1$, and let's see what their average profits will be if restaurants calculate them based on the last 10, 100, or 1000 previous customers:

from random import expovariate

def past_customers_profits(number_of_past_customers):
    return [expovariate(1) for i in range(number_of_past_customers)]

NUMBER_OF_PAST_CUSTOMERS = [10, 100, 1000]
NUMBER_OF_RESTAURANTS = 10

for num in NUMBER_OF_PAST_CUSTOMERS:
    print(f'Number of past customers: {num}')
    for i in range(NUMBER_OF_RESTAURANTS):
        profits = past_customers_profits(num)
        average_profit = sum(profits) / len(profits)
        print(round(average_profit, 4), end=' ')
    print('\n')

Number of past customers: 10
1.1687 1.1914 1.502 1.0502 0.8826 1.1807 0.9438 1.0653 0.9084 0.581 

Number of past customers: 100
0.997 1.0553 0.9809 0.9306 1.135 1.1536 0.861 1.0772 1.0035 0.8586 

Number of past customers: 1000
1.0381 1.0262 1.0289 1.0267 0.9765 1.092 0.9872 1.0433 0.9685 0.9875 

As we can see, the more previous customers we take, the closer the average profits are to each other: in the case of 10 customers, the range is from 0.58 to 1.5, and in the case of 1000 customers, the range is from 0.97 to 1.04. It turns out that if we take more and more customers, the average profits will tend towards a certain number. This number does not depend on the specific profits of each customer, and is called the expected profit of a random customer.

We say that the Law of large numbers holds for a random variable $X$ if for any infinite sequence of its realizations $x_1, x_2, \dots$, the average value of the first $k$ realizations $\frac 1k (x_1 + \dots + x_k)$ tends towards the same number as $k$ approaches infinity. This number is called the expected value of $X$ and denoted as $\mathbb{E}X$.

Borel's law of large numbers

In the topic Probabilities as frequencies, we discussed Borel's law of large numbers. This is a special case of the Law of large numbers. To see this, let's fix some event $A$ and consider a random variable $X_A$ that equals 1 if $A$ occurs for a given outcome, and 0 otherwise. In this case, $\frac 1k (x_1 + \dots + x_k)$ is the proportion of outcomes where $A$ occurred. Since $X_A$ takes values within a bounded interval, the Law of large numbers holds, and thus

$$\frac {x_1 + \dots + x_k}k \to \mathbb{E}X_A.$$

Since the same limit is equal to $\mathbb{P}(A)$, we obtain that $\mathbb{E}X_A = \mathbb{P}(A)$.

The expected value is a convenient measure of location when dealing with sums. However, there are problems in which the expected value provides little information. Suppose we want to know how much the typical employee earns in two companies. In the first company, let there be 10 employees with the same salary of $50,000 per year, and in the second company, let there be 10 employees, one of whom earns $\$$275,000 per year, while the other 9 earn $\$$25,000 per year. Let's calculate the average salaries:

import numpy as np

first = [50000] * 10
second = [275000] + [25000] * 9

print(np.mean(first), np.mean(second))

50000.0 50000.0

As you can see, the average salaries are the same, although it is obvious that a typical employee of the first company earns twice as much as an employee of the second one. A more informative measure of location for this problem will be the median. But this and more is to follow in the next topic.

Conclusion

Time for a recap of this lesson:

• Measures of central tendency, like the expected value, provide a single number summarizing the bulk of variable values.

• The expected value is calculated by averaging past realizations, demonstrating future trends.

• The Law of large numbers asserts that averages tend towards a specific number with more trials.

• Borel's law of large numbers is a special case, connecting expected value to probabilities.

• However, the expected value might lack informativeness in certain scenarios, requiring alternative measures like the median.