Probabilities as frequencies

Now that we know how to conduct experiments, it is time we understand, how can we get the best out of them. This will be related to the output of experiments, which will give birth to the concept of probability.

Events

We would call events statements about the outcome of the experiment, the truth of which depends only on the value of the outcome. For example, if our experiment is "pick a random watermelon from a shelf at the supermarket", then statements "watermelon has 24 stripes" and "watermelon weighs 10 kg" are events, while "watermelon weighs 10 kg and the weather is good" is not.

If for a given outcome, the event is true, then we can say that the event has occurred. For example, if the die rolled 6, then the events "an even number was rolled" and "a number greater than three was rolled" occurred. Meanwhile, the events "an odd number was rolled" and "a number less than four was rolled" did not.

In Python, events are simply functions that take the outcome as an argument and return bool. In the example below, we conduct one trial of an experiment flip_until_second_head from the previous section. Then we will check if events "flipped the coin more than four times", "the first coin flip resulted in heads", and "two tails were flipped in a row" occur.

def more_than_four_flips(outcome):
    return len(outcome) > 4

def first_is_head(outcome):
    return outcome[0] == 'H'

def two_tails_in_a_row(outcome):
    for current, next in zip(outcome, outcome[1:]):
        if current == 'T' and next == 'T':
            return True
    return False

outcome = flip_until_second_head()

print('Outcome:', outcome)
print('Flipped the coin more than four times:', more_than_four_flips(outcome))
print('The first coin flip resulted in heads:', first_is_head(outcome))
print('Two tails were flipped in a row:', two_tails_in_a_row(outcome))

Outcome: ['T', 'H', 'T', 'T', 'H']
Flipped the coin more than four times: True
The first coin flip resulted in heads: False
Two tails were flipped in a row: True

Events describe in yes/no format the properties of the outcome of a specific experiment. However, in real life, we are usually interested in the outcomes of phenomena that are yet to happen. When deciding whether to buy a lottery ticket, we do not know what the outcome will be, but we are interested in how likely we are to win: if 1000 people buy a ticket, how many of them will be lucky? Such questions lead us to the idea that we need to look at events in the context of not just one, but multiple trials. Thus, we come to the concept of the probability of an event.

Probabilities

With a sequence of trials, we can estimate how likely the event of interest will occur in the next trial by simply looking at the fraction of trials in which the event occurred. For example, let's estimate how likely a fair coin will land heads at least 20 times in 30 tosses by conducting 1000 trials:

def flip_coin_30_times():
    return random.choices(['H', 'T'], k=30)

def at_least_20_heads(outcome):
    return sum([flip == 'H' for flip in outcome]) >= 20

NUMBER_OF_TRIALS = 1000
occurrences = 0
for i in range(NUMBER_OF_TRIALS):
    if at_least_20_heads(flip_coin_30_times()):
        occurrences += 1

print(occurrences / NUMBER_OF_TRIALS)

0.043

The chance is about 1 in 20. Not very much! However, could it be that this fraction is highly dependent on what outcomes our trials had? If it can be 1 out of 20 as well as 15 out of 20, then our estimate is not very useful. The best way to check this is to repeat the whole procedure several times:

NUMBER_OF_TRIALS = 1000
for i in range(10):
    occurrences = 0
    for j in range(NUMBER_OF_TRIALS):
        if at_least_20_heads(flip_coin_30_times()):
            occurrences += 1
    print(occurrences / NUMBER_OF_TRIALS, end=' ')

0.039 0.052 0.058 0.061 0.055 0.042 0.04 0.044 0.051 0.042

There is indeed a variation in the results, but all the values are quite close to each other. Let's try to increase the number of trials by a hundred times.

# imagine doing it yourself and not in python
NUMBER_OF_TRIALS = 100000
for i in range(10):
    occurrences = 0
    for j in range(NUMBER_OF_TRIALS):
        if at_least_20_heads(flip_coin_30_times()):
            occurrences += 1
    print(occurrences / NUMBER_OF_TRIALS, end=' ')

0.04935 0.04981 0.04911 0.048 0.04919 0.05057 0.0494 0.04827 0.04954 0.04883 

As we can see, the more numbers we generate, the closer the fractions are to each other.

The number that all fractions converge to is called the probability of the event $A$ and is denoted as $\mathbb{P}(A)$. In our case, the probability of the event "coin lands heads at least 20 times in 30 tosses" is close to 0.05.

More on probability

Since the probability is the "fraction" of trials in which the event occurs, it is necessarily between 0 and 1. If the probability is close to 1, this means that the event occurs in almost all trials, so we can expect it to occur at the next trial as well. Conversely, if the probability of an event is close to 0, then we expect that it will not occur at the next trial.

The big advantage of probabilities is that they allow us to assess uncertainty about events not only qualitatively, but also quantitatively. If it is known that the probability of getting heads is 0.5, this means that if you toss a coin several times, about half of the outcomes will be heads. The more tosses we do, the closer the proportion of outcomes with heads will be to half.

Borel's law of large numbers grounds a very simple way of estimating the probability of any event, which may be described by the following code:

def probability(experiment, event, number_of_trials):
    occurrences = 0
    for i in range(number_of_trials):
        if event(experiment()):
            occurrences += 1
    return occurrences / number_of_trials

The more number_of_trials, the more accurate the estimate. This method does not allow you to determine probabilities exactly and often requires a lot of calculations or real data. Nevertheless, it is applicable in cases where it is difficult or not possible to estimate the probability in any other way.

To run your own experiments, you can now use the choice, choices, and randrange methods from the random module. If you are eager to learn more, you can always check out the official documentation.

Conclusion

In conclusion, understanding experiments, events, and probabilities is vital for quantitative analysis.

• Events, expressed as Python functions, enable outcome assessment.

• Borel's Law of Large Numbers simplifies probability estimation through repeated trials, converging to a specific probability.

• Probabilities, ranging from 0 to 1, offer a quantitative measure of uncertainty.

• The provided Python code demonstrates a practical approach for running experiments and estimating probabilities.