Expected value of a continuous random variable

When analyzing data in numerical statistics, machine learning, and other fields, you may need to obtain the average of a random variable. In particular, it is often necessary to find out the average value of a continuous random variable. It happens because in the real world, random processes are more frequently described by continuous random variables than by discrete ones. To calculate a value close to the average of a continuous random variable, you will use such a characteristic as the mathematical expectation of a continuous random variable.

Defining the expected value

You can calculate the expected value of a continuous random variable by analogy with the expected value of a discrete random variable. You might remember that to obtain the expected value of a discrete random variable that takes values from the set $\left \{ x_{1}, x_{2}, ..., x_{k} \right \}$, you need to use the following formula:

$\mathrm{E}[X]=\sum_{j=1}^{k} x_j \cdot P(X=x_j)$That is, you multiply each value of a random variable by its probability and sum up all such products. Then, let $f(x)$ be the probability density function (PDF) for the continuous random variable $X$ that takes an infinite number of possible values. You can't take the probability at the point $P(X=x_j)$ for a continuous random variable, since it is equal to $0$ by properties. If the set of solutions is infinite, then it is almost impossible to get a probability equal to a specific value. In this case, you have to take the probability in some neighborhood of the point, for example, on the interval $[x_j, x_j+ \Delta x]$. It turns out that you need to calculate something like the following expression:

$\mathrm{E}[X] \approx \sum_{j=1}^{k} x_j \cdot P(x_j \leq X < x_j + \Delta x).$

In turn, the probability on the interval of length $\Delta x$ is the area under the graph of the PDF $f(x)$, as below:

Let's approximate the probability of a point in the neighborhood: $P(x_j \leq X < x_j + \Delta x) \approx f(x_j) \Delta x$, $j = 1, ..., k.$

You need to consider all the points where a continuous random variable is defined, as in the discrete case. Thus, let's try to reduce the length of the interval $\Delta x$ and increase the number of points to consider as many of them as possible. Visually, it will look like this:

Now you understand that if you let $\Delta x$ tend to zero, the sum $\sum_{j=1}^{k} f(x_j)\Delta x$ will be equal to the integral $\int_{- \infin}^{+ \infin} f(x) dx$ in the limit. This integral is equal to the area under the $f(x)$ graph, which you see above. This means that you need to integrate, not sum. Now let's formulate the definition of the mathematical expectation.

Definition. If $X$is a continuous random variable with the PDF $f(x)$, then the expected value of $X$ is defined as:

$\mathrm{E}[X] = \int_{- \infin}^{+ \infin} x \cdot f(x) dx.$In other words, comparing with the discrete case formula, you replaced the probability mass function with the probability density function and the sum with the integral in the formula.

All properties for the expected value of a discrete random variable are preserved for the expected value of a continuous random variable.

Examples

Let's consider a simple example. Imagine that you have a bicycle, and you are careful and afraid to damage the tire of the wheel. You are interested in where damage can happen, in what place the wheel can be pierced. First, you need to enter some kind of metric, a way to determine where the exact location of the damage is. The wheel is round and visually the same everywhere. You can take the valve stem as a starting point. Then you introduce a random variable equal to the angle formed by the damage and the valve stem of the wheel, see below:

What value can the random variable $X$ take? The wheel is round, which means any angle from 0 to 360.

It can be any angle, not only an integer, for example, 7.8 degrees. Note that there isn't any finite set or countable set of possible angles, which means that your random variable is really continuous. Now, how will you define the density function? It is simple: a uniform distribution determines the random variable. Recall that for the interval $[a,b]$ the density function $f(x)$ is defined as follows:

$f(x) = \begin{cases} \frac{1}{b-a} &, a \leq x \leq b \\ 0 & ,otherwise. \end{cases}$

What is your interval? It is an interval from $0$ to $360$. Thus, f(x) is defined as:

$f(x) = \begin{cases} \frac{1}{360} &, 0 \leq x \leq 360 \\ 0 & ,otherwise. \end{cases}$

Now let's calculate the expected value. Note that the function $f(x)$ is equal to zero on the intervals (−∞,0) and (360,+∞), so the corresponding integral is equal to zero, and it makes sense to take the integral only on the interval [0,360].

$E[X]$ is calculated like this:

$\mathrm{E}[X] = \int_{- \infin}^{0} 0 dx+ \int_{0}^{360} x \cdot \frac{1}{360} dx + \int_{360}^{+ \infin} 0 dx = \frac{1}{360} \int_{0}^{360} x dx.$Next, you should calculate the resulting integral:

$\mathrm{E}[X] = \frac{1}{360} \int_{0}^{360} x dx =\frac{1}{360} \cdot \frac{x^2}{2} \bigg|_{0}^{360} =\frac{1}{360} \Bigl[ \frac{360^2}{2} -0 \Bigr ] =180 .$You get the expected value equal to $180$. Notice that this is really the average value on the interval $[0, 360]$.

What do you think the expected value of the uniform distribution will be in the general case? The expected value will be equal to $(a+b)/2$. See how this expression appeared:

$\mathrm{E}[X] = \frac{1}{(b-a)} \int_{a}^{b} x dx =\frac{1}{(b-a)} \cdot \frac{x^2}{2} \bigg|_{a}^{b} =\frac{1}{(b-a)} \Bigl[ \frac{b^2 - a^2}{2} \Bigr ] =\frac{1}{(b-a)} \Bigl[ \frac{(b-a)(b+a)}{2} \Bigr ] = \frac{a+b}{2} .$With such a distribution, you will always get the expected value equal to the average value. But you are lucky: this case is not usual, more often the expected value will be close to the average value but not equal to it.

Properties of the expected value of a continuous random variable

The properties of the expected value of a continuous random variable are essentially the same as in the case of a discrete random variable. Below is a list of them, but without too many details, since these properties should be familiar to you already.

• If a random variable takes only one value, then its expected value is equal to that very value: $\mathrm{E}[C]=C$.

• The expected value is linear, regarding multiplication by a constant: $\mathrm{E}[aX]=a\mathrm{E}[X]$.

• The expected value is linear, regarding addition: $\mathrm{E}[X+Y]=\mathrm{E}[X]+ \mathrm{E}[Y]$ (provided both parts are well-defined).

• If a random variable is non-negative, then its expected value is also non-negative: $X\ge0\Rightarrow \mathrm{E}[X]\ge0$.

• The expected value is monotonous: if $X\ge Y$, then $\mathrm{E}[X]\ge\mathrm{E}[Y]$.

• If the random variables $X$ and $Y$ are independent, then $\mathrm{E}[XY]=\mathrm{E}[X]\cdot \mathrm{E}[Y]$ (note that independence is crucial here).

Conclusion

The expected value defines a value close to the average of a random variable. You can calculate the expected value of a continuous random variable by analogy with a discrete random variable. You need to calculate the integral of the product of the probability density function and the variable $x$ on the segment $(-\infin, +\infin)$. In addition, all the properties of the expected value are preserved as in the discrete case.