Projections

In various ways, you are likely to be familiar with the concept of projection. You could have encountered it in geography class or through work connected with graphics, or maybe in therapy sessions. Or you could have come across it in philosophy readings or sci-fi movies. Usually, this term is associated with transferring certain properties of one object to another or carrying a certain image between two different spaces. Interestingly, this intuitive idea also holds in mathematics, where projections are used in different fields. In this discussion, you will focus on projections of vectors, which are related to the concept of orthogonality and offer a compelling geometric interpretation.

Some may perceive the outline of projections as somewhat simplistic or not immediately applicable. However, it's crucial to recognize that vector projections are an immensely valuable tool for fundamental constructions in linear algebra. Their significance should not be underestimated.

A helpful visual intuition

Imagine a light source sending a broad, parallel beam of light perpendicular to a straight line. In this scenario, a projection of a vector onto this line can be visualized as its shadow (see the picture).

Although this representation is not the most precise definition of a projection, it does provide some valuable insights. For instance, when a vector lies on the line, its projection coincides with the vector itself. This represents the maximum length a projection can achieve, as in any other case, a vector will have a greater magnitude than its projection.

A geometric definition

Let’s now give a precise geometric definition of a projection, which does not use these physical entities. Up until now, we were discussing a projection of a vector onto a line, however, it turns out to be more natural to define a projection of a vector onto another vector.

Consider two vectors $\vec{a}$ and $\vec{e}$:

Then draw a line on which the vector $\vec{e}$ lies:

After that, draw a perpendicular from the and of vector $\vec{a}$ to this line:

A projection of $\vec{a}$ onto $\vec{e}$ is a vector $\mathrm{proj}_{\vec{e}}(\vec{a})$ directed from the beginning of the vector $\vec{a}$ to the intersection of the line on which $\vec{e}$ lies and the constructed perpendicular (the red vector in the following picture)

The gray arrows show how this definition is connected with the ‘shadow’ construction.

It is understandable, that this particular geometric concept could look quite contrived, but trust us, it is not. Knowing projections of a vector onto some given set of vectors could be very useful, especially if you choose these last to be a basis.

An abstract point of view

The essence of linear algebra lies in the ability to describe a wide range of geometric objects through the use of linear equations and algebraic concepts. This approach allows us to work with complex geometric images without necessarily visualizing them. Additionally, integrating linear algebra into programming problems becomes more manageable as you work with variables, which aligns much more naturally with machine computing rather than pictures. Therefore, in this context, you will define a projection using the notion of an inner product for abstract Euclidean spaces.

Consider a Euclidean space $V$ with inner product $\lang\cdot\mid\cdot\rang$. Summarizing everything that was said about projections up until this place, we can say that a projection of a vector $\vec{a}\in V$ onto a vector $\vec{e}\in V$ is a vector of length $\|\vec{a}\|\cdot|\cos(\alpha)|$, where $\alpha$ is an angle between $\vec{a}$ and $\vec{e}$. Notice that it means that the projection of a vector can not be longer than vector itself. The projection is lying on the same line with $\vec{e}$ and it is directed in the same way as $\vec{e}$ if $0^{\circ}\le\alpha < 90^{\circ}$ and in the opposite way if $90^{\circ} < \alpha \le 180^{\circ}$. Here are the pictures describing these two situations:

In the left one $\alpha>90^{\circ}$ and in the right one $\alpha<90^{\circ}$. The vector $\vec{n}$ is a unit (length one) vector codirected with $v$, therefore

$$\vec{n} = \frac{1}{\|\vec{e}\|} \cdot \vec{e}$$

All in all, you can write the following, as

$$\mathrm{proj}_{\vec{e}}(\vec{a}) = \|\vec{a}\|\cdot\cos(\alpha)\cdot\vec{n}$$Now let’s rewrite it in terms of the inner product:

$$\cos(\alpha) = \frac{\lang\vec{a},\vec{e}\rang}{\|\vec{a}\|\cdot\|\vec{e}\|}\\ \mathrm{proj}_{\vec{e}}(\vec{a}) =\cancel{ \|\vec{a}\|}\cdot \frac{\lang\vec{a},\vec{e}\rang}{\cancel{\|\vec{a}\|}\cdot\|\vec{e}\|}\cdot\frac{1}{\|\vec{e}\|}\cdot\vec{e} = \frac{\lang\vec{a},\vec{e}\rang}{\|\vec{e}\|^{2}}\cdot\vec{e}$$

Or finally

$$\boxed{\mathrm{proj}_{\vec{e}}(\vec{a}) = \frac{\lang\vec{a},\vec{e}\rang}{\lang\vec{e},\vec{e}\rang}\cdot\vec{e}}$$This is the most general definition of a projection of $\vec{a}$ onto $\vec{e}$ in a generic Euclidean space.

Numeric example

Consider a plane $\mathbb{R}^{2}$ as a Euclidean space with a standard dot product as an inner product. Let’s find a projection of $\vec{a} = \begin{pmatrix}4&3\end{pmatrix}^{\mathsf{T}}$ onto $\vec{e} = \begin{pmatrix}-1&-2\end{pmatrix}^{\mathsf{T}}$:

$$\mathrm{proj}_{\vec{e}}(\vec{a}) = \frac{4\cdot(-1) + 3\cdot(-2)}{(-1)^{2} + (-2)^{2}} \cdot \begin{pmatrix}-1\\-2\end{pmatrix}= \begin{pmatrix}2\\4\end{pmatrix}$$This calculation is illustrated higher; a projection vector is given in red, you can check using the squared greed, that it is indeed $\begin{pmatrix}2&4\end{pmatrix}^{\mathsf{T}}$.

Conclusion

These are some facts for you to take away

• A projection of a vector $\vec{a}$ onto a vector $\vec{e}$ in a Euclidean space with an inner product $\lang\cdot\mid\cdot\rang$ is a vector $$\mathrm{proj}_{\vec{e}}(\vec{a}) = \frac{\lang\vec{a},\vec{e}\rang}{\lang\vec{e},\vec{e}\rang}\cdot\vec{e}$$
• If $\vec{a}$ is colinear to $\vec{e}$ then $\mathrm{proj}_{\vec{e}}(\vec{a}) = \vec{a}$ ;
• If $\vec{a}$ is orthogonal to $\vec{e}$ then $\mathrm{proj}_{\vec{e}}(\vec{a}) =\vec{0}$;
• The geometrical purpose of $\mathrm{proj}_{\vec{e}}(\vec{a})$ is, in some sense, to show the degree of alignment of vector $\vec{a}$ with $\vec{e}$;
• Or it could be thought of as a ‘shadow’ of $\vec{a}$ on line vector $\vec{e}$ lies at.