Euclidean spaces

Dot product is a magnificent tool when it comes to $\mathbb{R}^{n}$. We saw how it flawlessly connected geometric notions with the familiar algebraic manipulations. Could we define a similar operation for an arbitrary vector space $V$?

When we defined dot product, we were using standard basis of $\mathbb{R}^{n}$. So, probably, even more important is the question, could we define this operation without using any specific basis? The answers to both these questions are positive. Here we are going to see, that vector space bundled with this new operation creates a marvelous construction, which is at least as good to describe geometric concepts with algebraic language as $\mathbb{R}^{n}$ with dot product.

Increasing the abstraction level

The dot product was so great because of its variable features and properties. There are three of them which are the most important for us. Let's say, that $V$ is an arbitrary real $n$-dimensional vector space, $\vec{v}$ and $\vec{w}$ are any two vectors from $V$. We are calling an operation taking $\vec{v}$ and $\vec{w}$ and giving some real number as an output, that is bilinear, symmetric, and positively defined as an inner product. To denote the inner product, we are using this brand-new angle-bracket notation $\left\langle\vec{v},\vec{w}\right\rangle$. Here is what bilinear, symmetric, and positively defined means

Bilinearity:$$\left\langle\alpha_{1}\vec{a}_{1} + \alpha_{2}\vec{a}_{2},\beta_{1}\vec{b}_{1} + \beta_{2}\vec{b}_{2}\right\rangle = \alpha_{1}\beta_{1} \left\langle \vec{a}_{1},\vec{b}_{1}\right\rangle + \alpha_{1}\beta_{2} \left\langle \vec{a}_{1},\vec{b}_{2}\right\rangle + \alpha_{2}\beta_{1} \left\langle \vec{a}_{2},\vec{b}_{1}\right\rangle + \alpha_{2}\beta_{2} \left\langle \vec{a}_{2},\vec{b}_{2}\right\rangle$$Symmetry:

$$\left\langle \vec{a},\vec{b}\right\rangle = \left\langle \vec{b},\vec{a}\right\rangle$$

Positive definition:

$$\left\langle\vec{a},\vec{a}\right\rangle \ge 0$$Note that, as was mentioned, these three properties hold for the dot product, therefore dot product is an example of inner product.

Inner products of basis vectors are the key

In the case of the dot product, the products of the basis vectors with themselves were $1$, and with the others $0$, in the case of the inner product, we have much more freedom in choosing these values, but note that the product of the basis vectors cannot be completely arbitrary numbers. For example, let $V$ be a two-dimensional space with an arbitrary basis $\{\vec{e}_{1},\vec{e}_{1}\}$. Suppose that the inner products of basis vectors are given in the following way

$$\left\langle\vec{e}_{1},\vec{e}_{1}\right\rangle = a, \qquad \left\langle\vec{e}_{1},\vec{e}_{2}\right\rangle = a, \qquad \left\langle\vec{e}_{2},\vec{e}_{2}\right\rangle = b$$Here $a$ and $b$ are some real numbers. Consider an inner product of a couple of arbitrary vectors $\vec{v} = v_{1}\vec{e}_{1} + v_{2}\vec{e}_{2}$ and $\vec{w} = w_{1}\vec{e}_{1} + w_{2}\vec{e}_{2}$. By bilinearity

$$\left\langle\vec{v},\vec{w}\right\rangle = \left\langle v_{1}\vec{e}_{1} + v_{2}\vec{e}_{2}, w_{1}\vec{e}_{1} + w_{2}\vec{e}_{2}\right\rangle = v_{1}w_{1} \left\langle\vec{e}_{1},\vec{e}_{1}\right\rangle + v_{1}w_{2} \left\langle\vec{e}_{1},\vec{e}_{2}\right\rangle + v_{2}w_{1} \left\langle\vec{e}_{2},\vec{e}_{1}\right\rangle + v_{2}w_{2} \left\langle\vec{e}_{2},\vec{e}_{2}\right\rangle$$By symmetry

$$\left\langle\vec{v},\vec{w}\right\rangle = v_{1}w_{1} \left\langle\vec{e}_{1},\vec{e}_{1}\right\rangle + (v_{1}w_{2} + v_{2}w_{1}) \left\langle\vec{e}_{1},\vec{e}_{2}\right\rangle + v_{2}w_{2} \left\langle\vec{e}_{2},\vec{e}_{2}\right\rangle$$Substitute initial values

$$\left\langle\vec{v},\vec{w}\right\rangle = (v_{1}w_{1} + v_{1}w_{2} + v_{2}w_{1})a + v_{2}w_{2}b$$

Now if $\vec{v} = \vec{w}$

$$\left\langle\vec{v},\vec{w}\right\rangle = \left\langle\vec{v},\vec{v}\right\rangle = av_{1}^{2} + av_{1}v_{2} + bv_{2}^{2}$$The inner product is positively defined, meaning $\left\langle\vec{v},\vec{v}\right\rangle \ge 0$, therefore$$\left\langle\vec{v},\vec{v}\right\rangle = av_{1}^{2} + 2av_{1}v_{2} + av_{2}^{2} + bv_{2}^{2} - av_{2}^{2} = a(v_{1} + v_{2})^{2} + (b - a)v_{2}^{2}\ge 0$$For this condition to hold, we need $a \ge 0$ and $b\ge a$ ($a$ and $b$ are not equal to $0$ at the same time).

Therefore, knowing the value of inner products of basis vectors, we can always find the inner product of two arbitrary vectors knowing their coordinates in this basis. The only condition is that these values are in accordance with the definition.

Euclidean space

The main intuition beside the dot product was its connection with the geometry of $n$-dimensional space. This intuition could be generalized with the help of inner product.

Let’s say $V$ is a vector space and $\lang\cdot,\cdot\rang$ some inner product. A pair $E = (V,\lang\cdot,\cdot\rang)$ (a vector space with inner product) is called a Euclidean space. It's no coincidence that the name of the ancient geometer appeared here! The reason is, in Euclidean spaces, every geometric statement that we are familiar with holds true! Promising, isn't it?

However, for all this to work, first, we need to learn how to measure things in Euclidean spaces. But having a dot product as an example, it won’t be difficult.

The length of a vector $\vec{v}$ in a Euclidean space is just a square root of its ‘inner square’:

$$\|\vec{v}\| = \sqrt{\left\lang\vec{v},\vec{v}\right\rang}$$(By the way, this is the main reason we wanted the inner product to be positively defined in the first place.)

A cosine of an angle $\angle\left(\vec{v},\vec{w}\right)$ between vectors $\vec{v}$ and $\vec{w}$ is given by the following quotient

$$\cos(\angle\left(\vec{v},\vec{w}\right)) = \frac{\left\lang\vec{v},\vec{w}\right\rang}{\|\vec{v}\|\cdot\|\vec{w}\|}$$The angle $\varphi$ itself is going to be given by the inverse cosine

$$\angle\left(\vec{v},\vec{w}\right) = \arccos\left(\frac{\left\lang\vec{v},\vec{w}\right\rang}{\|\vec{v}\|\cdot\|\vec{w}\|}\right)$$The two vectors $\vec{v}$ and $\vec{w}$ in a Euclidean space are called orthogonal if their inner product is equal to zero (denoted $\vec{v}\perp\vec{w}$).

Geometry without visualization

Let $V$ be a two-dimensional vector space with a basis $\{\vec{e}_{1},\vec{e}_{2}\}$. The following relations $$\left\lang\vec{e}_{1}\,\vec{e}_{1}\right\rang = 2\\ \left\lang\vec{e}_{1}\,\vec{e}_{2}\right\rang = -1\\ \left\lang\vec{e}_{2}\,\vec{e}_{2}\right\rang = 1$$define an inner product $\lang\cdot,\cdot\rang$ (it’s not hard to check that this is indeed an inner product, using the above-mentioned argument). Let’s find the angle between vectors $\vec{v} = \vec{e}_{1} + \vec{e}_{2}$ and $\vec{w} = \vec{e}_{1} + 2\vec{e}_{2}$ in the Euclidean space $E = (V,\lang\cdot,\cdot\rang)$.

Before solving this problem, we’d really like to empathize the idea of the solution, namely its utterly algebraic nature. There is no specific geometric picture behind $V$ and $\{\vec{e}_{1},\vec{e}_{2}\}$ is a completely arbitrary basis, its vectors are not perpendicular and $\vec{e}_{1}$ has length $\sqrt{2}$ in $E$. This example is much more abstract than the ones we considered in the dot product discussion.

One could ask why we need such a hypothetical construction. The answer is in universality! This algebraic approach allows to introduce a geometric sense in any problem, where vector spaces appear. For example, in vector analysis, statistics, or model building. This means that Euclidean spaces are tools, and the actual interpretation comes from a particular problem.

Now let’s go through calculations. The lengths of $\vec{v}$ and $\vec{w}$ are$$\|\vec{v}\| = \sqrt{\lang\vec{v},\vec{v}\rang} = \sqrt{\lang(\vec{e}_{1} + \vec{e}_{2}),(\vec{e}_{1}+\vec{e}_{2})\rang} = \sqrt{\lang\vec{e}_{1},\vec{e}_{1}\rang + 2\lang\vec{e}_{1},\vec{e}_{2}\rang + \lang\vec{e}_{2},\vec{e}_{2}\rang} = \sqrt{2 + 2\cdot(-1) + 1} = 1\\[2mm] \|\vec{w}\| = \sqrt{\lang\vec{w},\vec{w}\rang} = \sqrt{\lang(\vec{e}_{1} + 2\vec{e}_{2}),(\vec{e}_{1} + 2\vec{e}_{2})\rang} = \sqrt{\lang\vec{e}_{1},\vec{e}_{1}\rang + 4\lang\vec{e}_{1},\vec{e}_{2}\rang + 4\lang\vec{e}_{2},\vec{e}_{2}\rang} = \sqrt{2 + 4\cdot(-1) + 4} = \sqrt{2}$$The inner product of $\vec{v}$ and $\vec{w}$ is

$$\lang\vec{v},\vec{w}\rang = \lang(\vec{e}_{1} + \vec{e}_{2}),(\vec{e}_{1} + 2\vec{e}_{2})\rang = \lang\vec{e}_{1},\vec{e}_{1}\rang + 3\lang\vec{e}_{1},\vec{e}_{2}\rang + 2\lang\vec{e}_{2},\vec{e}_{2}\rang = 2 + 3\cdot(-1) + 2\cdot 1 = 1$$Therefore$$\cos\left(\angle(\vec{v},\vec{w})\right) = \frac{\lang\vec{v},\vec{w}\rang}{\|\vec{v}\|\cdot\|\vec{w}\|} = \frac{1}{\sqrt{2}} \Longrightarrow \angle(\vec{v},\vec{w}) = 45^{\circ}$$

Which functions define inner product?

Having a $n$-dimensional vector space $V$ with a basis $\{\vec{e}_{1},\vec{e}_{2},\dots,\vec{e}_{n}\}$ one can always think about its element $\vec{v} = x_{1} \vec{e}_{1} + x_{2} \vec{e}_{2} + \dots + x_{n} \vec{e}_{n}$ as a tuple $\begin{pmatrix}x_{1}&x_{2}&\dots& x_{n}\end{pmatrix}^{\mathsf{T}}$from $\mathbb{R}^{n}$. This notation is more natural for a whole variety of specific problems. In this situation, it’s usually more convenient to define the inner product as a function of coordinates.

It could sound a little bit unclear, but let’s look at a two-dimensional example. An inner product in $\mathbb{R}^{2}$ could be given like this

$$\left\lang\vec{x},\vec{y}\right\rang = \left\lang\begin{pmatrix}x_{1}\\x_{2}\end{pmatrix},\begin{pmatrix}y_{1}\\y_{2}\end{pmatrix}\right\rang = x_{1}y_{1} - x_{1}y_{2} - x_{2}y_{1} + 2x_{2}y_{2}$$It’s straightforward, that this function of coordinates is symmetric and bilinear with respect to vectors $\vec{x}$ and $\vec{y}$. Let’s check, that it’s positively defined, though. To prove this, let’s consider the product of $\vec{x}$ with itself$$\lang\vec{x},\vec{x}\rang = \left\lang\begin{pmatrix}x_{1}\\x_{2}\end{pmatrix},\begin{pmatrix}x_{1}\\x_{2}\end{pmatrix}\right\rang = x_{1}^{2} - 2x_{1}x_{2} + 2x_{2}^{2}$$let’s mess with this expression a little bit$$x_{1}^{2} - 2x_{1}x_{2} + 2x_{2}^{2} = \left(x_{1}^{2} - 2x_{1}x_{2} + x_{2}^{2}\right) +x_{2}^{2} = (x_{1} - x_{2})^{2} +x^{2}_{2}\ge 0$$This expression is greater than zero, as it is a sum of two whole squares, and it’s equal to zero if and only if $x_{1} = x_{2} = 0$, therefore, the “positively defined” part of an inner product definition is checked.

Now, to find an inner product of vectors, just plug in the coordinates into the expression:$$\left\lang\begin{pmatrix}-7\\3\end{pmatrix},\begin{pmatrix}13\\10\end{pmatrix}\right\rang = -7\cdot 13 - (-7)\cdot10 - 3\cdot 13 + 3\cdot 10 = 0$$Note that we’ve just shown that $\begin{pmatrix}-7&3\end{pmatrix}^{\mathsf{T}}$ and $\begin{pmatrix}13&10\end{pmatrix}^{\mathsf{T}}$ are orthogonal in $V$with respect to this inner product.

Conclusion

• An inner product is a bilinear, symmetric, positively defined operation, which takes two vectors and gives a number.
• The dot product is the simplest example of an inner product.
• To define an inner product, one does not need any specific basis, however, when it comes to calculation, it could be useful to work with inner products of some basis vectors.
• Together with a chosen inner product, a vector space is a Euclidean space in which one can calculate lengths and angles.
• The formulas for the length of a vector and an angle between vectors are exactly the same as in the case of the dot product$$\|\vec{v}\| = \sqrt{\lang\vec{v},\vec{v}\rang};\quad\angle(\vec{v},\vec{w}) = \arccos\left(\frac{\lang\vec{v},\vec{w}\rang}{\|\vec{v}\|\cdot\|\vec{w}\|}\right).$$
• One can use a suitable function of coordinates of vectors to define an inner product.