Basis of a vector space

Until now, we have discussed components of a vector only in the context of coordinates of a point in space. But the set of points in space is only a special case of an abstract vector space. Recall that a vector space is a way to combine mathematical objects of the same type in order to perform certain operations on them. It is possible to generalize the concept of coordinates to more exotic vector spaces such as spaces of functions or spaces of matrices through the notion of the basis. The basis of a vector space is a minimal subset of vectors, through which all other vectors can be represented. This means that you can always use the basis as a coordinate system. The choice of basis allows you to look at the problem from different perspectives.

Basis and coordinate systems

Let's start with vectors on a plane. We have already established that vectors on a plane could be perceived both as arrows and as lists of two numbers. Consider vector $\vec{a} = [3, 4]$.

Let's name two vectors $\vec{i} = [1, 0]$ and $\vec{j} = [0, 1]$. Assume that we are given those two vectors and no coordinate axes. Can we define the vector $\vec{a}$ through vectors $\vec{i}$ and $\vec{j}$?

In vector spaces, we can add vectors up and multiply them by numbers. Let's try to obtain vector $\vec{a}$ by using those operations.

Hooray! We've managed to represent vector $\vec{a}$ in the form of a linear combination of vectors $\vec{i}$ and $\vec{j}$

$$\vec{a} = 3 \vec{i} + 4\vec{j}.$$

The coefficients in that linear combination look familiar, don't they? They are the coordinates of the vector $\vec{a}$ in the original system of coordinates!

So, now we have another interpretation of the components of the two-dimensional vector. They are just scalars used in the linear combination of vectors $\vec{i}$ and $\vec{j}$, which represents that vector. Vectors $\vec{i}$ and $\vec{j}$ are called basis vectors. Any two-dimensional vector can be represented as a linear combination of vectors $\vec{i}$ and $\vec{j}$.

The charm of that interpretation is that vector space's basis is not unique. The vectors $\vec{i}$ and $\vec{j}$ are no different from any other pair of non-collinear vectors. For instance, let's use vectors $\vec{s} = [1, 1]$ and $\vec{t} = [1, 2]$ (coordinates are given in the basis $\vec{i}$, $\vec{j}$ as a reference point).

The vectors $\vec{s}$ and $\vec{t}$ define new coordinate axes.

Let's find the coordinates of $\vec{a}$ in a new basis solving the following equation with respect to $a_1$, $a_2$.

$$\vec{a} = 3 \cdot \begin{bmatrix}1 \\ 0\end{bmatrix} + 4 \cdot \begin{bmatrix}0 \\ 1\end{bmatrix} = a_1 \cdot \begin{bmatrix}1 \\ 1\end{bmatrix} + a_2 \cdot \begin{bmatrix}1 \\ 2\end{bmatrix}.$$

It has a unique solution if vectors $\begin{bmatrix}1 \\ 1\end{bmatrix}$ and $\begin{bmatrix}1 \\ 2\end{bmatrix}$are linearly independent. It is convenient to rewrite that system in the following form:

$$\begin{cases} a_1 + a_2 = 3 \\ a_1 + 2 \cdot a_2 = 4 \end{cases}$$

To solve it, we should subtract the first equation from the second one

$$\begin{cases} a_1 + a_2 = 3 \\ a_1 + 2 \cdot a_2 = 4 \end{cases} \Leftrightarrow \begin{cases} a_1 + a_2 = 3 \\ a_1+2\cdot a_2 - a_1-a_2 = 1 \end{cases} \Leftrightarrow \begin{cases} a_1 = 3-a_2 \\ a_2 = 1 \end{cases} \Leftrightarrow \begin{cases} a_1 = 2 \\ a_2 = 1 \end{cases}$$

We've obtained the coordinates of $\vec{a}$ in the new basis.

Notice that the vector $\vec{a}$ is still the same vector on the plane. All that has changed is its coordinates. And this is a really important concept. By changing the basis, you can change the perspective from which you look at the problem. For example, you can change the basis in $n$-dimensional space to extract meaningful features from the numerical data. In one basis, it might be hard to find patterns that could be clearly seen in the other. And even more, vector spaces don't boil down to the tuples of $n$ numbers. They find their application in other forms in many different fields of engineering and science.

Continuing our investigation in two-dimensional space, we can come to the conclusion that any two non-collinear vectors (i.e linearly independent) on the plane can form the basis. But if two vectors are collinear, then any linear combination of them is collinear to the original pair. That implies that there exists at least a vector on the plane that can't be obtained as a linear combination of the two collinear vectors. So, collinear vectors can't form the basis. That takes us to the definition of the basis in an abstract vector space.

Formal definition

For convenience, let's first introduce the definition of a span.

Span of a set of vectors $B$ is the set of all possible linear combinations of vectors from the set $B$. If there is only one vector in the set $B = \{\vec{b}_1\}$, then the span of $B$ will contain all vectors, that are collinear to the vector $b_1$. If there are two vectors in $B = \{\vec{b}_1, \vec{b}_2\}$, then the span of $B$ will contain all possible linear combination $c_1 \vec{b}_1 + c_2 \vec{b}_2$, $c_1,c_2\in\mathbb{R}$.

The basis of a vector space $V$ is a set of vectors $B \subset V$ (finite or possibly infinite) that satisfies two properties:

1. Vectors in the set $B$ are linearly independent.

2. Span of $B$ is equal to $V$.

Vector space dimension

Until now, we used the number of components of a tuple of numbers interchangeably with the dimension of the vector space. The basis allows generalizing the concept of dimension to vector spaces that do not consist of lists of numbers.

Let's return to the example of two-dimensional space. Can there be a basis that consists of three or more vectors? No, as any three vectors on the plane (or any three lists of two numbers) are linearly dependent. It is discussed in the topic dedicated to linear independence. Can there be one vector in the basis of the plane? No again. Through one vector you can represent only vectors that are collinear to it. So, we arrive at the conclusion that any basis on the plane consists of exactly two vectors.

That conclusion can be generalized. A vector space $V$ can have many different bases, but it can be proven that all bases have the same number of elements. The number of elements in the basis is called the dimension of a vector space: if a vector space $V$ has a basis with $n$ elements, then the dimension of $V$ is equal to $n$. It is denoted by $dim V=n$.

How a change of basis can be useful?

Let's look at specific examples when it is convenient to use different bases.

Consider a wheel of a car. Acceleration of its parts has two components: tangential and normal. The vector of tangential acceleration points along the direction of motion, while the vector of normal acceleration points perpendicular to the tangent vector, towards the center of curvature.

For further analysis, it might be convenient to choose the direction of the axis $x$ to be parallel to the tangential acceleration and the $y$ axis along the normal acceleration.

That idea of the choice of perspective could be generalized to other vector spaces. For example, in three-dimensional space, if your problem involves a parallelepiped, it might be convenient to bind the coordinate axes to the edges of the parallelepiped.

Conclusion

Let's sum up the knowledge that we have gained about the basis of a vector space:

• The basis of a vector space is a set of linearly independent vectors which span the given vector space.
• The idea of a basis is closely related to the idea of the choice of perspective you are looking at the problem with. A wise choice of basis can simplify the problem significantly.
• All possible bases in the same vector space have an equal number of elements. The number of elements in a basis is generally called the dimension of the vector space.