A change of basis

If you recall, vector spaces can have many different bases. Different bases are useful for different tasks. For example, it is often possible to represent a matrix in a basis where it is diagonal, which is exceptionally simple. This is essential for operations with matrices which occur not only in linear algebra, but also in its applications like physics and data science. Is it possible to switch between bases ? Yes of course! In this topic, we'll see how we can change the basis we're working with and what effect it will have.

What happens when we change the basis?

Suppose we have a vector space $V$ such that $\dim V = n$. Let's choose two bases: $\mathbf{e} = (e_1, \ldots, e_n)$, the "old" basis and $\mathbf{e'} = (e'_1, \ldots, e'_n)$, the "new" basis. We can decompose each vector of the new basis in the old basis, that is, present it as a linear combination of vectors $e_1, \ldots, e_n$ with said coefficients:

$$e'_1 = c_{11} e_1 + c_{21}e_2 + \ldots + c_{n1}e_n, \\ \ldots \\ e'_n = c_{1n} e_1 + c_{2n}e_2 + \ldots + c_{nn}e_n.$$In the matrix view, this can be rewritten as follows:

$$e'_j = (e_1, \ldots, e_n) \cdot \begin{pmatrix} c_{1j} \\ \vdots \\ c_{nj} \end{pmatrix} \ \ \ \text{for all} \ j = 1, \ldots, n,$$where $\begin{pmatrix} c_{1j} \\ \vdots \\ c_{nj} \end{pmatrix}$ is a column of coefficients. So there is a matrix made up of these columns:

$$C = \begin{pmatrix} c_{11} & \ldots & c_{1n} \\ \vdots& \ddots &\vdots \\ c_{n1} & \ldots & c_{nn} \end{pmatrix}.$$This matrix is called the transition matrix from the basis $\mathbf{e}$ to the basis $\mathbf{e}'$. This is a $n\times n$-matrix and its $j$-ith column consists of the coordinates of the vector $e'_j$ in the old basis $e_1, \ldots, e_n$.

For example, we have a vector space $\mathbb{R}^3$ and $\mathbf{e}= (e_1, e_2, e_3)$ is a standard basis:$$e_1 = (1, 0, 0), \\ e_2 = (0, 1, 0), \\ e_3 = (0, 0, 1).$$Consider a new basis $\mathbf{e' } = (e'_1, e'_2, e'_3)$, where $e'_1 = e_2,$ $e'_2 = e_3,$ $e'_3 = e_1$. What is the transition matrix from $\mathbf{e}$ to $\mathbf{e'}$? We find the coordinates of the vectors $e'_1, e'_2, e'_3$ in the basis $\mathbf{e}$:

$$e'_1 = (0, 1, 0), \\ e'_2 = (0, 0, 1), \\ e'_3 = (1, 0, 0)$$and from the columns of the coordinates of the vectors we make the matrix:

$$\begin{pmatrix} 0 & 0 & 1 \\ 1 & 0 & 0 \\ 0 & 1 & 0 \end{pmatrix}.$$This is the transition matrix from $\mathbf{e}$ to $\mathbf{e'}$.

Properties of a transition matrix

The transition matrix has several important properties:

1) The transition matrix always has a non-zero determinant.

2) Any matrix with a non-zero determinant is a transition matrix from a given basis to some other basis.

3) If $C$ is a transition matrix from the basis $\mathbf{e}$ to the basis $\mathbf{e}'$, then $C^{-1}$ is a transition matrix from the basis $\mathbf{e}'$ to the basis $\mathbf{e}$.

4) If $C$ is a transition matrix from $\mathbf{e}$ to $\mathbf{e}'$ and $C'$ is a transition matrix from $\mathbf{e}'$ to $\mathbf{e}''$, then $C \cdot C'$ is a transition matrix from $\mathbf{e}$ to $\mathbf{e}''$.

How can we find a transition matrix?

Now that we know what transitions matrices are in theory, let's try to find one!

Suppose we know the coordinates of the vectors of both bases $\mathbf{e} = (e_1, \ldots, e_n)$ and $\mathbf{e'} = (e'_1, \ldots, e'_n)$ of the vector space $\mathbb{K}^n$ in the standard basis. We write the coordinates of these vectors of the basis $\mathbf{e}$ as the columns of the matrix $A$, and the coordinates of the vectors of the basis $\mathbf{e'} \ -$ as the columns of the matrix $B$ next to $A$. Then we get a $n \times 2n$-matrix $(A|B)$ and bring it to the row echelon form. Since the determinant of the matrix $A$ is non-zero, we get the identity matrix $E$ in its place:

$$(A|B) \leadsto (E|C).$$Then $C$ is the desired transition matrix.

Consider the following example. Suppose we have two bases of the vector space $\mathbb{R}^3$:$$e_1 = (-2, 3, 2), e_2 = (-2, 2, 1), e_3 = (-1, 1, 1) \\ \text{and} \\ e'_1 = (-2, 1, -1), e'_2 = (0, 2, 3), e'_3 = (1, 0, 1).$$Let's find a transition matrix from $\mathbf{e}$ to $\mathbf{e'}$. We write the $n \times 2n$-matrix:

$$\left(\begin{array}{ccc|ccc} -2 & -2 & -1 & -2 & 0 & 1\\ 3 & 2 & 1 & 1 & 2 & 0\\ 2 & 1 & 1& -1 & 3 & 1 \end{array}\right).$$Now we get this matrix to a row echelon form to get the identity matrix on the left:

$$\left(\begin{array}{ccc|ccc} -2 & -2 & -1 & -2 & 0 & 1\\ 3 & 2 & 1 & 1 & 2 & 0\\ 2 & 1 & 1& -1 & 3 & 1 \end{array}\right) \overset{r_3 +r_1, \ \ r_2+r_1}{\longrightarrow} \left(\begin{array}{ccc|ccc} -2 & -2 & -1 & -2 & 0 & 1\\ 1 & 0 & 0 & -1 & 2 & 1\\ 0 & -1 & 0& -3 & 3 & 2 \end{array}\right) \overset{r_1 \rightleftarrows r_2}{\longrightarrow} \\ \longrightarrow \left(\begin{array}{ccc|ccc} 1 & 0 & 0 & -1 & 2 & 1\\ -2 & -2 & -1 & -2 & 0 & 1\\ 0 & -1 & 0& -3 & 3 & 2 \end{array}\right) \overset{r_2 \rightleftarrows r_3}{\longrightarrow} \left(\begin{array}{ccc|ccc} 1 & 0 & 0 & -1 & 2 & 1\\0 & -1 & 0& -3 & 3 & 2 \\ -2 & -2 & -1 & -2 & 0 & 1\ \end{array}\right) \overset{-r_2}{\longrightarrow} \\ \longrightarrow \left(\begin{array}{ccc|ccc} 1 & 0 & 0 & -1 & 2 & 1\\0 & 1 & 0& 3 & -3 & -2 \\ -2 & -2 & -1 & -2 & 0 & 1\ \end{array}\right) \overset{r_3 + 2 \cdot r_1}{\longrightarrow} \left(\begin{array}{ccc|ccc} 1 & 0 & 0 & -1 & 2 & 1\\0 & 1 & 0& 3 & -3 & -2 \\ 0& -2 & -1 & -4 & 4 & 3\ \end{array}\right) \overset{r_3 + 2 \cdot r_2}{\longrightarrow} \\ \longrightarrow \left(\begin{array}{ccc|ccc} 1 & 0 & 0 & -1 & 2 & 1\\0 & 1 & 0& 3 & -3 & -2 \\ 0& 0 & -1 & 2 & -2 & -1\ \end{array}\right) \overset{-r_3}{\longrightarrow} \left(\begin{array}{ccc|ccc} 1 & 0 & 0 & -1 & 2 & 1\\0 & 1 & 0& 3 & -3 & -2 \\ 0& 0 & 1 & -2 & 2 & 1\ \end{array}\right).$$So, the matrix$$C = \begin{pmatrix} -1 & 2 & 1 \\ 3 & -3 & -2 \\ -2 & 2 & 1 \end{pmatrix}$$is a transition matrix from $\mathbf{e}$ to $\mathbf{e'}$.

Change of coordinates

The change of bases doesn't go without a trace. Since we use the basis to represent other vectors, it's natural to assume that the change of the basis will alter these representations.

Suppose we have an arbitrary vector $x \in V$, $\mathbf{e} = (e_1, \ldots, e_n)$ and $\mathbf{e'} = (e'_1, \ldots, e'_n)$ are two bases of the vector space $V$, and $C$ is a transition matrix from $\mathbf{e}$ to $\mathbf{e'}$.

The vector $x$ has the following coordinates:

$$X = \begin{pmatrix} x_1 \\ \vdots \\ x_n \end{pmatrix} \ - \text{in the basis } \mathbf{e}, \\ \ \\ X' = \begin{pmatrix} x'_1 \\ \vdots \\ x'_n \end{pmatrix} \ - \text{in the basis } \mathbf{e'}.$$From the definition of the transition matrices, we get the following rules:

$$X = C \cdot X', \\ \ \\ X' = C^{-1} \cdot X.$$Here's a picture which will help you to memorize these formulas:

These rules tell us how the basis change will affect the coordinates of arbitrary vectors. Let's see how this works in an example. We have two bases of the vector space $\mathbb{R}^3$:$$e_1 = (-2, 3, 2), e_2 = (-2, 2, 1), e_3 = (-1, 1, 1) \\ \text{and} \\ e'_1 = (-2, 1, -1), e'_2 = (0, 2, 3), e'_3 = (1, 0, 1).$$Suppose the vector $v$ has the following coordinates in the basis $\mathbf{e'}$:

$$v = (2, 1, 3).$$What are the coordinates of this vector $v$ in the basis $\mathbf{e}$?

We have seen above that the transition matrix in this case is

$$C = \begin{pmatrix} -1 & 2 & 1 \\ 3 & -3 & -2 \\ -2 & 2 & 1 \end{pmatrix}.$$Then, the coordinates of $v$ in the basis $\mathbf{e}$ can be found by the following formula:

$$\begin{pmatrix} -1 & 2 & 1 \\ 3 & -3 & -2 \\ -2 & 2 & 1 \end{pmatrix} \cdot \begin{pmatrix} 2 \\ 1 \\ 3 \end{pmatrix} = \begin{pmatrix} 3 \\ -3 \\ 1 \end{pmatrix}.$$So, the vector $v$ has the following coordinates in the basis $\mathbf{e}$:

$$v= \begin{pmatrix} 3 \\ -3 \\ 1 \end{pmatrix}.$$

Conclusion

In this topic, we've learned about a transition matrix. The transition matrix is what we get when we change bases.

There's a fairly simple process for finding transition matrices, which requires basic matrix manipulations. The transition matrix allows us to find the coordinates of arbitrary vectors in a new basis. It can be extremely useful if we're stuck working with an inconvenient basis.