Linear operators and matrices

You will finally discover the intimate connection between matrices and linear transformations, which is fundamental to understanding both. With it, you'll be able to easily analyze a system of equations completely with just a couple of calculations!
Now you will begin to reap the fruits of your labor. You'll put to work practically everything you've learned so far. This topic will even be much easier than the previous ones: you have already done all the hard work, it's time to enjoy!

The transformation generated by a matrix

Recent topics insisted that any $m \times n$ matrix $A$ induces a linear transformation $L_A$ thanks to the properties of the matrix product:

$$L_A: \, \mathbb{R}^n \rightarrow \mathbb{R}^m, \text{\, given by \,} L_A(v)=Av \text{\, with \,} v=v_1 e_1 + \dots + v_n e_n$$

Furthermore, the null space and the range of $L_A$ tell us several properties of the system of $A$. But let's stop for a minute to improve our notation. Although it's important to refer to the individual entries of the matrix, from now on, the main focus will be its columns. Let's denote them as $A_1, \dots, A_n$ and write the matrix in terms of them as follows:

$$A = \left[A_1\, |\, \dots |\, A_n \, \right]$$Clearly, the columns $A_1, \dots, A_n$ are vectors in $\mathbb{R}^m$. Here comes a little surprise. Do you remember that any linear transformation is completely determined by its values on a basis? Let's see what this looks like for $L_A$. Let's only transform the first vector of the canonical basis:

$$L_A (e_1) = A \, e_1 = A_1$$

By transforming $e_1$, you recover the first column of $A$! In fact, the same happens with the other vectors of the basis, and this is a key point:

As a direct consequence, you can reduce the product $Av$ to a linear combination:

$$L_A(v) = Av = v_1 A_1 + \dots + v_n A_n \in \mathbb{R}^m$$

The matrix of a transformation

There's something suspicious here. You already know that all the information of an arbitrary linear transformation $T: \mathbb{R}^n \rightarrow \mathbb{R}^m$ is contained in its values in the basis:

$$T(v) = v_1 T(e_1) + \dots +v_nT(e_n)$$So if you saved those values $\{T(e_1), \dots, T(e_n) \}$ somewhere you could rebuild the whole transformation without any problem. But look closely at the last equation... doesn't it look very similar to the last equation in the previous section, where you saw how to calculate the transformation of a matrix?

It seems that you can no longer avoid the fact that the place where you must store the values of the transformation is a matrix!

$$[T] = \left[ T(e_1) \, |\, \dots |\, T(e_n) \, \right]$$And $[T]$ is called the matrix associated with $T$. This means that:

$$T(v) = v_1 T(e_1) + \dots +v_nT(e_n) = [T] v = L_{[T]} (v)$$Thus, to fully know all the possible values of $T$, it is enough to just calculate their values in the basis and store them in a matrix!

In summary, you can think of every $m \times n$ matrix as if it were a linear transformation that deforms $\mathbb{R}^n$ into $\mathbb{R}^m$ and, conversely, every linear transformation between these spaces is completely determined by an $m \times n$ matrix.

Calculating the matrix of a transformation

Before you take a closer look at the relationship we just discovered, let's get our hands dirty with some examples.

Let's start with the following transformation:

$$T: \mathbb{R}^2 \rightarrow \mathbb{R}^3 \qquad T\begin{pmatrix}x \\ y \end{pmatrix} = \begin{pmatrix}x-y \\ 3x +2y \\ y \end{pmatrix}$$The first thing you have to do is calculate its values in the basis:$$\begin{align*} T(e_1)&=\begin{pmatrix}1 \\ 0 \end{pmatrix} = \begin{pmatrix}1 \\ 3 \\ 0 \end{pmatrix} \\ T(e_2) &=\begin{pmatrix}0 \\ 1 \end{pmatrix} = \begin{pmatrix} -1 \\ 2 \\ 1 \end{pmatrix} \end{align*}$$

Finally, use these values for the columns of the matrix of $T$:

$$[T] = \left[ T(e_1) \, |\, T(e_2) \, \right] = \begin{bmatrix} 1&-1 \\ 3&2 \\ 0&1 \end{bmatrix}$$Suppose now that you need to solve such a system of linear equations $[T]\ x = b, \ x \in \mathbb{R}^2, b \in \mathbb{R}^3$. The columns of $[T]$ are linearly independent (check it!), so the range of $T$ is a plane. By the dimension theorem, the null space of $T$ only contains $0$ and this, in turn, means that $T$ is injective. In consequence:

1. $[T]\ x = b$ doesn't always have a solution. For instance, if $b=(0,0,0)^T$ there aren't any solutions.
2. When $[T]\ x = b$ has a solution, it's unique. For example, if $b=(0,5,1)^T$ the unique solution is $(1,1) ^T$

You've just derived a couple of properties of $T$ and proved that a particular system of equations doesn't always have a solution just by building the matrix of $T$. Great, don't you think?

Now let's take the following operator:

$$T: \mathbb{R}^3 \rightarrow \mathbb{R}^3 \qquad T\begin{pmatrix}x \\ y\\z \end{pmatrix} = \begin{pmatrix} x+y+z \\ 2y-3z \\ z \end{pmatrix}$$Transforming the basis:

$$\begin{align*} T(e_1) &=\begin{pmatrix}1 \\ 0 \\0 \end{pmatrix} = \begin{pmatrix}1 \\ 0 \\ 0 \end{pmatrix} \\ T(e_2) &=\begin{pmatrix}0 \\ 1 \\ 0 \end{pmatrix} = \begin{pmatrix} 1 \\ 2 \\ 0 \end{pmatrix} \\ T(e_3) &=\begin{pmatrix}0 \\ 0 \\ 1 \end{pmatrix} = \begin{pmatrix} 1 \\ -3 \\ 1 \end{pmatrix} \end{align*}$$Thus the matrix of $T$ is:

$$[T] = \left[ T(e_1) \, |\, T(e_2) |\, T(e_3) \, \right] = \begin{bmatrix} 1&1&1 \\0 & 2&-3 \\ 0 & 0 &1 \end{bmatrix}$$

Now let's look at the operator. It is clear from the columns that its range is all $\mathbb{R}^3$. This means that the system $[T]\,x=b$ has a solution for any $b \in \mathbb{R}^3$. By the dimension theorem, the null space of $T$ is just $\{ 0\}$, in consequence $T$ is injective. But as it's an operator, it turns out that it's also bijective! Thus, the system $[T]\,x=b$ always has a unique solution.

Two sides of the same coin

Hopefully, the above examples have given you an idea of the power of the relationship between linear transformations and matrices. Consider the set of all linear transformations from $\mathbb{R}^n$ into $\mathbb{R}^m$, denoted by $L(\mathbb{R}^n, \mathbb{R}^m)$, and the set of $m \times n$ matrices $M_{m\times n}$.

To begin with, you know that they are vector spaces. But the interesting thing is that now you can connect them with a function! It is natural

that this function would be the one that assigns its matrix to each linear transformation. It is quite easy to show that:

$$[T +S] = [T] + [S] \text{ \, and \,} [\lambda T] = \lambda [T]$$

You have just built a linear transformation between $L(\mathbb{R}^n, \mathbb{R}^m)$ and $M_{m\times n}$! Fortunately, this function is injective and surjective, which implies that it is an isomorphism! In particular, the dimension theorem tells you that both spaces have the same dimension, $m \, n$.

This means that these sets are essentially the same, you can think of them as two ways of thinking of the same object. Since they are almost the same, you can study the properties of one through those of the other. Sometimes a result is easier to understand for matrices and then it is also true for transformations and vice versa.

Have you ever wondered why the matrix product has such a strange definition? Well, it turns out that it is made to mimic the behavior of the composition of linear transformations, but with matrices. If, after all, both sets are almost the same, it is not crazy to think that operations defined in one have an analog in the other. This very useful connection is reflected in the following result:

Conclusion

Let $L(\mathbb{R}^n, \mathbb{R}^m)$ be the set of all linear transformations from $\mathbb{R}^n$ into $\mathbb{R}^m$, and the set of $m \times n$ matrices $M_{m\times n}$.

Let $T \in L(\mathbb{R}^n, \mathbb{R}^m)$ and $A \in M_{m\times n}$.

• The values of $L_A$ in the basis are the columns of $A$

• $L_A(v) = Av = v_1 A_1 + \dots + v_n A_n$

• $[T] = \left[ T(e_1) \, |\, \dots |\, T(e_n) \, \right]$

• $[T +S] = [T] + [S] \text{ \, and \,} [\lambda T] = \lambda [T]$

• There is an isomorphism between $L(\mathbb{R}^n, \mathbb{R}^m)$ and $M_{m\times n}$. You can think of linear transformations and matrices as the same set but seen in different ways.

• If $S: \, \mathbb{R}^m \rightarrow \mathbb{R}^n$ and $T: \, \mathbb{R}^n \rightarrow \mathbb{R}^p$ are linear, then $[ T \circ S ] = [T] \, [S]$