Actions with linear operators

Just as vectors are essential in understanding the movement and relationships within a space, linear transformations provide a framework to explore how one vector space can be transformed into another.

You're already familiar with the basics of linear transformations, but now you'll learn to manipulate them just as fluently as vectors. Also, you'll start using an effective operation that only functions have, and that behaves like the product of matrices.

The linear transformations are vectors, too!

Vectors may be abstract objects, don't you think? But so far, you've been able to work with them fluidly and get a lot of useful results thanks to their structure: addition and scalar product have given you all the flexibility you need. What's more, we often don't think about vectors individually but rather exploit these operations. It doesn't matter if you are adding vectors in the plane or matrices, in the end, the operation behaves in the same way, and you get similar results.

Well, it turns out that among the linear transformations, you can also define a sum and a product by a scalar. But remember, the sum of two vectors and their product by a scalar must also be another vector, so the result of combining the transformations must be another transformation.

Let's take two vector spaces $V$ and $W$. From now on, denote the set of all linear transformations from $V$ to $W$ as $L(V, W)$. If $S,T \in L(V,W)$, then define their sum, $S+T$, as the transformation that assigns to each vector $v \in V$ the vector $T(v) + S(v) \in W$. Similarly, for any scalar $\lambda$, you can define a new transformation $\lambda T$ whose value in $v$ is simply $\lambda T(v) \in W$. In summary:

But here comes the litmus test: does this sum and this scalar product make $L(V, W)$ a vector space? The answer is yes! And the proof is simple, the only fine point is that the set needs to have a $0$. Well, this role is fulfilled by the constant function $\mathscr{O}$ given by $\mathscr{O}(v) = 0$ for all $v \in V$.

In particular, the set of linear operators on $V$, denoted by $L(V, V)$, is simply written as $L(V)$.

Composing transformations

Functions are like processes that are applied to objects in a set, so they can be chained together to form more complex processes that connect different sets. Since linear transformations are functions, of course, you can compose them, you only need one more space, say $U$. If $S \in L(U,V)$ and $T \in L(V,W)$, then their composition is:

$$T \circ S:U \rightarrow W, \quad (T \circ S)(u) = T(S(u))$$That is, first $S$ transforms $u \in U$ into $S(u) \in V$ and then $T$ transforms the latter into $T(S(u)) \in W$.

But what really matters is whether the composition preserves linearity, that is, whether $T \circ S$ is linear. Fortunately, the answer is also affirmative and will have invaluable consequences later. So $T \circ S \in L(U,W)$.

This new operation is quite peculiar and unique, it is an independent quality of the vector space structure. In reality, it is similar to the product of matrices. Actually, you'll find out later that this relationship is much more intimate than it may seem, and incidentally it has similar properties:

• $(T \circ S) \circ R = T \circ ( S \circ R)$
• $I \circ T = T = T \circ I$
• $(T_1 + T_2) \circ S = (T_1 \circ S) +( T_2 \circ S)$
• $T \circ (S_1 + S_2) = (T \circ S_1 ) + (T \circ S_2)$
• $T \circ (\lambda S ) = (\lambda T) \circ S = \lambda (T \circ S )$

Looking carefully at these properties, it is impossible not to notice their resemblance to multiplication. This is why they are often also called a product. But this is only an appearance, because the composition is not commutative, since the expression $S \circ T = T \circ S$ doesn't even make sense. Even when $T,S \in L(V)$ it is rare for commutativity to hold, another similarity with the product of matrices.

Taking powers

Let's go further with the product. As with the multiplication of numbers, you can define the powers of an operator $T \in L(V)$ through composition. For example, the square of $T$ is the product of $T$ with itself, that is, the composition of $T$ with $T$:

$$T^2 = T \circ T$$

Larger powers are defined in a similar way, but in general, you can write them compactly as $T^n = T \circ T^ {n-1}$. Again in analogy with numbers, you can define the powers $0$ and $1$ as $T^ 0 = I$ (the identity function) and $T^1 = T$. Thinking of the composition as the successive application of the same transformation, it is evident that the power complies with the elemental properties:

• $T^m T^n = T^{m + n}$
• $(T^m)^n = T^{mn}$

But the functions are not numbers, and the composition is not a product, therefore it has other properties. Sometimes after applying an operator several times, it stops having any effect. These types of operators are called idempotent, and the simplest example is the projection on a line.

Take a vector $u$ in the plane, after projecting it with a transformation $P \in L(\mathbb{R}^2)$ on a line, no matter how many more times you project it again, the result doesn't change.

The geometry of transformations

Since linear transformations are vectors, let's start treating them as such with some examples. Let's go to the plane and take a vector $u=(x,y)$, you are going to investigate the geometry of the projections in the coordinate axes $P, Q \in L(\mathbb{R}^2)$ given by:

$$P(x,y)=(x,0) \qquad Q(x,y)=(0,y)$$

Let's start combining the projections. Just by adding them, you have that $(P+Q) (u) = P(u) +Q(u) = (x,0) + (0,y) = (x,y) = u = I(u)$ and so $P+Q =I$.

If you now scale $P$ by a scalar, say $5$, then you get that $(5P)(u) = 5P(u) = 5(x,0)=(5x,0)$, which means that the result is simply a stretch of the projection.

A curious combination is $P-Q$. Note that $(P-Q)(u) = P(u)-Q(u) = (x,0)-(0,y)=(x,-y)$ and therefore $P-Q$ is the transformation that reflects each vector about the horizontal axis. What geometric meaning do you imagine that $Q-P$ have?

Since $P$ and $P$ are projections, they are idempotent. For example, $P^2(u) = P(P(u)) = P(x,0)=(x,0)$, so any power of $P$ is equal to $P$, you can verify that the same thing happens with $Q$.

It's time to compose the transformations! On one hand, $(P\circ Q)(u) = P(Q(u)) = P(0,y)= (0,0)$ and on the other hand$(Q \circ P)(u) = Q(P(u)) = Q(x,0)=(0,0)$. This means that $P\circ Q = Q \circ P = \mathscr{O}$,

Even more surprising, you can easily calculate their powers since $(P\circ Q)^2 (u)=\mathscr{O}^2(u) = \mathscr{O}(u)$.

Conclusion

Let $V$ and $W$ be two vector spaces.

• $L(V, W)$ is the set of linear transformations from $V$ into $W$
• $L(V)$ is the set of linear operators in $V$
• $L(V, W)$ is a vector space with the operations:
• $$T+S:\, V \rightarrow W, \quad (T+S)(v) = T(v) + S(v)$$
• $$\lambda T:\, V \rightarrow W, \quad (\lambda T)(v) = \lambda T(v)$$
• If $S \in L(U,V)$ and $T \in L(V,W)$, then their composition is in $L(U, W)$ and is given by:
• $$T \circ S:U \rightarrow W, \quad (T \circ S)(u) = T(S(u))$$
• Usually $T \circ S \neq S \circ T$
• If $T \in L(V)$ then their powers are:

$$\begin{align*} T^0 &= I \\ T^1 &= T \\ T^2 &= T \circ T \\ & \vdots \\ T^n &= T \circ T^{n-1} \end{align*}$$

• If $T \in L(V)$ then it's idempotent if $T^2 = T$