Eigenvalues and eigenvectors

Have you ever played a game of billiards? When you hit another ball using the cue ball, it moves and changes direction. Similarly, in mathematics, a linear operator acts like a function in vector spaces. You apply the operator to a vector, and it changes its direction.

However, among all the vectors, there's a special kind called eigenvectors that behave differently. In this topic, we'll explore eigenvectors in detail. You will see how to formally define eigenvectors and how to find them. The topics you have already learned will help you calculate those values! By understanding eigenvectors, you can gain insights into how linear operators behave and their role in a variety of applications, ranging from physics to data analysis.

What are eigenvalues and eigenvectors?

An eigenvector $v$ is a vector that, when transformed by a linear operator $A:V \to V$, is only scaled by a scalar value. This value $\lambda$ is called the eigenvalue of the transformation.

In our billiards example, this would happen upon hitting a ball perfectly straight so it follows the direction in which the shot was lined up.

As an example, consider a matrix $A$ and an eigenvector $v$

$$\begin{align*}A= \begin{pmatrix} 3 & 1 \\ 1 & 3 \end{pmatrix} , v= \begin{pmatrix} 1 \\ 1 \end{pmatrix} \end{align*}$$Through the multiplication of $A$ and $v$, you can check if the result is a scalar multiple of $v$:

$$\begin{align*} Av= \begin{pmatrix} 3 & 1 \\ 1 & 3 \end{pmatrix} * \begin{pmatrix} 1 \\ 1 \end{pmatrix} = \begin{pmatrix} 4 \\ 4 \end{pmatrix} \end{align*}$$As you can see $Av=4v$, where the scalar multiple is $4$. Therefore, $v$ is an eigenvector for $A$ with eigenvalue $\lambda=4$.

Finding all eigenvalues of a linear operator

Now let's try to find the eigenvalues of a certain linear operator $A$. The set containing all eigenvalues of a specific linear operator is called its spectrum and is denoted as $Spec A$.

To better understand the steps for finding all eigenvalues, let's simultaneously calculate the eigenvalues and eigenvectors for the matrix $A= \bigl( \begin{smallmatrix} 3 & 1 \\ 2 & 2 \end{smallmatrix} \bigr)$.

The steps for finding all eigenvalues that belong to the spectrum are as follows:

1. Set up the equation $Av=\lambda v$
2. Assume a nonzero vector $v$ as the eigenvector. This assumption is necessary to seek an actual eigenvector with the special property of producing a scalar multiple of itself when multiplied by an operator $A$.
   For our example, let's assume $v= \bigl( \begin{smallmatrix} x \\ y \end{smallmatrix} \bigr)$.
3. Substitute the values of $A$ and $v$ into the equation from 1. and you get:
   $$\begin{pmatrix} a_{1,1} & a_{1,2} & \dots & a_{1,m} \\ a_{2,1} & a_{2,2} & \dots & a_{2,m} \\ \vdots & \vdots & \ddots & \vdots \\ a_{n,1} & a_{n,2} & \dots & a_{n,m} \end{pmatrix} \begin{pmatrix} v_1 \\ v_2 \\ \vdots \\ v_n \end{pmatrix} = \lambda \begin{pmatrix} v_1 \\ v_2 \\ \vdots \\ v_n \end{pmatrix}$$By extracting each row we get $n$ equations:$$a_{1,1}v_1+a_{1,2}v_2+\dots+a_{1,m}v_n=\lambda v_1 \\ a_{2,1}v_1+a_{2,2}v_2+\dots+a_{2,m}v_n=\lambda v_2 \\ \vdots \\ a_{n,1}v_1+a_{n,2}v_2+\dots+a_{n,m}v_n=\lambda v_n$$

   Substituting $A$ and $v$ from our example will lead to:

   $$\begin{pmatrix} 3 & 1 \\ 2 & 2 \end{pmatrix} \begin{pmatrix} x \\ y \end{pmatrix} = \lambda \begin{pmatrix} x \\ y \end{pmatrix}$$

   $$3x+y= \lambda x \\ 2x+2y= \lambda y$$

4. Rearrange the equation to set it equal to zero, with $I$ being the identity matrix of size $n\times n$.
   $$(A- \lambda I)v=0$$Rearranging the equations from our example will result in:$$(3- \lambda)x+y=0 \\ 2x+(2- \lambda )y=0$$

5. Calculate the determinant of $(A- \lambda I)$ and solve the equation for $\lambda$:$$det(A- \lambda I)=0$$

   Use the determinant to find eigenvalues and eigenvectors. By setting the determinant of the matrix (A - λ * I) to zero, you can determine the eigenvalues that satisfy the condition for non-trivial eigenvectors, where non-trivial eigenvectors are those that are not the zero vector. The solution for this equation looks like this $\xi_A(λ)=b_nλ^n+b_{n−1}λ^{n−1}+…+b_1λ+b_0$ and is called the characteristic polynomial $\xi$ of the linear operator $A$. The solutions for $\lambda$ will be the eigenvalues of $A$.
   By calculating the determinant for our instance $det \bigl( \begin{smallmatrix} 3-\lambda & 1 \\ 2 & 2-\lambda \end{smallmatrix} \bigr) =0$, you will receive the following equation:$$\begin{align*} (3-\lambda)(2-\lambda)-(1)(2)&=0 \\ \iff (3-\lambda)(2-\lambda)-2&=0 \\ \iff (\lambda-1)(\lambda-4)&=0 \end{align*}$$ Now you can solve for $\lambda$ and therefore extract the eigenvalues of $A$.$$\lambda -1=0 \iff \lambda=1 \\ \lambda -4=0 \iff \lambda=4$$

6. Substitute each eigenvalue back into the equation $(A-\lambda I)v=0$, to find their corresponding eigenvectors.
   For $\lambda=1$ you get the following equations:$$\begin{pmatrix} 3-1 & 1 \\ 2 & 2-1 \end{pmatrix} \begin{pmatrix} x \\ y \end{pmatrix} = \begin{pmatrix} 0 \\ 0 \end{pmatrix} \implies \begin{matrix} 2x+y=0 \\ 2x+y=0 \end{matrix}$$ By observing the equations, we find that the eigenvectors corresponding to the eigenvalue $\lambda = 1$ are of the form $v=\bigl( \begin{smallmatrix} -\frac{1}{2}y\\ y\end{smallmatrix} \bigr)$, where $y$ is a real number.$$\begin{align*} 2x+y&=0 \\ \iff 2x&=-y \\ \iff x&=\frac{-y}{2} \end{align*}$$ In particular, choosing $y = 1$ yields the eigenvector $v=\bigl( \begin{smallmatrix} -\frac{1}{2} \\1 \end{smallmatrix} \bigr)$. It should be mentioned that the eigenspace associated with the eigenvalue $\lambda = 1$ consists of all possible scalar multiples of the eigenvector $v=\bigl( \begin{smallmatrix} -\frac{1}{2} \\1 \end{smallmatrix} \bigr)$, including the zero vector.
   
   To keep the topic short, calculating the eigenvector for $\lambda =4$ is omitted. Upon doing so, the corresponding eigenvector will be $\bigl( \begin{smallmatrix} 1 \\ 1 \end{smallmatrix} \bigr)$.

Conclusion

In this topic, you learned about eigenvalues, eigenvectors, and how to calculate them. Try to remember the essential formulas! The most important points are:

• An eigenvector is a vector that, when transformed by a linear operator, is only scaled by a scalar value. $Av=\lambda v$
• This scalar value is called the eigenvalue of the transformation. Solving the equation $det(A- \lambda I)=0$ will result in the polynomial $\xi_A(λ)=b_nλ^n+b_{n−1}λ^{n−1}+…+b_1λ+b_0$.
• The set of all eigenvalues of a certain linear operator is called its spectrum.

Have fun practicing!