MathAlgebraLinear algebraMatrices

Calculating matrix inverse

Provided by: Edvancium

11 minutes read

Although calculating inverses is usually complicated, you'll find out a universal algorithm to find them and also a general expression for it in terms of a lot of determinants.

To develop the algorithm, you'll need the elementary row operations that you already handle really well. Actually, the elemental matrices will be the building blocks. So let's not wait any longer, and go there!

Elementary matrices

As you know, applying an elementary row operation to a matrix $A$ is exactly the same as applying that operation to the identity (which is easy), forming a new matrix $E$ and then taking the product $EA$ . For this reason, $E$ is called elementary.

To find the inverse of a matrix, you are going to multiply a bunch of elementary matrices. For now, the first thing to note about any elementary matrix is that it's always invertible. Moreover, you can easily calculate its inverse.

Take, for example, the elementary matrix resulting from swapping the first two rows from the identity matrix $E = \begin{bmatrix}0 & 1 \\ 1&0 \end{bmatrix}$ . If you swap the rows again, you retrieve the identity matrix. But this is equivalent to premultiplying by the elementary matrix that swaps these rows, which in this particular case is precisely $E$ . This means that the inverse of $E$ is itself, which you can easily check:

$E E = \begin{bmatrix}0 & 1 \\ 1&0 \end{bmatrix} \begin{bmatrix}0 & 1 \\ 1&0 \end{bmatrix} = \begin{bmatrix}1 & 0 \\ 0&1 \end{bmatrix} = I$

In general, the inverse of any elementary matrix is in fact the elementary matrix that applies the same elementary operation, but in the "inverse" direction. That is when the operation is:

To multiply a row by a number $c$ , the inverse is to multiply the same line by $1/c$ .
To add a scalar multiple from one line to another, the inverse is simply to subtract the same scalar multiple from the first line to the second.
To exchange two rows, its inverse is itself.

For example, if you obtained the elementary matrix $E$ from the identity by multiplying a row by $5$ , then it is enough to multiply that same row by $1/5$ to recover the identity. So, if $F$ is the elementary matrix of this last operation, then $EF=1$ and by the same reasoning $FE=1$ . Thus $F = E^{-1}$ .

You're almost done. The last thing you must know is that products and inverses get along well.

If $A$ and $B$ are invertible, then $AB$ is also invertible, and its inverse is: $(AB)^{-1} = B^{-1} A^{-1}$

Proof

You only have to check out the definition of the inverse matrix. On one hand: $(AB) (B^{-1} A^{-1}) = A (BB^{-1}) A^{-1} = A I A^{-1} = A A^{-1} = I$ And on the other hand: $(B^{-1} A^{-1}) (AB) = B^{-1}( A^{-1}A)B = B I B^{-1} = B B^{-1} = I$

\blacksquare

How to find the inverse

Let's put all your work together:

Let $A$ be a square matrix of size $n$ . Then $A$ is invertible if and only if its reduced row echelon form is the identity. That is, if there exists elementary matrices $E_1, E_2, \dots, E_k$ such that: $E_k \, \dots \,E_2 \, E_1 A =I$ In this case, the inverse of $A$ is:

$A^{-1} = E_k \, \dots \,E_2 \, E_1$

Proof

If $E_k \, \dots \,E_2 \, E_1 A =I$ , then by multiplying both sides by the left by $(E_k \, \dots \,E_2 \, E_1)^{-1}$ it's clear that $A = (E_k \, \dots \,E_2 \, E_1)^{-1}$ Therefore, by multiplying by $E_k \, \dots \,E_2 \, E_1$ by the right, $A (E_k \, \dots \,E_2 \, E_1) = (E_k \, \dots \,E_2 \, E_1)^{-1} (E_k \, \dots \,E_2 \, E_1) = I$ . Thus, $A$ is invertible, and $A^{-1} = E_k \, \dots \,E_2 \, E_1$ .

For the rest of the proof, you need the following lemma:

Let $B$ be an invertible matrix and $E_1, E_2, \dots, E_k$ be elementary matrices.

If the matrix $T$ defined as $T = E_k \, \dots \,E_2 \, E_1 B$ is in row-echelon form, then all of the entries of its main diagonal are different from zero.

Proof

Suppose that there is an entry in the main diagonal of T which is zero.

That is, there is some $i \in \{1, 2, \dots, n\}$ such that $t_{ii} = 0$ .

Recall that $T$ is in row-echelon form. Think about the rows that are below the I-th one. They must have their first non--zero entry to the right of the I-th position. But as T is square, this causes the last row can't have a non-zero entry. Thus the last row of T is the zero vector.

Now, take any square matrix $C$ . What does the last row of TC look like? As the last row of $T$ is zero, the last row of $T \, C$ is the zero vector. In consequence, $T \, C$ cannot be the identity matrix $I$ . So that $C$ isn't the inverse of $T$ . Finally, as $C$ is an arbitrary square matrix, this result holds for every square matrix. Thus $T$ cannot be invertible, which is a contradiction.

\blacksquare

Great, let's now assume that if $A$ is invertible, first put it into a row-echelon form by applying some $l$ elementary row operations. By the lemma, all of the entries in the main diagonal of the matrix $E_l \, \dots \,E_2 \, E_1 A$ are non-zero. Then, by applying some more elementary row operations, you can convert all these entries in $1$ 's. After that, you can easily continue applying elementary row operations in order to transform into $1$ 's all of the entries out of the diagonal. Hence the reduced row echelon form of $A$ is $I$ .

Let's say that the total number of elementary row operations needed was $k$ . Then there exist elementary matrices $E_1, E_2, \dots, E_k$ such that $E_k \, \dots \,E_2 \, E_1 A =I$ . Hence by, as $A$ is invertible, $A^{-1} = I A^{-1} = E_k \, \dots \,E_2 \, E_1 A A^{-1} = E_k \, \dots \,E_2 \, E_1$ .

\blacksquare

Although it may not seem so at first sight, this result gives us an infallible algorithm to find the inverse of any invertible matrix:

Find the reduced row echelon form of $A$ .
If the result is the identity, you can move forward, otherwise $A$ is not invertible.
Save all the elementary matrices $E_1, E_2, \dots, E_k$ used in the previous process.
Then $A^{-1}$ is the product of these elementary matrices in the reverse order in which they were used. That is $A^{-1} = E_k \, \dots \,E_2 \, E_1$ .

But wait a minute. The product $E_k \, \dots \,E_2 \, E_1$ is the same as $(E_k \, \dots \,E_2 \, E_1) I$ . This means that the operations that are applied to $A$ in order to transform it into $I$ then have to be applied again to $I$ in the same order to convert it into $A^{-1}$ . So, why not apply the operations at the same time on both matrices?

First, join the two matrices into an augmented one by putting $I$ to the right of $A$ : $[A \, | \, I]$ After that, start applying elementary row operations to the big matrix in order to row reduce $A$ – this means that every operation applied to $A$ is immediately applied to $I$ . At the end, when on the left $A$ has become $I$ , on the right $I$ will have become $A^{-1}$ :

$[I \, | \, A^{-1}]$ For example, the inverse of the matrix $A = \begin{bmatrix}2&3 \\ 1&4 \end{bmatrix}$ is $A^{-1} = \begin{bmatrix}4/5&-3/5 \\ -1/5 & 2/5 \end{bmatrix}$ . Our method establishes that first of all, you have to build the augmented matrix $[A \, | \, I]$ and apply operations till $A$ is converted into $I$ and at that moment the original $I$ will be $A^{-1}$ . You can visualize it in the following way:

The process of finding the inverse

Better let's see the method in action!

An invertible matrix

Suppose you want to find the inverse of the following matrix:

$A = \begin{bmatrix}2&3 \\ 1&4 \end{bmatrix}$ The first step is to define the bigger matrix:

$[A∣I] = \left[\begin{array}{ll|ll} 2 & 3 & 1 & 0 \\ 1 & 4 & 0 & 1 \end{array}\right]$ Now compute the reduced row echelon form of $A$ . You can start by interchanging the two rows:

$\left[\begin{array}{ll|ll} 1 & 4 & 0 & 1 \\ 2 & 3 & 1 & 0 \end{array}\right]$ Now, subtract two times the first row from the second one:

$\left[\begin{array}{ll|ll} 1 & 4 & 0 & 1 \\ 0 & -5 & 1 & -2 \end{array}\right]$ Now divide the second row by $-5$ :

$\left[\begin{array}{ll|ll} 1 & 4 & 0 & 1 \\ 0 & 1 & -1/5 & 2/5 \end{array}\right]$ Finally, subtract $4$ times the second row from the first one:

$\left[\begin{array}{ll|ll} 1 & 0 & 4/5 & -3/5 \\ 0 & 1 & -1/5 & 2/5 \end{array}\right]$ You've just transformed $A$ into $I$ . So, the desired inverse is: $A^{-1} = \frac{1}{5} \begin{bmatrix}4&-3 \\ -1&2 \end{bmatrix}$

The inverse in terms of determinants

The determinant is a really versatile tool, and you can even use it to build the inverse of a matrix $A$ . The first step is to build a new matrix with all the possible cofactors of $A$ :

$\operatorname{cof}(A) = \begin{bmatrix} c_{11} & c_{12} & \cdots & c_{1 n} \\ c_{21} & c_{22} & \cdots & c_{2 n} \\ \vdots & \vdots & \ddots & \vdots \\ c_{n 1} & c_{n 2} & \cdots & c_{n n} \end{bmatrix} \in \mathbb{R}_{n\times n}$ Then you only have to transpose it and divide every entry by the determinant of the original matrix. The whole process is the following:

If $A$ is invertible, then:

$A^{-1}=\frac{1}{\operatorname{det}(A)} \operatorname{cof}(A)^T$

Proof

As always, proofs are optional. In this case, you should be familiarized with the properties of the determinant.

First, let's denote $X = A^{-1}$ . The strategy is to get every entry of $X$ . You know that $AX=I$ . Let $j \in \{1, \dots, n \}$ . On one hand, by the matrix product properties, the $j$ -th column of $A X$ is the product between $A$ and the $j$ -th column of $X$ , that is $A X_j$ . On the other hand, as $AX=I$ , the $j$ -th column of $A X$ is just the $j$ -th column of $I$ which is simply $e_j$ . Thus: $A X_j = e_j$ Nice! This is a simple system of linear equations with unknown vector $X_j$ , so you can apply the Cramer's Rule, to get every entry of $X_j$ . Then, for every $i \in \{1, \dots, n \}$ :

$x_{ij}= \frac{ \det(A_{(i)})}{\det(A)} = \frac{ \det([A_1 | \dots | A_{i-1}| e_j |A_{i+1} | \dots | A_n])}{\det(A)}$

Here comes the important part. Look at the numerator. It is the determinant of a matrix whose $i$ -th column is the vector $e_j$ . That's a column with a bunch of zeros. Actually, all its entries are zero, except the $j$ -th which is $1$ . Therefore, you can expand the determinant along this column, which means that only one term will survive:

$\det([A_1 | \dots | A_{j-1}| e_j |A_{j+1} | \dots | A_n]) = c_{ji} = (-1)^{i+j} m_{ji}$ But the minor $m_{ij}$ of matrix $[A_1 | \dots | A_{j-1}| e_j |A_{j+1} | \dots | A_n]$ is exactly the minor $m_{ij}$ of matrix $A$ because in both cases, removing the $i$ -th row or the $j$ -th column, the result is exactly the same (after all, both matrices are the same except for the $j$ -th column, which is exactly the one you removed!). Thus

$x_{ij} = \frac{ \det([A_1 | \dots | A_{i-1}| e_j |A_{i+1} | \dots | A_n])}{\det(A)} = \frac{(-1)^{i+j} m_{ji}}{\det(A)}$ Finally, by the definition of the transpose, $m_{ji}$ is the entry $(i,j)$ of $\operatorname{cof}(A)^T$

$\blacksquare$

As a simple but curious example, you can use this formula to find the inverse of any invertible matrix of size $2$ . After easily computing all the possible cofactors of any matrix $A=\begin{bmatrix} a & b \\ c & d \end{bmatrix}$ , you get that $\operatorname{cof}(A)=\begin{bmatrix} d & -c \\ -b & a \end{bmatrix}$ . Thus:

$A^{-1}=\frac{1}{\operatorname{det}(A)} \operatorname{cof}(A)^T=\frac{1}{a d-b c} \begin{bmatrix} d & -b \\ -c & a \end{bmatrix}$ Let's apply it to the matrix whose inverse you have already calculated, $A = \begin{bmatrix}2&3 \\ 1&4 \end{bmatrix}$ . Then $\det(A) = 8 - 3 = 5$ . Hence:

$A^{-1} = \frac{1}{5} \begin{bmatrix}4&-3 \\ -1&2 \end{bmatrix}$ This is exactly the same result as before! You didn't need to apply row operations or other cumbersome operations to calculate it! Nice.

Conclusion

Every elementary matrix is invertible, and its inverse is also elementary.
The product of invertible matrices $A$ and $B$ is also invertible, and $(AB)^{-1} = B^{-1} A^{-1}$ .
$A$ is invertible if and only if its reduced row echelon form is $I$ . In this case, its inverse is the result of applying on $I$ exactly the same elementary row operations that you used to transform $A$ into $I$ .
When the matrix is invertible, $A^{-1}=\frac{1}{\operatorname{det}(A)} \operatorname{cof}(A)^T$ . Where $\operatorname{cof}(A)$ is the matrix of cofactors of $A$ .
For any invertible $2\times 2$ matrix $A=\begin{bmatrix} a & b \\ c & d \end{bmatrix}$ , its inverse is obtained by simply switching the entries in the main diagonal, changing the signs for the other entries, and finally dividing every entry by the determinant, $\frac{1}{a d-b c} \begin{bmatrix} d & -b \\ -c & a \end{bmatrix}$ .

5 learners liked this piece of theory. 0 didn't like it. What about you?

Report a typo

Calculating matrix inverse

Elementary matrices

How to find the inverse

An invertible matrix

The inverse in terms of determinants

Conclusion

Related topics