Determining linear dependence

Imagine you're an architect, and you're given a set of building blocks to construct a model. Some blocks are unique, while others are just a combination of scaled versions of other blocks. Now you're asked to create a new, unique design using any blocks of your choice. Of course, you want to add as much unique information to your model. Therefore you only use the unique building blocks. But how can you find out which of them are unique?

In the world of linear algebra, vectors are like these building blocks: Vectors that can be constructed using scaled versions of others don't provide any new direction to the vector space, much like the redundant blocks in our model. Finding these vectors consists of determining the linear dependence between them. In this topic, you will learn three methods to determine linear dependence and its usage in linear algebra.

Approach 1: Vector Equations

The vector equations approach to determining linear dependence stems from the fundamental principles of linear algebra. It operates on the idea that vectors are linearly dependent if and only if at least one of the vectors can be written as a linear combination of the others. A linear combination, in this context, means that you can multiply each of the other vectors by some scalar, and then add them together to get the vector in question.

To determine linear dependency or independence, we often set up a system of linear equations to find the coefficients for the linear combination. If there exists a non-trivial solution (a solution other than all zeros) to the system, then the vectors are linearly dependent. If the only solution is the trivial solution (all zeros), then the vectors are linearly independent.

Let's illustrate this concept with an example. Suppose you have three vectors in a three-dimensional space:

$u = \begin{pmatrix} 2 \\ 0 \\ 1 \end{pmatrix} \quad v = \begin{pmatrix} 1 \\ 1 \\ -1 \end{pmatrix} \quad w = \begin{pmatrix} 4 \\ 2 \\ -1 \end{pmatrix}$

You want to determine if these vectors are linearly dependent. To do this, set up the following system of equations to check whether $w$ can be expressed as a linear combination of $u$ and $v$. That is, you want to find scalars $x$ and $y$ such that: $au + bv = w \quad \text{, which is equivalent to} \quad a(u_1, ... , u_n)^T + b(v_1, ..., v_n)^T = (w_1, ..., w_n)^T.$Rearranging this into each individual equation will result in the linear system:$au_1 + bv_1 = w_1 \\ au_2 + bv_2 = w_2 \\ ... \\ au_n + bv_n = w_n$Inserting the vectors in our example, you receive:

\(\begin{align*} 2a + b &= 4 \\ b &= 2 \\ a - b &= -1 \end{align*}\)From the second equation, you see that $b = 2$. Substituting $b$ into the third equation, you get $a - 2 = -1$, therefore $a = 1$. Using both $a$ and $b$ in the first equation results in $2 + 2 = 4$, confirming our found coefficients.

Since you've found a non-trivial solution $(a = 1 , b = 2)$, you can conclude that $w$ can be expressed as a linear combination of $u$ and $v$. Thus the vectors $u$, $v$, and $w$ are linearly dependent.

Approach 2: Row Reduction

Since you learned about determining linear dependence between vectors, what about rows/columns of a matrix?

Row reduction, also known as Gaussian elimination, is a method used to solve systems of linear equations and to determine properties of vectors and matrices like linear dependence. The basic idea is to perform a sequence of elementary row operations on a matrix to transform it into a simpler form, which makes it easier to analyze and solve systems of equations or determine linear dependence.

1. Setup: When given a set of vectors, write them as columns of a matrix. This matrix will represent the system you're studying. (Skip if a Matrix is already given)

2. Row Echelon Form: Apply elementary row operations to the matrix to transform it into row echelon form. Remember: In row echelon form, the leading coefficient (the first nonzero entry) of each row is to the right and below the leading coefficient of the previous row. Rows of zeros, if any, are at the bottom.

3. Reduced Row Echelon Form: Starting from the last row with a nonzero entry (pivot row), work your way upwards. Use the pivot to clear out the entries below it by subtracting multiples of the pivot row from the rows below. In reduced row echelon form, each pivot is 1, and the entries above and below each pivot are zero.

4. Analyze the Results: Once you've reached reduced row echelon form, you can read off solutions to systems of equations (if applicable), and you can also determine linear dependence among vectors based on the number of pivots.

Let's take a look at an example. You are given three vectors, $u$, $v$, and $w$, and are asked to determine the linear dependence between them. The vectors are as follows: $u = \begin{pmatrix} 1 \\ 2 \\ 3 \end{pmatrix} \quad v = \begin{pmatrix} 2 \\ 4 \\ 6 \end{pmatrix} \quad w = \begin{pmatrix} 1 \\ 1 \\ 1 \end{pmatrix}$

In this example, you have three rows and two pivots. Since the number of pivots (2) is less than the number of vectors (3), the vectors are linearly dependent.
Note: By simple inspection, it is obvious that the second vector is two times the first one. Therefore they are linearly dependent.

Approach 3: Determinant

Last but not least, the determinant of a matrix encodes information about its linear transformation properties, and it can be used to identify linear dependence among a set of vectors. It's important to note that the determinant method is applicable to square matrices only. Despite this reduced applicability, this is the quickest method.

1. Construct the Matrix: Form a matrix with the given vectors as columns. (Skip if a matrix is already given)

2. Calculate the Determinant: Calculate the determinant of the matrix.

3. Analyze the Result: If the determinant is zero, the vectors are linearly dependent. If the determinant is nonzero, the vectors are linearly independent.

Let's again take a look at an example. You are given three vectors, $u$, $v$, and $w$, and are asked to determine the linear dependence between them. The vectors are as follows: $u = \begin{pmatrix} 1 \\ 2 \\ 3 \end{pmatrix} \quad v = \begin{pmatrix} 2 \\ 4 \\ 6 \end{pmatrix} \quad w = \begin{pmatrix} 3 \\ 6 \\ 9 \end{pmatrix}$

In this example, the determinant of the matrix formed by the vectors is zero, indicating that the vectors are linearly dependent.
Note: Again, through simple inspection, you can notice that the second vector is two times the first one, whilst the last one is three times the first one. This is due to using simple examples for showcasing!

Conclusion

In this topic, you have explored the concept of linear dependency and three methods to determine it:

• The vector equations approach checks if any vector can be expressed as a linear combination of others. This approach involves solving a system of linear equations.

• The row reduction approach consists of simplifying a matrix to reduced row echelon form, making linear dependence apparent.

• The determinant approach involves building a matrix by putting the vectors as its columns. If the determinant is zero, they're linearly dependent. Otherwise, they're linearly independent.

Understanding these methods and their applications will provide you with effective tools to analyze and solve problems in various fields that use linear algebra, as determining linear dependence is quite common, and solving it quickly is of great advantage.