Vectors, matrices, scalars

Maybe you want to describe the velocity of a projectile or determine the magnetic flux passing through a surface, or perhaps you need to gather and prepare data to be used in a machine learning model. Be it in physics, pure mathematics, computer science, economics, biology, or pretty much any other field of science, you're almost guaranteed to encounter vectors. In this topic, we're going to define these mathematical elements, explore some of their representations, and their relationship to matrices.

The idea of a vector

Most often, people characterize a vector as an ordered set of elements of the same type.

Programmers use vectors to describe specific objects. In this case, elements of a vector represent the characteristics of the described object. This way we can characterize a person by a set of (height, weight) or (eye color, hair color).

Physics uses vectors to describe different phenomena. It's convenient to represent these vectors as a directed line segment, which has magnitude and direction. Such vectors are usually depicted like this:

With this vector representation, we can describe a myriad of physical quantities: velocity, acceleration, force, electric and magnetic fields, and so on. As arrows are directed line segments, they can help us visualize the direction of a given physical quantity. For example:

Weight (W) is a force exerted by the gravitational pull of the earth on an object. It always points down to the earth (the direction that all falling objects follow).

The initial velocity (Vo) of a projectile tells us how fast the object is moving at the beginning of the observation (t=0).

Vectors

Mathematicians like to generalize. Therefore, if you ask a mathematician what a vector is, the answer will be that a vector is an element of a vector space. This means that vectors can be added together and multiplied by a scalar (for example, scalars could be real or complex numbers); both of these operations output another vector. The simplest case of a vector space is a set of lists of the same length $n$, consisting of real numbers, which are called components. The number of components (i.e. $n$) is called the dimension of the vector space.

Vectors could have alternate notations in different contexts. In a geometric context, vectors are traditionally denoted as a Latin letter with an arrow or with a dash above: $\vec{a}$ or $\bar{a}$. In algebraic context, vectors are denoted as plain Latin letters or bold Latin letters: $a$, $\bm{a}$.

Components of a vector $\vec{a}$ are usually denoted by the name of the vector with a subscript, denoting the number of the component: $a_1, a_2,$ and so on up to $a_n$. All components of a vector are usually written in the form of a horizontally or vertically aligned list:

$\vec{a} = \begin{bmatrix} a_1, \ldots,a_n \end{bmatrix} \text{ or } \vec{a} = \begin{bmatrix} a_1 \\ \ldots \\ a_n \end{bmatrix}$

Sometimes components of the vector are also listed in round brackets. This notation has no additional meanings to it and depends entirely on the preferences of the writer:

$\vec{b} = \begin{pmatrix} b_1, \ldots,b_n \end{pmatrix} \text{ or } \vec{b} = \begin{pmatrix} b_1 \\ \ldots \\ b_n \end{pmatrix}$

For example, a two-dimensional vector with the first component $3$ and the second component $4$ could be written as

${a} = \begin{bmatrix} 3, 4 \end{bmatrix} \text{ or } {a} = \begin{pmatrix}3 \\ 4\end{pmatrix}$

From now on, for the sake of clarity, we will stick to the arrow notation throughout this topic.

Now imagine a plane. If you set a Cartesian coordinate system on it, then you turn it into a vector space. Then any point of this plane can be represented as a vector (list of two numbers). Its components will be the coordinates of the point along the x-axis and the y-axis:

Note that the points with coordinates $(3, 4)$ and $(4, 3)$ are different points, i.e. the order of the components of the vector does matter.

In the example with the plane, we were dealing with two-dimensional vectors $\vec{a}$ and $\vec{b}$, where:

$\vec{a} = \begin{bmatrix} a_1 \\ a_2\end{bmatrix} = \begin{bmatrix} 3 \\ 4 \end{bmatrix} \text{ and } \vec{b} = \begin{bmatrix} b_1 \\ b_2\end{bmatrix} = \begin{bmatrix} 4 \\ 3\end{bmatrix}$

A three-dimensional vector looks like this:

Note that in this case, it is more convenient to represent the vector as a directed line segment from the origin. This way, it is easier to see what all three of its components are equal to.

Scalars

A one-dimensional vector in the Cartesian coordinate system will turn into a single number that determines the position of a point on a straight line relative to the origin:

In mathematics, a value that can be expressed by a single number is called a scalar. We can say that a scalar is a special case of a vector when its dimension is equal to one. In our examples, each component of a two-dimensional and three-dimensional vector is a scalar.

Wait, it turns out that a vector can also consist of vectors? Yes, it really can. Components of a vector could be defined as anything that could be summed up and multiplied by a number. For example, real-valued functions are defined on the same set of input values. And this allows us to move on to the concept of a matrix.

Vectors and matrices

So far, we have expressed vectors as lists of numbers. But most often it's convenient to store and process data represented in the form of rectangular arrays. We call such arrays matrices, and we can think of them as vectors of vectors:

$[ \vec{u}, \ \vec{v},\ \vec{w}] \quad ; \quad \vec{u} = \begin{bmatrix} u_x\\ u_y \\u_z \end{bmatrix}, \ \vec{v} = \begin{bmatrix} v_x\\ v_y \\ v_z \end{bmatrix}, \ \vec{w} = \begin{bmatrix} w_x\\ w_y \\ w_z \end{bmatrix}$

We could write this as a matrix, by stacking them side by side as columns:

$M = \begin{bmatrix} u_x & v_x & w_x \\ u_y & v_y & w_y \\ u_z & v_z & w_z \end{bmatrix}$Or by stacking each vector on top of each other as rows:

$M = \begin{bmatrix} u_x & u_y & u_z \\ v_x & v_y & v_z \\ w_x & w_y & w_z \end{bmatrix}$Usually, we use column vectors when we perform mathematical manipulations.

You can read more about matrices in the following topic: Introduction to matrices.

Example: color pictures

Digital pictures are made of a finite number of pixels. For grayscale photos, one pixel is just a number that tells us the amount of light sampled by the camera at that spot. We just need to define upper and lower bounds and assign them to pure white and pure black. For example, when working with 8-bit integers, 255 represents white (maximum amount of light) and 0 represents black (no light). So, a rectangular picture is stored as a rectangular array of numbers, i.e. as a matrix:

Matrices and matrix operations are used extensively in image processing. There are endless algorithms that use matrix operations to solve problems of edge detection, noise suppression, classification of images, identification of shapes, and so on. Moreover, in many machine learning methods it's essential to consider not the matrix as a whole, but its rows. As we've already discussed today, rows of a matrix are vectors, so they are also crucial for image processing.

Conclusion

To sum up, what we have covered today, here are some crucial points:

• A vector contains an ordered set of elements that describe something

• A vector that has $n$ elements is called $n$-dimensional vector

• Order of the components of a vector matters, this means that vector $(1,2)\neq(2,1)$

• A scalar is a special case when a vector is $1$-dimensional and so has only $1$ component

• We can stack vectors and as a result, we will get matrices, which are useful for storing data