Currently I'm reading the Mathematics for Machine Learning book, and in the beginning of section 2.7 of the book, the following is given:
2.7 Linear Mappings
In the following, we will study mappings on vector spaces that preserve their structure, which will allow us to
define the concept of a coordinate. In the beginning of the chapter, we
said that vectors are objects that can be added together and
multiplied by a scalar, and the resulting object is still a vector. We
wish to preserve this property when applying the mapping: Consider
two real vector spaces V,W. A mapping Φ : V → W preserves the
structure of the vector space ifΦ(x + y) = Φ(x) + Φ(y) (2.85)
Φ(λx) = λΦ(x) (2.86)
for all x, y ∈ V and λ ∈ R.
Generally, when vectors are added together, or multiplied by a scalar, the result is also a vector (the same type of object).
How are the above properties of a linear transformation preserving this?
As long as the LHS and RHS of both given properties belong to the same vector space, can I not consider the vector properties preserved? Why do they have to be equal?
What does it mean by they preserve the structure of a vector space?
Is there a particular reason Φ(λx + ψy) = λΦ(x) + ψΦ(y) has a form similar to a linear combination (y = a1.x1 + a2.x2 +…)?
Best Answer
The reason the author says this is that it's a kind of motivation for the definition of a linear mapping. Just saying a linear map is any $f : V \rightarrow W$ for vector spaces $V$ and $W$ which satisfies:
$$f(u+v) = f(u) + f(v)$$
and
$$f(\lambda u) = \lambda f(u)$$
is indeed the definition, but it is not enlightening in the slightest, so the author is attempting to give some insight into why we have this definition.
Let's go over what vector properties mean. First, we have that adding vectors produces a vector, so, for vectors in boldface, we have:
$$\mathbf{u + v} = \mathbf{u} + \mathbf{v}$$
For example:
$$\begin{bmatrix} 1 + 2 \\ 4+5 \\ -1 + 15 \end{bmatrix} = \begin{bmatrix} 1 \\ 4 \\ -1 \end{bmatrix} + \begin{bmatrix}2 \\ 5 \\ 15 \end{bmatrix}$$
We also have the ability to pull out scalars:
$$\mathbf{au} = a\mathbf{u}$$
For example:
$$\begin{bmatrix} 10 \\ 40 \\ -10 \end{bmatrix} = 10\begin{bmatrix} 1 \\ 4 \\ -1 \end{bmatrix}$$
What this tells us is that the linear map is compatible with the rules of vector arithmetic. This is one of the author's intentions behind saying that it maintains structure.
Indeed, if $\mathbf{w} = \lambda \mathbf{u} + \delta \mathbf{v}$, if $f$ is a linear mapping, then $f(\mathbf{w}) = \lambda f(\mathbf{u}) + \delta f(\mathbf{v})$ (can you prove this?). So a linear map uniquely preserves the relationship between different vectors. Can you find three vectors $\mathbf{a}, \mathbf{b}, \mathbf{c}$ such that $\mathbf{a} = \mathbf{b} + \mathbf{c}$, but for $g(x) = x^2$, we have $g(\mathbf{a}) \neq g(\mathbf{b}) + g(\mathbf{c})$? Why does this tell us that $g$ is not a linear map? Can you find a counterexample to prove that $g$ is not a linear map?
This idea that maps "preserve the decomposition of a vector into a sum of scaled vectors" will be vital to defining things like a basis of a vector space, and other useful properties.
In this case, once again consider $g(x) = x^2$. This function runs from $\mathbb{R}$ to $\mathbb{R}$, both of which are vector spaces, but it does not satisfy our properties. Hence while the output is a vector, the way that the output relates to its component parts is not the same as the way that the input relates to its component parts.
Once again, this is related to the idea of a basis, in which we find we may write a vector as a unique linear combination of a finite set of vectors (if the vector space is finite dimensional), and linear maps preserve that combination, but I do not know if you have been exposed to a basis yet.