I am trying to understand the concept of tensors. I seem to understand that they are generalization of vectors: They are subject to similar basis transformations with vectors but I am somewhat confused about their relations with matrices. In 3 dimensional Euclidian space, a rank 1 tensor is represented with a vector, a rank 2 tensor is represented with a 3×3 matrix, a rank 3 tensor is a 3x3x3 sized three dimensional array and so on. Assuming we are in 3 dimensional space, for example, is it possible to have an arbitrary sized matrix; say 6×7 sized matrix as the representation of a tensor? Or can they be only represented with arrays with dimensions of three's powers? I think that this is not possible since an arbitrary sized matrix cannot undergo basis transformations.
My second question is about tensor multiplications. This operation seems like a cartesian product between the elements of two tensors, for example the tensor product of a rank 1 (3 elements) and rank 2 (9 elements) tensor generates a rank 3 three tensor (27 elements). Is this operation related to standard matrix multiplication somehow? For example, we can do matrix multiplication between 3×1 and 1×3 vectors and obtain a 3×3 matrix. Isn't this a tensor product between two rank 1 tensors in the same time? Is this a coincidence?
Best Answer
I think putting tensors in the right context will clear much of this up. I'll stick with $\mathbb R^3$ since that's the example you use. Rank $2$ tensors are elements of the so-called tensor product of $\mathbb R^3$ with itself, which is denoted by $\mathbb R^3 \otimes \mathbb R^3$. This space consists of all linear combinations of expressions of the form $u \otimes v$ under the stipulations that: $$u\otimes(v + w) = u\otimes v + u\otimes w,$$ $$(u+v)\otimes w = u\otimes w + v\otimes w, \text{and}$$ $$u\otimes (cv) = c(u \otimes v) = (cu) \otimes v$$ where $u,v,w$ are vectors and $c,d$ are scalars.
Taking the standard basis $e_1,e_2,e_3$ of $\mathbb R^3$, any rank $2$ tensor can then be written as a linear combination of the $9$ "pure" tensors $e_i \otimes e_j$ for $i,j = 1,2,3$. The $9$ scalars you take as coefficients in such a linear combination make up the $3 \times 3$ matrix which "represents" that rank $2$ tensor. A rank $3$ tensor, an element of the tensor product $\mathbb R^3 \otimes \mathbb R^3 \otimes \mathbb R^3$, would then consists of linear combinations of the $27$ pure tensors: $$e_i \otimes e_j \otimes e_k$$ where $i,j,k=1,2,3$. The $27$ coefficients in such a linear combination make up the $3 \times 3 \times 3$ array you mention.
Tensor multiplication is then just given by the good ol' distributive property. For instance, the product of the rank $1$ tensor $2e_1+ 3e_2$ and the rank $2$ tensor $-2(e_1 \otimes e_2) + 2(e_2 \otimes e_3)$ is: $$[2e_1 + 3e_2] \otimes [-2(e_1 \otimes e_2) + 2(e_2 \otimes e_3)]$$ $$-4(e_1 \otimes e_1 \otimes e_2)+4(e_1\otimes e_2 \otimes e_2)-6(e_2 \otimes e_1\otimes e_2)+6(e_2 \otimes e_2\otimes e_3).$$ For two rank $1$ tensors $$ae_1+be_2+ce_3 \text{ and } xe_1+ye_2+ze_3,$$ tensor multiplication gives a rank $2$ tensor whose coefficient matrix (i.e. the matrix whose entries are the coefficients of the $e_i \otimes e_j$ terms) is the product of the matrices $$\begin{pmatrix}a\\b\\c\end{pmatrix} \text{ and } \begin{pmatrix}x&y&z\end{pmatrix},$$ as you alluded to in your question. However, in general there is no simple relation between tensor multiplication and matrix multiplication.