[Math] How to represent matrix multiplication in tensor algebra

linear algebramultilinear-algebratensor-productstensors

How can we represent matrix multiplication in tensor algebra?

Even if we assume all matrices represent contravariant tensors only, clearly matrix multiplication does not correspond to the multiplication operation of the tensor algebra (the tensor product), since the former is grade-preserving or grade-reducing, whereas the latter is always grade-increasing.

And then if we allow matrices to represent either contravariant or covariant or mixed variance tensors, then things get even more confusing.

For instance, a quadratic form then can be represented by the same matrix as the bilinear form it generates via polarization.

Seemingly we must implicitly be using the universal property relating $V \otimes V$ (tensor product) and $V \times V$ (Cartesian product). But we can define the same type of (matrix) multiplication for $V \otimes V^*, V^* \otimes V,$ or $V^* \otimes V^*$ or between elements of any two.

Thus now even the claim that matrices represent linear transformations and that matrix multiplication is the composition of linear maps seems suspect to me.

Is this just a result of the fact that linear algebra was invented before multilinear algebra/tensor analysis, and thus people were abusing notation when using matrices without realizing it but then the convention stuck? Or is there something more to this which I am missing?

Related but more abstract and slightly different question: How do we describe standard matrix multiplication using tensor products?

Relevant wikipedia articles: https://en.wikipedia.org/wiki/Outer_product#Tensor_multiplication, https://en.wikipedia.org/wiki/Kronecker_product

Best Answer

Fix a basis $\{e_1, \ldots, e_n\}$ of $V$, and consider the dual basis $\{f_1, \ldots, f_n \}$ of $V^\ast$. Then we have a basis $$\{e_1\otimes f_1,\ldots, e_i \otimes f_j, \ldots, e_n \otimes f_n\}$$ for $V \otimes V^\ast$, and the matrix $$A = (a_{ij})$$ is just a way of representing the element $$\sum_{i=1}^n \sum_{j=1}^n a_{ij} \; e_i \otimes f_j \in V \otimes V^\ast.$$

Of course an element of $V \otimes V^\ast$ gives a linear map $V \to V$ by

$$(w \otimes f)(v) := f(v) w$$

and extending by linearity. Given two such elements, we can compose the corresponding functions:

$$(w' \otimes f')(w \otimes f)(v) = (w' \otimes f')(f(v) w) = f(v) f'(w) w' = f'(w) \; (w' \otimes f)(v)$$

so composition of linear maps is given by

$$(w' \otimes f') \circ (w \otimes f) = f'(w) \; (w' \otimes f)$$

extended by linearity. If you write your elements in the $e_i \otimes f_j$ basis and apply this operation to them, you'll see that the usual definition of matrix multiplication pops right out.

Of course all the calculations with explicit tensors above can be rephrased in terms of the universal property of the tensor product if you like.

This is all assuming you want the matrix to represent an element of $V \otimes V^\ast$ rather than an element of $V \otimes V$ or $V^\ast \otimes V^\ast$. But you can work out what should happen in cases like that the same way.