[Math] What are the Differences Between a Matrix and a Tensor

matricestensors

What is the difference between a matrix and a tensor? Or, what makes a tensor, a tensor? I know that a matrix is a table of values, right? But, a tensor?

Best Answer

Maybe to see the difference between rank 2 tensors and matrices, it is probably best to see a concrete example. Actually this is something which back then confused me very much in the linear algebra course (where we didn't learn about tensors, only about matrices).

As you may know, you can specify a linear transformation $a$ between vectors by a matrix. Let's call that matrix $A$. Now if you do a basis transformation, this can also be written as a linear transformation, so that if the vector in the old basis is $v$, the vector in the new basis is $T^{-1}v$ (where $v$ is a column vector). Now you can ask what matrix describes the transformation $a$ in the new basis. Well, it's the matrix $T^{-1}AT$.

Well, so far, so good. What I memorized back then is that under basis change a matrix transforms as $T^{-1}AT$.

But then, we learned about quadratic forms. Those are calculated using a matrix $A$ as $u^TAv$. Still, no problem, until we learned about how to do basis changes. Now, suddenly the matrix did not transform as $T^{-1}AT$, but rather as $T^TAT$. Which confused me like hell: how could one and the same object transform differently when used in different contexts?

Well, the solution is: because we are actually talking about different objects! In the first case, we are talking about a tensor that takes vectors to vectors. In the second case, we are talking about a tensor that takes two vectors into a scalar, or equivalently, which takes a vector to a covector.

Now both tensors have $n^2$ components, and therefore it is possible to write those components in a $n\times n$ matrix. And since all operations are linear resp. bilinear, the normal matrix-matrix and matrix-vector products together with transposition can be used to write the operations of the tensor. Only when looking at basis transformations, you see that both are, indeed, not the same, and the course did us (well, at least me) a disservice by not telling us that we are really looking at two different objects, and not just at two different uses of the same object, the matrix.

Indeed, speaking of a rank-2 tensor is not really accurate. The rank of a tensor has to be given by two numbers. The vector to vector mapping is given by a rank-(1,1) tensor, while the quadratic form is given by a rank-(0,2) tensor. There's also the type (2,0) which also corresponds to a matrix, but which maps two covectors to a number, and which again transforms differently.

The bottom line of this is:

  • The components of a rank-2 tensor can be written in a matrix.
  • The tensor is not that matrix, because different types of tensors can correspond to the same matrix.
  • The differences between those tensor types are uncovered by the basis transformations (hence the physicist's definition: "A tensor is what transforms like a tensor").

Of course, another difference between matrices and tensors is that matrices are by definition two-index objects, while tensors can have any rank.