[Math] Are rank 3 tensors always cubes

matricestensor-ranktensors

a matrix is $A\in \mathbb{R}^{3\times 3}$. It is symmetric and contains 3 row vectors and 3 column vectors containing elements $a_{i,j}$. It looks like a square and, as long as the two dimensions are of equal order, the matrix is always a square .

a 3-rank tensor is $B\in \mathbb{R}^{3\times 3\times 3}$. Is it symmetric for its 3 dimensions? If we can call its first two ranks row and columns vectors also, what do we call the 3rd-rank vectors? and given that the matrix $A$ was a square, does this make 3-rank tensors cubes?

What is a correct way to visualize a 3-rank tensor in its entirety, element-by-element ($b_{i,j,k}$, etc) as you would visualize $A$ in its entirety by indexing its elements one by one into rows and columns? and how to make the concept of a 3-rank tensor intuitive for someone with a strictly statistical (matrix algebra) background (who doesn't look at vectors as bearing physics connotations like "direction", but are merely containers of elements and nothing more)

Best Answer

(Why are you saying that a rank-2 tensor is always symmetric? That's certainly not true. Maybe you are not referring to symmetry in the sense of $A_{ij} = A_{ji}$?)

The choice of notation to write a matrix as rows and columns, or to think of a rank-3 tensor as a cube, is entirely a choice of notation and is not required by the objects anywhere.

When thinking of a matrix as a linear transformation $U \rightarrow V$, its terms are elements of $U^* \otimes V$, where $U^*$ is the dual space to $U$. This is the usual meaning of $U_i^j$. But sometimes you find yourself with a rank-2 tensor $U \otimes V$ or $U^* \otimes V^*$, where neither or both sides are dual spaces, and I find these more natural to think of as 'columns of columns' or 'rows of rows', like this:

$$A_{ij} = \big( (A_{11}, A_{12}, A_{13}), (A_{21}, A_{22}, A_{23}), (A_{31}, A_{32}, A_{133}) \big)$$

This lets one maintain the notation that an inner product is always a contraction of a row with a column.

Anyway, a mental model for a rank-3 tensor could be a cube, or a matrix whose entries are themselves rows or columns, or a column of columns of columns. Whatever you want. Once you're dealing with $>2$ dimensions, it tends to be a lot easier to use https://en.wikipedia.org/wiki/Einstein_notation rather than trying to figure out how to write the object out as a matrix-like thing. If you really want a visualization, though, I suggest a matrix whose components are columns or rows, like this:

$$A_{ij}^k = \begin{pmatrix} \begin{pmatrix} A_{11}^1 & A_{12}^1 \end{pmatrix} & \begin{pmatrix} A_{21}^1 & A_{22}^1 \end{pmatrix} \\ \begin{pmatrix} A_{11}^2 & A_{12}^2 \end{pmatrix} & \begin{pmatrix} A_{21}^2 & A_{22}^2 \end{pmatrix} \\ \end{pmatrix}$$

Easier to write. You have to be careful when multiplying it with anything to keep track of which index is which -- $A_{ij}^k v^i \neq A_{ij}^k v^j$, as the former multiplies the outer dimension but the latter multiplies the inner one.