Why do tensors have a generic rank

algebraic-geometrytensors

There is an open dense set of constant rank tensors in $\mathbb{C}^{n_1} \otimes \mathbb{C}^{n_2} \otimes \ldots \otimes \mathbb{C}^{n_d}$. Intuitively, this means that if the entries of a hypermatrix are chosen by iid Gaussian random variables, with probability 1 the hypermatrix will all have rank $r$ for some constant $r$. It seems intuitive for there to be a generic multilinear rank, since choosing the entries of a matrix randomly and independently should almost never result in a relation between the rows or columns of a matrix. However, the existence of a generic rank seems remarkable and even counterintuitive. Is there any way to look at tensor rank so that generic rank seems intutive?

Also note that there is no generic rank in general for tensors over $\mathbb{R}$, and the generic rank over $\mathbb{C}$ is in general not the maximum possible rank, which is precisely why there are tensors with no best low rank approximations.

Best Answer

Let $V$ be the tensor product you defined and let $W = \mathbb{C}^{n_1} \times \cdots \times \mathbb{C}^{n_d}$ be the analogous Cartesian product.

The set $V_{\leq r}$ of all tensors of rank $\leq r$ is the image of the morphism

$\mathbb{C}^r \times W^r \to V$

taking a tuple of scalars and pure tensors to the corresponding linear combination. By Chevalley's Theorem (https://en.wikipedia.org/wiki/Constructible_set_(topology)), the image is a Zariski constructible set. Finite unions and complements of constructible sets are again constructible, so the set $V_r := V_{\leq r} \setminus V_{\leq r-1}$, of tensors of rank exactly $r$, is constructible.

(This is false over $\mathbb{R}$ as far as real points are concerned! For example, the morphism $\mathbb{A}^1_{\mathbb{R}} \to \mathbb{A}^1_\mathbb{R}$ given by $t \mapsto t^2$. The real points in the image are not a Zariski constructible set.)

Continuing... we can write $V$ as the union of the sets $V_r$ for all $r$. Since $V$ is irreducible, for some $r$ the closure $\overline{V_r}$ is all of $V$. Every constructible set contains a dense open subset of its closure, so this $V_r$ contains a Zariski dense open subset of $V$.

Sadly, as you pointed out, it's not the case that $V_{\leq r}$ is the closure of $V_r$, so the maximal rank need not be the generic rank. But this does show that some rank is generic.

This explains the phenomenon you described -- the other tensors are all, collectively, supported on a set of Lebesgue measure zero. So as you suspected, it doesn't really matter what random process you use to generate tensors, as long as (e.g.) it is absolutely continuous with respect to $N$-dimensional Lebesgue measure.