Solved – Why did statisticians define random matrices

distributionsmathematical-statisticsrandom matrixrandom variable

I studied mathematics a decade ago, so I have a math and stats background, but this question is killing me.

This question is still a bit philosophical to me. Why did statisticians develop all sort of techniques in order to work with random matrices? I mean, didn't a random vector solve the problem? If not, what is the mean of the diferent columns of a random matrix? Anderson (2003, Wiley) considers a random vector a special case of a random matrix with only one column.

I don't see the point of having random matrices (and I'm sure that's because I'm ignorant). But, bear with me. Imagine I have a model with 20 random variables. If I want to compute the joint probability function, why should I picture them as a matrix instead of a vector?

What am I missing?

ps: I'm sorry for the poorly tagged question, but there were no tags for random-matrix and I can't create one yet!

edit: changed matrix to matrices in the title

Best Answer

It depends which field you're in but, one of the big initial pushes for the study of random matrices came out of atomic physics, and was pioneered by Wigner. You can find a brief overview here. Specifically, it was the eigenvalues (which are energy levels in atomic physics) of random matrices that generated tons of interest because the correlations between eigenvalues gave insight into the emission spectrum of nuclear decay processes.

More recently, there has been a large resurgence in this field, with the advent of the Tracy-Widom distribution/s for the largest eigenvalues of random matrices, along with stunning connections to seemingly unrelated fields, such as tiling theory, statistical physics, integrable systems, KPZ phenomena, random combinatorics and even the Riemann Hypothesis. You can find some more examples here.

For more down-to-earth examples, a natural question to ask about a matrix of row vectors is what its PCA components might look like. You can get heuristic estimates for this by assuming the data comes from some distribution, and then looking at covariance matrix eigenvalues, which will be predicted from random matrix universality: regardless (within reason) of the distribution of your vectors, the limiting distribution of the eigenvalues will always approach a set of known classes. You can think of this as a kind of CLT for random matrices. See this paper for examples.

Related Solutions

Random Generation – Generating Random Matrices with Constraints on Row and Column Length

As @cardinal said in a comment:

Actually, after a little thought, I think you algorithm is exactly the Sinkhorn-Knopp algorithm with a very minor modification. Let $X$ be your original matrix and let $Y$ be a matrix of the same size such that $Y_{ij}=X^2_{ij}$. Then, your algorithm is equivalent to applying Sinkhorn-Knopp to $Y$, where at the final step you recover your desired form by taking $\hat{X}_{ij}=sgn(X_{ij})\sqrt{Y_{ij}}$. Sinkhorn-Knopp is guaranteed to converge except in quite pathological circumstances. Reading up on it should be very helpful.

...it seems that the iterative algorithm I suggested in the original question is very similar to the Sinkhorn-Knopp algorithm. Interestingly, it also seems very similar to iterative proportional fitting (IPF), which, as described on the IPF wikipedia page, is related to Newton's method and expectation maximization (all have the same limit).

These iterative methods are often applied to problems which lack a closed form solution, so I will tentatively assume that the answer to the question is negative: there is no way to achieve the desired solution without row/column iteration.

Solved – Generating random matrices with sum and maximality constraints

OK, to move these efforts along (but with some diffidence) I offer this approach: generate the diagonal elements first. Make them large constants. Generate all off-diagonal elements iid according to any (non-negative) distribution you want. Normalize rows. Check the column-max condition. Repeat if violated.

By making the initial constants sufficiently large, the expected number of repetitions can be made small.

Clearly the diagonal elements are iid, the non-diagonal elements are iid, but (of course) the two distributions differ.

Here's some code to play with.

n <- 8                                     # Matrix dimension
y <- rep(1 + 3/sqrt(n),n)                  # A large constant compared to entries in x
x <- matrix(runif(n^2), ncol=n)            # Here, uniform distributions off diagonal
x[cbind(1:n,1:n)] <- y                     # (Paste in the diagonal)
z <- t(apply(x, 1, function(u) u/sum(u)))  # Normalize the rows
which(1:n != apply(z, 2, which.max))       # Find all columns violating the conditions

(One hopes for integer(0) as the output; otherwise, indexes of columns whose maxima are not diagonal will be output.) I have experimented with n ranging from 3 through 300.

It's instructive to plot the columns:

plot(z[1,], type="n")
apply(z, 2, function(u) lines(u, col=(256*runif(1))))

Best Answer

Related Solutions

Random Generation – Generating Random Matrices with Constraints on Row and Column Length

Solved – Generating random matrices with sum and maximality constraints

Related Question