Multivariate normal distribution [edit] The FIM for a $N$-variate
multivariate normal distribution, $X \sim N(\mu(\theta),
\Sigma(\theta))$ has a special form. Let the $K$-dimensional vector of
parameters be $\theta=\left[\begin{array}{lll}\theta_{1} & \ldots &
\theta_{K}\end{array}\right]^{\top}$ and the vector of random normal
variables be $X=\left[\begin{array}{lll}X_{1} & \ldots &
X_{N}\end{array}\right]^{\top} .$ Assume that the mean values of these
random variables are $\mu(\theta)=\left[\mu_{1}(\theta) \quad \ldots
\quad \mu_{N}(\theta)\right]^{\top}$, and let $\Sigma(\theta)$ be the
covariance matrix. Then, for $1 \leq m, n \leq K$, the $(m, n)$ entry
of the FIM is: $^{[16]}$ $$ \mathcal{I}_{m, n}=\frac{\partial
\mu^{\top}}{\partial \theta_{m}} \Sigma^{-1} \frac{\partial
\mu}{\partial \theta_{n}}+\frac{1}{2}
\operatorname{tr}\left(\Sigma^{-1} \frac{\partial \Sigma}{\partial
\theta_{m}} \Sigma^{-1} \frac{\partial \Sigma}{\partial
\theta_{n}}\right) $$ where $(\cdot)^{\top}$ denotes the transpose of
a vector, $\operatorname{tr}(\cdot)$ denotes the trace of a square
matrix, and: $$ \begin{aligned} \frac{\partial \mu}{\partial
\theta_{m}} &=\left[\begin{array}{lll} \frac{\partial
\mu_{1}}{\partial \theta_{m}} & \frac{\partial \mu_{2}}{\partial
\theta_{m}} & \cdots & \frac{\partial \mu_{N}}{\partial \theta_{m}}
\end{array}\right]^{\top} \\ \frac{\partial \Sigma}{\partial
\theta_{m}} &=\left[\begin{array}{cccc} \frac{\partial
\Sigma_{1,1}}{\partial \theta_{m}} & \frac{\partial
\Sigma_{1,2}}{\partial \theta_{m}} & \cdots & \frac{\partial
\Sigma_{1, N}}{\partial \theta_{m}} \\ \frac{\partial
\Sigma_{2,1}}{\partial \theta_{m}} & \frac{\partial
\Sigma_{2,2}}{\partial \theta_{m}} & \cdots & \frac{\partial
\Sigma_{2, N}}{\partial \theta_{m}} \\ \vdots & \vdots & \ddots &
\vdots \\ \frac{\partial \Sigma_{N, 1}}{\partial \theta_{m}} &
\frac{\partial \Sigma_{N, 2}}{\partial \theta_{m}} & \cdots &
\frac{\partial \Sigma_{N, N}}{\partial \theta_{m}} \end{array}\right]
\end{aligned} $$
Most importantly: What is the variable "m" in the definition of the multivariate normal Fisher Information??
This Wikipedia Definition does not make sense as the fisher information is a metric tensor induced by the hypothesis space and therefore is guaranteed to be symmetric. Writting each entry of the fisher information as $\mathcal{I}_{m,n}$ where $1\leq m, n\leq K$ suggests the Fisher Information Matrix is not symmetric. We do not even have an upper bound on "m"!
Also Is K, the # parameters equal to the N, the number of different mean variables?
What's supposed to be the dimensions of the Fisher info for a multivariate normal?
Best Answer
m is an index into the parameters.
Yes you do have an upper bound on m. The notation you show means that both m and n are bounded by K.
K is the number of parameters, which has nothing to do with the number of mean variables.
The dimension is given in the notation: it's a square matrix.