I've been reading "A primer of multivariate statistics" by Richard J. Harris, page 546, which shows how to derive the Hotelling $T^2$ statistic, after seeing this related but different question (I have the degrees of freedom $n_1$ and $n_2$ in mine).
The following is left as an exercise for the student as it is outside the scope of a course I did last year:
Let $Y=a^TX$, where $a$ is a constant vector and $X$ is a sample
member. Also $\mathcal{N(a^T\mu,a^T\Sigma a)}$Show that
$\displaystyle\max_{\substack{a}}{\space}
t^2(a)=\displaystyle\max_{\substack{a}}\space\frac{n_1n_2[a^T(\bar{x}_1-\bar{x}_2)]^2}{(n_1+n_2)a^TS_Ua}=\frac{n_1n_2}{n_1+n_2}(\bar{x}_1-\bar{x}_2)^TS_U^{-1}(\bar{x}_1-\bar{x}_2)$which is known as the two-sample Hotelling $T^2$ statistic.
My notes say that the maximum is achieved for:
$a^*=S_U^{-1}(\bar{x}_1-\bar{x}_2)$
I can't find a proof for this despite much searching and it's not something I can work out on my own.
My thinking is that we differentiate with respect to $a$, set equal to zero and solve for $a$, and then substitute back into the original?
I can't see how one would differentiate that though and if that is how a proof is done?
Best Answer
Unless I am missing something, this can be seen from using Cauchy-Schwarz & Spectral Decomposition (as shown in pages 78-80 in Applied Multivariate Statistical Analysis, by Richard A. Johnson and Dean W. Wichern, 6th edition).
First Cauchy-Schwarz: For two $p \times 1$ vectors $\mathbf{b}$ and $\mathbf{d}$, $$ (\mathbf{b}' \mathbf{d})^2 \leq (\mathbf{b}' \mathbf{b})(\mathbf{d}' \mathbf{d}),$$ with equality iff $\mathbf{b} = c \, \mathbf{d}$ for some constant $c$.
If $\mathbf{B}$ is a symmetric positive definite matrix, using spectral-decomposition, define $\mathbf{B}^{1/2}$ and $\mathbf{B}^{-1/2}$ appropriately. Then, applying the C-S inequality above to the vectors $\mathbf{B}^{1/2}\mathbf{b}$ and $\mathbf{B}^{-1/2}\mathbf{d}$, one gets a generalized version: $$ (\mathbf{b}' \mathbf{d})^2 \leq (\mathbf{b}' \mathbf{B} \mathbf{b})(\mathbf{d}' \mathbf{B}^{-1}\mathbf{d}),$$ with equality iff $\mathbf{b} = c \, \mathbf{B}^{-1}\mathbf{d}$ for some constant $c$.
Now if $\mathbf{b} \neq \mathbf{0}$, and since $\mathbf{B}$ is symmetric positive definite, we can divide both sides by $\mathbf{b}' \mathbf{B} \mathbf{b}$ to get: $$ \frac{(\mathbf{b}' \mathbf{d})^2 } {\mathbf{b}' \mathbf{B} \mathbf{b}} \leq \mathbf{d}' \mathbf{B}^{-1}\mathbf{d}$$.
Appropriate replacements by corresponding vectors and matrices should give the result stated.
HTH.