Let $A \in \mathbb{R}^{n\times n}$ be a positive definite matrix, and $d_1, …, d_m$ be $A$-conjugate directions. Show that $d_1, …, d_m$ are linearly independent.
This means that
\begin{equation}
\forall \textbf{h}\in \mathbb{R}^n\backslash \{0\}, \textbf{h}^\top \textbf{A}\textbf{h}>0 \qquad \textbf{(1)}
\end{equation}
And
\begin{equation}
\forall i \in \{1, …, m\}, d_i \neq 0, \forall j \neq i, \textbf{d}_i^\top \textbf{A}\textbf{d}_j = 0 \qquad \textbf{(2)}
\end{equation}
Let us show that $\textbf{d}_i, …, \textbf{d}_m$ is linearly independent by contradiction
Let us suppose that:
\begin{equation}
\exists i^\star \in \{1,…, m\}, (\lambda_i)\in \mathbb{R}^m, d_{i^\star} = \sum_{j\neq i^\star}\lambda_j\textbf{d}_j \qquad \textbf{(3)}
\end{equation}
We can suppose that $i^\star = m$ without loss of generality and then:
\begin{align*}
\forall i \in \{1,…, m-1\}\quad \textbf{d}_i\textbf{A}\textbf{d}_m &= 0 \quad \text{by (2)} \qquad \textbf{(4)}\\
\text{However}: \textbf{d}_i\textbf{A}\textbf{d}_m &= \textbf{d}_i^\top\textbf{A}\sum_{j = 1}^{m-1}\lambda_j\textbf{d}_j \quad \text{by (3)} \qquad \textbf{(5)}\\
&= \sum_{j = 1}^{m-1}\lambda_j \textbf{d}_i^\top\textbf{A}\textbf{d}_j \qquad \textbf{(6)}\\
&= \lambda_i \underbrace{\textbf{d}_i^\top\textbf{A}\textbf{d}_i}_{>0 \text{ by (1)}} \qquad \textbf{(7)}
\end{align*}
Therefore
$$
\forall i \in \{1, …, m-1\}, \lambda = 0
$$
This means that $\textbf{d}_m = \sum 0\textbf{d}_i = 0$, contradiction
My question how can we go from (6) to (7). How can we convert $\lambda_j\textbf{d}_j$ to $\lambda_i\textbf{d}_i$?
Best Answer
It uses the $A$-conjugacy: $d_i^TAd_j=0$ for all $i\ne j$, so $$ \sum_j \lambda_j d_i^TAd_j =\lambda_id_i^TAd_i, $$ on other words: all but one summand are zero.