I've shown that for a matrix $A\in \mathbb{R}^{n\times n}$ and an arbitrary $v\in \mathbb{R}^n$, we have the inequality $$\|Av\|\geq \lvert \langle u, v\rangle\rvert\cdot \|A\|$$ where $\|\cdot\|$ is the $2$-norm and $u$ satisfies $\|u\| = 1$ and $\|Au\| = \|A\|$ (the operator norm of $A$ corresponding to the vector $2$-norm). To do this, I used the singular value decomposition. However, I was wondering if there's a different proof that relies on more general properties of Hilbert spaces (under the assumption that $u$ exists in the more general setting where $\langle \cdot, \cdot\rangle$ and $\|\cdot\|$ are the Hilbert space inner product and norm). My own short proof is below:
We first handle the case where $A = \operatorname{diag}(d_1, \ldots, d_n)$ for some $d_1\geq \cdots\geq d_n\geq 0$. Then, $u = e_1$, so $$\lvert \langle u, v\rangle\rvert^2\|A\|^2 = d_1^2v_1^2\leq \sum_{i=1}^n d_i^2v_i^2 = \|Av\|^2$$ Now, we consider general $A$, which has a singular value decomposition $A = U\Sigma V^{\mathrm{T}}$ for $U$ and $V$ orthogonal and $\Sigma$ diagonal in the above form. First, we recall that multiplication by orthogonal matrices preserves the dot product, i.e. $\langle Qx, Qy\rangle = \langle x, y\rangle$ for orthogonal $Q$. This allows us to write $$\|Av\|^2 = \langle U\Sigma V^{\mathrm{T}}v, U\Sigma V^{\mathrm{T}}v\rangle = \langle \Sigma V^{\mathrm{T}}v, \Sigma V^{\mathrm{T}}v\rangle = \|\Sigma V^{\mathrm{T}}v\|^2$$ for arbitrary $v\in \mathbb{R}$. Furthermore, as $\|Au\|^2 = \|\Sigma V^{\mathrm{T}}u\|^2$ and $\|V^{\mathrm{T}}u\|^2 = \|u\|^2 = 1$, we have that $\|A\| = \|\Sigma\|$ and that $y = V^{\mathrm{T}}u$ satisfies $\|\Sigma y\| = \|\Sigma\|$. Combining our earlier results, $$\|Av\|^2 = \|\Sigma V^{\mathrm{T}}v\|^2\geq \lvert\langle y, V^{\mathrm{T}}v\rangle\rvert^2\|\Sigma\|^2 = \lvert\langle V^{\mathrm{T}}u, V^{\mathrm{T}}v\rangle\rvert^2\|A\|^2 = \lvert\langle u, v\rangle\rvert^2\|A\|^2$$
Best Answer
An easier proof is to use the fact that for any $y\in\mathbb R^n$, if $y\perp u$, then $Ay\perp Au$. Now we can write $v=\langle u,v\rangle u+y$ for some $y\perp u$. Let $x_1=\langle u,v\rangle Au$ and $x_2=Ay$. Then we have that $x_1\perp x_2$ and $Av=x_1+x_2$. So $\|Av\|\ge\|x_1\|=|\langle u,v\rangle|\|Au\|=|\langle u,v\rangle|\|A\|$.
Hint for proof of the fact: Suppose that there is some $y\perp u$ with $\langle Ay,Au\rangle\neq0$. We may assume that $\|y\|=1$. Show that there exists some $\theta$ such that $u':=u\cos\theta+y\sin\theta$ satisfies $\|Au'\|>\|Au\|$. This will give us a contradiction.