[Math] Norm with symmetric positive definite matrix

Question

If B is $n \times n$ real symmetric positive definite matrix, then $(x,y) = x^T By$ defines an inner product on $R^n$. How do you prove that $\|x\|=(x^T B x)^{1/2}$ is a norm on $R^n$?

Answer 1

Any real inner product $(\cdot, \cdot)$ on any real vector space $V$, of finite dimension or not, defines a norm via $\Vert x \Vert^2 = (x, x)$ or $\Vert x \Vert = (x, x)^{1/2}$; the defining properties of the norm follow directly from the inner product axioms, which as I recall may be stated as:

1.) $(x_1 + x_2, y_1) = (x_1, y_1) + (x_2, y_1)$, $(x_1, y_1 + y_2) = (x_1, y_1) + (x_1, y_2), \;\; \forall x_1, x_2, y_1, y_2 \in V$;

2.) $(x, y) = (y, x), \;\; \forall x, y \in V$;

3.) $(ax, y) = (x, ay) = a(x, y), \; \; \forall x, y \in V, a \in \Bbb R$;

4.) $(x, x) \ge 0, \;\; \forall x \in V; \;\; (x, x) = 0 \Leftrightarrow x = 0$.

A norm $\Vert \cdot \Vert$ must obey:

A.) $\Vert x \Vert \ge 0, \;\; \forall x \in V; \; \; \Vert x \Vert = 0 \Leftrightarrow x = 0$;

B.) $\Vert ax \Vert = \vert a \vert \Vert x \Vert, \;\; \forall x \in V, a \in \Bbb R$;

C.) $\Vert x + y \Vert \le \Vert x \Vert + \Vert y \Vert, \;\; \forall x, y \in V$.

It is a straightforward matter to see that (A - C) follow from (1 - 4), provided we take $\Vert x \Vert = (x, x)^{1/2}$; indeed, we have

$\Vert x \Vert = 0 \Leftrightarrow (x, x) = \Vert x \Vert^2 = 0 \Leftrightarrow x = 0, \tag{1}$

from (4); here as elsewhere $(x, x) \ge 0$ is implicitly invoked to assure us that $\Vert x \Vert$ is well defined via $\Vert x \Vert^2 = (x, x) \ge 0$. This establishes that $\Vert x \Vert$ satisfies (A). As for (B), we have

$\Vert ax \Vert^2 = (ax, ax) = a^2(x, x) = \vert a \vert^2 \Vert x \Vert^2 \tag{2}$

by (3); thus

$\Vert ax \Vert = \vert a \vert \Vert x \Vert, \tag{3}$

providing verification. As for (C),the so-called triangle inequality, it is the most complicated property of $\Vert \cdot \Vert$ to derive from (1 - 4); doing so is facilitated by making use of the Cauchy-Schwarz inequality,

$\vert (x, y) \vert \le \Vert x \Vert \Vert y \Vert, \; \; \forall x, y \in V, \tag{4}$

which itself follows from (1 - 4), though proving it here would take us too far afield. A complete demonstration may be found in this widipedia entry. Having Cauchy-Schwarz at hand, we see that

$\Vert x + y \Vert^2 = (x + y, x + y) = (x, x + y) + (y, x + y)$ $= (x, x) + (x, y) + (y, x) + (y, y) = \Vert x \Vert^2 + (x, y) + (y, x) + \Vert y \Vert^2 \tag{5}$

by repeated application of (1). Furthermore,

$(x, y) \le \vert (x, y) \vert; \; \; (y, x) \le \vert (y, x) \vert, \tag{6}$

so that

$(x, y) + (y, x) \le \vert (x, y) \vert\ + \vert (y, x) \vert, \tag{7}$

whence (5) becomes

$\Vert x + y \Vert^2 = (x + y, x + y) \le \Vert x \Vert^2 + \vert(x, y) \vert + \vert (y, x) \vert + \Vert y \Vert^2; \tag{8}$

now applying Cauchy-Schwarz,

$\Vert x + y \Vert^2 \le \Vert x \Vert^2 + \Vert x \Vert \Vert y \Vert + \Vert y \Vert \Vert x \Vert + \Vert y \Vert^2$ $= \Vert x \Vert^2 + 2\Vert x \Vert \Vert y \Vert + \Vert y \Vert^2 = (\Vert x \Vert + \Vert y \Vert)^2; \tag{9}$

(9) immediately yields

$\Vert x + y \Vert \le \Vert x \Vert + \Vert y \Vert, \tag{10}$

establishing the triangle inequality for $\Vert x \Vert = (x, x)^{1/2}$. Combining these results, we see that $\Vert x \Vert = (x, x)^{1/2}$ is in fact a norm on $V$ for any inner product $(\cdot, \cdot)$.

When $\dim V < \infty$, it is evident the map $V \times V \to \Bbb R$ taking $x, y \in V$ to $x^Ty$ is itself an inner poduct on $V$; (1 - 4) are easily verified in this case. We are given, and it is also easy to verify from (1 - 4), that $(x, y) = x^TBy$ is an inner product on $V$ for positive definite symmetric $B$. For example, for (2) we have

$(x, y) = x^TBy = (x^TBy)^T = y^TB^Tx = y^TBx = (y, x), \tag{11}$

in which the equality $x^TBy = (x^TB^Ty)^T$ holds since both sides are scalars, and $y^TB^Tx = y^TBx$ holds by virtue of the symmetry of $B$. Likewise (4) follows immediately from the positive definiteness of $B$; (1) and (3) are even easier to derive. These remarks extend to the case of $\dim V$ infinite, though it is customary to write $\langle x, y \rangle$ instead of $x^Ty$ under such circumstances, since the notion "transpose of a matrix" is intrinsically linked to the expression of vectors and operators in some basis. But either way one views such matters, being that $(x, y) = x^TBy = \langle x, By \rangle$ is a bona fide inner product on $V$, it follows from what has been done above that $\Vert x \Vert = (x, x)^{1/2} = (x^TBx)^{1/2} = \langle x, By \rangle$ is a norm on $V$. QED.

Notes: In fact, when $\dim V < \infty$ there exists a positive definite symmetric matrix $C$ such that $C^2 = B$; this is seen by noting that $B$ symmetric implies it may be diagonalized by some orthogonal matrix $O$: we have $O^TBO = \Lambda$ and thus $B = O \Lambda O^T$, with $\Lambda$ diagonal postive definite. Then there also exists a unique positive definite diagonal matrix $\Lambda^{1/2}$ with $(\Lambda^{1/2})^2 = \Lambda$. Setting $C = O \Lambda^{1/2} O^T$, we have $C^2 = O \Lambda^{1/2} O^T O \Lambda^{1/2} O^T = O \Lambda O^T = O \Lambda O^T = B$; also $C^T = (O \Lambda O^T)^T = O \Lambda O^T = C$. Thus with $\Vert x \Vert^2 = x^TBx$ we have $\Vert x \Vert^2 = x^TC^2 x = x^T C^TCx = (Cx)^T(Cx)$, and in fact $(x, y) = x^TC^2y = x^T C^TC y = (Cx)^T(Cy)$. This shows that the inner product $x^TBy$ and hence the norm $x^TBx$ may be viewed as the ordinary inner product $x^Ty$ or norm $x^Tx$ transformed under the action of the linear map $C:V \to V$. Expanding $x$ in an orthonormal eigenbasis $e_i$ of $B$ and/or $C$ (which exists since $B$ is symmetric) we may write $x = \sum x_i e_i$ whence $Cx = \sum x_i \lambda_i^{1/2} e_i$ and $x^TBy = (Cx)^T(Cy) = \sum \lambda_i^2 x_i y_i$; the effect of the transformation $C$ on $x^Ty$ is thus to re-scale each basis element $e_i$ by a factor $\lambda_i^{1/2}$. Such $C$ with $C^2 = B$, $C^T = C$ likely also exist in the case $\dim V$ infinite, though something like the holomorphic functional calculus may be required to prove it.

Hope this helps. Cheers,

and as ever,

Fiat Lux!!!