[Math] Proof of Matrix Norm Inequality (Hadamard product)

hadamard-productinequalitylinear algebramatricesmatrix-norms

Let $◦$ be the entry-wise (Hadamard) product operator, where for two matrices
$$A = (a_{ij} )_{1≤i≤n,1≤j≤m}, B = (b_{ij} )_{1≤i≤n,1≤j≤m}∈ R^{n×m}$$
we define $$A ◦ B := (a_{ij} b_{ij} )_{1≤i≤n,1≤j≤m} $$
Show that $$ ||A ◦ B||_2 ≤ ||A||_2 · ||B||_2 $$
where 2-norm is defined as the matrix norm induced by the vector norm.

For a normal matrix product I know : $$\sup_x \frac{||ABx||}{||x||} $$ $$ ≤\sup_x \frac{||A||· ||Bx||}{||x||}$$ $$≤||A||·||B||$$

Not sure how to deal with Hadamard products. Any help appreciated

Best Answer

The easiest approach is to observe, with $\big \Vert \mathbf x_k\big \Vert_2, \big \Vert \mathbf y_k\big \Vert_2 = 1$ and using the operator 2 norm (i.e. Schatten $\infty$ norm) for our matrices

$\text{max: }\mathbf x_1^*(A \otimes B)\mathbf y_1 =\big \Vert A \otimes B\big \Vert_2 = \sigma_1^{(A)}\sigma_2^{(B)} = \big \Vert A \big \Vert_2 \big \Vert B\big \Vert_2 $
where $\otimes$ denotes Kronecker Product

Then observe that $(A \circ B)$ is a principal submatrix of $(A\otimes B)$ so
$\big \Vert A \circ B\big \Vert_2 = \text{max: }\mathbf x_2^*(A \circ B)\mathbf y_2 \leq \text{max: }\mathbf x_1^*(A \otimes B)\mathbf y_1$

The Hadamard Product is rather hard to work directly with. The Kronecker Product is quite easy to work with, so it's desirable to prove an awful lot of HP inequalities via use of the KP.