Let $A \odot B$ denote the Hadamard or entry-wise product of two matrices with equivalent dimensions. In this post ( Hadamard product: Optimal bound on operator norm ) it is claimed without proof that if $A$ is positive-definite, then $$\|A \odot B\|_2 \leq \max_{i, j}|A_{i,j}| \|B\|_2$$ where $\| \cdot \|_2$ is the matrix operator norm induced from the Euclidean norm. Does anyone know how to prove this or have a reference? To me it does not seem so simple.
Matrices – Hadamard Product Matrix Norm Inequality
matrices
Related Solutions
A proof in linear algebra. I hope you're familiar with SVD.
Lemma 1 For any matrix $A$, $|tr(A)|\le \sum_i \sigma_i(A)$
Proof: By SVD decomposition, and properties of the trace function $$tr(A) = tr(U\Sigma V) = tr(\Sigma VU) $$ If $Z=VU$ then it is still an unitary matrix, and $$|tr(\Sigma Z)| = |\sum_i \sigma_i(A)z_{ii}|\le \sum_i |\sigma_i(A)z_{ii}|\le \sum_i \sigma_i(A) $$ since $|z_{ii}|\le 1$.
Lemma 2 For any matrix $A,B$, $\sigma_i(A^*B)\le \sigma_i(A)\sigma_1(B)$
Proof: Using Fischer minmax theorem, we know $$ \sigma_i(A^*B) = \max_{\dim V=i}\min_{x\in V,\,\|x\|=1}\|A^*Bx\| $$ but $$ \min_{x\in V,\,\|x\|=1} \|A^*Bx\| \le \max_{x\in V,\,\|x\|=1}\|Bx\| \min_{y\in BV,\,\|y\|=1}\|A^*y\| $$ so $$ \sigma_i(A^*B) \le \max_{\dim V=i}(\max_{x\in V,\,\|x\|=1}\|Bx\| \min_{y\in BV,\,\|y\|=1}\|A^*y\|) $$ $$ \le \max_{\dim V=i}\max_{x\in V,\,\|x\|=1}\|Bx\| \max_{\dim V=i}\min_{y\in BV,\,\|y\|=1}\|A^*y\| \le \sigma_1(B)\sigma_i(A^*) $$
The easiest approach is to observe, with $\big \Vert \mathbf x_k\big \Vert_2, \big \Vert \mathbf y_k\big \Vert_2 = 1$ and using the operator 2 norm (i.e. Schatten $\infty$ norm) for our matrices
$\text{max: }\mathbf x_1^*(A \otimes B)\mathbf y_1 =\big \Vert A \otimes B\big \Vert_2 = \sigma_1^{(A)}\sigma_2^{(B)} = \big \Vert A \big \Vert_2 \big \Vert B\big \Vert_2 $
where $\otimes$ denotes Kronecker Product
Then observe that $(A \circ B)$ is a principal submatrix of $(A\otimes B)$ so
$\big \Vert A \circ B\big \Vert_2 = \text{max: }\mathbf x_2^*(A \circ B)\mathbf y_2 \leq \text{max: }\mathbf x_1^*(A \otimes B)\mathbf y_1$
The Hadamard Product is rather hard to work directly with. The Kronecker Product is quite easy to work with, so it's desirable to prove an awful lot of HP inequalities via use of the KP.
Best Answer
See (3.7.9) of Charles R. Johnson, Matrix Theory and Applications, American Mathematical Society, 1990, p.113. Essentially, for any two square matrices $A$ and $B$, if $A=X^\ast Y$ for some (probably rectangular) matrices $X$ and $Y$, then $$ \pmatrix{\|B\|_2(I\circ X^\ast X)&A\circ B\\ (A\circ B)^\ast&\|B\|_2(I\circ Y^\ast Y)} =\pmatrix{X^\ast X&A\\ A^\ast&Y^\ast Y}\circ\pmatrix{\|B\|_2I&B\\ B^\ast&\|B\|_2I} $$ is positive semidefinite, by Schur product theorem. Consequently, $$ A\circ B=\|B\|_2(I\circ X^\ast X)^{1/2}\,C\,(I\circ Y^\ast Y)^{1/2} $$ for some matrix $C$ with $\|C\|_2\le1$ and in turn, $$ \|A\circ B\|_2\le\sqrt{\|I\circ(X^\ast X)\|_2\|I\circ(Y^\ast Y)\|_2}\,\|B\|_2. $$ In your case, as $A$ is positive definite, if you put $X=Y=A^{1/2}$, you will obtain $$ \|A\circ B\|_2\le\|I\circ A\|_2\|B\|_2=\max_ia_{ii}\|B\|_2=\max_{i,j}|a_{ij}|\|B\|_2. $$