Let $\mathcal S(H)$ be the set of linear self-adjoint operators on Hilbert space $H$ with inner product $\langle\cdot,\cdot\rangle$. For $A, B\in \mathcal S(H), v=\sum_{i,j}a_{i,j}u_i\otimes u_j\in H\otimes H$ where $u_i\in H$ and $a_{i,j}\in \mathbf C$, $\forall i,j$, define
$$\langle A,B\rangle_v:=\langle v, A\otimes B v\rangle=\sum_{i,j,m,n}a_{i,j}a^*_{m,n}\langle u_i,Au_m\rangle \langle u_j,B u_n\rangle$$
where $a^*_{m,n}$ stands for the complex conjugate of $a_{m,n}$. I would like to show $\langle\cdot,\cdot\rangle_v$ is an inner product of $\mathcal S(H)$. It is trivial to show the linearity of this functional. It is the positivity $\langle A,A\rangle_v\ge0$ that gives me pause. How would one prove it? Would it be some application of the Cauchy-Schwartz inequality or a sum of squares?
Operator inner product
cauchy-schwarz-inequalityhilbert-spacesinner-productsoperator-algebrastensor-products
Best Answer
This is not true. Indeed, here is a counterexample:
$$A = \begin{pmatrix} 1 & 0 \\ 0 & -1 \end{pmatrix}, \; v = e_1 \otimes e_2, \; \langle v, A \otimes Av \rangle = -1$$
Here is a suggested definition of an inner product on $\mathcal{S}(H)$ that I’ve already mentioned in comments: Consider a vector $v \in H \otimes \overline{H}$ s.t. $v = \sum_{i, j} a_{ij}u_i \otimes \overline{u_j}$, where $\{u_i\}$ is an ONB of $H$, and $(a_{ij})_{ij}$ is the matrix of a positive operator on $H$. Then I claim that $\langle A, B \rangle_v = \langle v, A \otimes \overline{B} v \rangle$ is actually an inner product. Indeed, positivity follows from the following observation:
$$\begin{split} \langle A, A \rangle_v &= \langle v, A \otimes \overline{A} v \rangle\\ &= \sum_{i, j, m, n} a_{ij}a^\ast_{mn} \langle u_i, Au_m \rangle \langle \overline{u_j}, \overline{A}\overline{u_n} \rangle\\ &= \sum_{i, j, m, n} a_{ij}a^\ast_{mn} \langle u_i, Au_m \rangle \langle Au_n, u_j\rangle \end{split}$$
Let $e_{ij} \in B(H)$ be the operator that sends $u_j$ to $u_i$ and $u_k$ to $0$ for any $k \neq j$ (i.e., in matrix form, the matrix that has $1$ at the $(i, j)$-entry and $0$ everywhere else). Consider the operator $V = \sum_{i, j} a_{ij} e_{ij}$. As $v \in H \otimes \overline{H}$ and $\{u_j\}$ is an ONB, we have $\sum_{i, j} |a_{ij}|^2 < \infty$, so $V$ is a Hilbert-Schmidt operator. By the assumption on $(a_{ij})_{ij}$, $V$ is a positive operator. We have,
$$\begin{split} \text{Tr}(VAVA^\ast) &= \text{Tr}(VAV^\ast A^\ast)\\ &= \text{Tr}(\sum_{i, j, m, n} a_{ij}a^\ast_{mn}e_{ij}Ae_{nm}A^\ast)\\ &= \sum_{i, j, m, n} a_{ij}a^\ast_{mn} \text{Tr}(e_{ij}Ae_{nm}A^\ast)\\ &= \sum_{i, j, m, n} a_{ij}a^\ast_{mn} \langle Au_n, u_j \rangle \langle A^\ast u_i, u_m \rangle\\ &= \sum_{i, j, m, n} a_{ij}a^\ast_{mn} \langle u_i, Au_m \rangle \langle Au_n, u_j \rangle\\ &= \langle A, A \rangle_v \end{split}$$
Since $V$ is positive, so is $AVA^\ast$ and $V^{1/2}AVA^\ast V^{1/2}$. Thus, $\text{Tr}(VAVA^\ast) = \text{Tr}(V^{1/2}AVA^\ast V^{1/2}) \geq 0$, i.e., $\langle A, A \rangle_v \geq 0$.
As promised in the comments, here is a proof that, given $\{u_i\}_{i = 1}^n \subset H$ (without any assumption on linear dependency or the dimension of $H$), a positive $n \times n$-matrix $(a_{ij})$, and let $v \in H \otimes \overline{H}$ be given by $v = \sum_{i, j} a_{ij}u_i \otimes \overline{u_j}$. Then $\langle A, A \rangle_v = \langle v, A \otimes \overline{A}v \rangle \geq 0$ for any $A \in B(H)$ (not just self-adjoint operators).
We first observe that we can actually assume $H$ is finite-dimensional. Indeed, $H_0 = \text{span}\{u_i\}$ is a finite-dimensional subspace of $H$ and $v \in H_0 \otimes \overline{H_0}$. Let $P: H \to H_0$ be the orthogonal projection. Then $\langle v, A \otimes \overline{A} v \rangle = \langle v, PAP \otimes \overline{PAP} v \rangle$. If the result is proved for the finite-dimensional $H_0$, then applying the result to $PAP \in B(H_0)$, we obtain that $\langle v, A \otimes \overline{A} v \rangle = \langle v, PAP \otimes \overline{PAP} v \rangle \geq 0$. Thus, we may assume $\dim(H) = m < \infty$. We may thus identify $H$ with $\mathbb{C}^m$ and $B(H)$ with $M_m(\mathbb{C})$.
Now, recall that there is a natural isometric isomorphism $\phi: H \otimes \overline{H} \to S_2(H) \simeq M_m(\mathbb{C})$, where $S_2(H)$ is the Hilbert space of Hilbert-Schmidt operators on $H$, equipped with the Frobenius inner product. The isomorphism is linearly spanned from $$\phi(\xi \otimes \overline{\eta}) = \theta_{\xi, \eta}$$ where $\theta_{\xi, \eta} \in S_2(H)$ is the operator defined by $\theta_{\xi, \eta}(h) = \langle h, \eta \rangle \xi$.
Consider the operator $U: \mathbb{C}^m \to H$ defined by $U(e_i) = u_i$ for all $1 \leq i \leq n$. Then,
$$\phi(v) = \sum_{i, j} a_{ij} \theta_{u_i, u_j} = \sum_{i, j} a_{ij} \theta_{Ue_i, Ue_j} = \sum_{i, j} a_{ij} U\theta_{e_i, e_j}U^\ast$$
The block matrix $(U\theta_{e_i, e_j}U^\ast)_{ij} \in M_n(B(H)) = M_n(M_m(\mathbb{C}))$ is positive. Indeed,
$$(U\theta_{e_i, e_j}U^\ast)_{ij} = \begin{pmatrix} U & & & \\ & U & & \\ & & \ddots & \\ & & & U \end{pmatrix} (\theta_{e_i, e_j})_{ij} \begin{pmatrix} U & & & \\ & U & & \\ & & \ddots & \\ & & & U \end{pmatrix}^\ast$$
And $\Theta = (\theta_{e_i, e_j})_{ij} \in M_n(M_n(\mathbb{C}))$ is positive, since,
$$\begin{split} (\Theta^\ast \Theta)_{ij} &= \sum_{k = 1}^n (\Theta^\ast)_{ik}\Theta_{kj}\\ &= \sum_{k = 1}^n \theta_{e_k, e_i}^\ast \theta_{e_k, e_j}\\ &= \sum_{k = 1}^n \theta_{e_i, e_k}\theta_{e_k, e_j}\\ &= n\theta_{e_i, e_j}\\ &= n\Theta_{ij} \end{split}$$
So $\Theta = \frac{1}{n} \Theta^\ast \Theta$, whence it is positive. This concludes the proof of the claim that $\Omega = (U\theta_{e_i, e_j}U^\ast)_{ij} \in M_n(B(H)) = M_n(M_m(\mathbb{C}))$ is positive.
Let $\alpha = (a_{ij})_{ij} \in M_n(\mathbb{C})$. Let $J \in M_m(\mathbb{C})$ be the matrix that has all its entries $1$. Note that $J$ is positive - indeed, $J = \frac{1}{m} J^\ast J$. Let $\odot$ denote the Hadamard product (i.e., entrywise product, $(A \odot B)_{ij} = A_{ij}B_{ij}$). We note that $J$ is the unity for the Hadamard product. Now, a direct calculation yields, $$[(\alpha \otimes J) \odot \Omega]_{ij} = a_{ij}U\theta_{e_i, e_j}U^\ast$$ where $(\alpha \otimes J) \odot \Omega \in M_n(M_m(\mathbb{C}))$ is in block diagonal form.
Note that $\alpha$ is positive, $J$ is positive, so $\alpha \otimes J$ is positive. Thus, $(\alpha \otimes J) \odot \Omega$ is the Hadamard product of two positive matrices, whence it is positive by the Schur product theorem. (In order to make this answer more self-contained, I'll add a proof of the Schur product theorem at the end.) Now let $L \in M_n(\mathbb{C})$ be the matrix that has all its entries $1$. Again, $L$ is positive. Let $I_m \in M_m(\mathbb{C})$ be the idenity matrix. By a direct calculation, we note that, for any $T \in M_n(M_m(\mathbb{C}))$,
$$\begin{split} (\frac{1}{n}\text{Tr} \otimes \text{Id})[(L \otimes I_m)T(L \otimes I_m)] &= \frac{1}{n} \sum^n_{k, i, j = 1} I_m \cdot T_{ij} \cdot I_m\\ &= \sum_{i, j = 1}^n T_{ij} \end{split}$$
Thus,
$$\begin{split} (\frac{1}{n}\text{Tr} \otimes \text{Id})[(L \otimes I_m)((\alpha \otimes J) \odot \Omega)(L \otimes I_m)] &= \sum_{i, j = 1}^n a_{ij}U\theta_{e_i, e_j}U^\ast\\ &= \phi(v) \end{split}$$
Since $L \otimes I_m$ is positive, $(\alpha \otimes J) \odot \Omega$ is positive, and $\frac{1}{n}\text{Tr} \otimes \text{Id}$ is a positive map, we have $\phi(v)$ is positive.
Finally, let $A \in B(H)$. We first note that, for any $\xi, \eta \in H$,
$$\phi((A \otimes \overline{A})(\xi \otimes \overline{\eta})) = \phi(A\xi \otimes \overline{A\eta}) = \theta_{A\xi, A\eta} = A\theta_{\xi, \eta}A^\ast = A\phi(\xi \otimes \overline{\eta})A^\ast$$
By linearity, we have $\phi(A \otimes \overline{A}w) = A\phi(w)A^\ast$ for any $w \in H \otimes \overline{H}$. Thus,
$$\begin{split} \langle A, A \rangle_v &= \langle v, A \otimes \overline{A}v \rangle\\ &= \langle \phi(v), A\phi(v)A^\ast \rangle_{S_2(H)}\\ &= \text{Tr}(\phi(v)(A\phi(v)A^\ast)^\ast)\\ &= \text{Tr}(\phi(v)^{1/2}A\phi(v)A^\ast \phi(v)^{1/2})\\ &\geq 0 \end{split}$$
as $\phi(v)$ is positive.
Remark $1$: The assumption that $(a_{ij})$ is positive is needed. Indeed, the counterexample I provided at the start can be easily altered to a counterexample to the modified construction: Let $v = e_1 \otimes \overline{e_2} \in \mathbb{C}^2 \otimes \overline{\mathbb{C}^2}$, $A = \begin{pmatrix} 1 & 0 \\ 0 & -1 \end{pmatrix}$. Then you still have $\langle v, A \otimes \overline{A} v \rangle = -1$.
Remark $2$: By a limiting argument, the result can be extended to when $\{u_i\}_{i \in I}$ is infinite, in the sense that, if,
$$\sum_{i, j \in I_0} a_{ij}u_i \otimes \overline{u_j}$$
converges in $H \otimes \overline{H}$ as $I_0 \subset I$ finite subsets increasing to $I$ (this is also needed to ensure $v = \sum_{i, j} a_{ij}u_i \otimes \overline{u_j}$ is well-defined in the first place), and if $(a_{ij})$ is a positive matrix, in the sense that $(a_{ij})_{i, j \in I_0} \in M_{|I_0|}(\mathbb{C})$ is positive for any finite $I_0 \subset I$, then the result still holds.
Finally, here is a proof of the Schur product theorem, i.e., for any two positive matrices $A, B \in M_k(\mathbb{C})$, the Hadamard product $A \odot B$ is positive.
Consider $A \otimes B$, which is a positive operator in $M_{k^2}(\mathbb{C})$ acting on $\mathbb{C}^k \otimes \mathbb{C}^k$. Consider the subspace $H \subset \mathbb{C}^k \otimes \mathbb{C}^k$ given by,
$$H = \text{span}\{e_i \otimes e_i\}_{i = 1}^k$$
Let $P: \mathbb{C}^k \otimes \mathbb{C}^k \to H$ be the orthogonal projection. Note that $\{e_i \otimes e_i\}_{i = 1}^k$ is an ONB of $H$. We observe that, the matrix of $P(A \otimes B)P$, w.r.t. the ONB $\{e_i \otimes e_i\}_{i = 1}^k$, is,
$$\begin{split} [P(A \otimes B)P]_{ij} &= \langle P(A \otimes B)P(e_j \otimes e_j), e_i \otimes e_i \rangle\\ &= \langle (A \otimes B)(e_j \otimes e_j), e_i \otimes e_i \rangle\\ &= \langle Ae_j, e_i \rangle \langle Be_j, e_i \rangle\\ &= A_{ij}B_{ij}\\ &= (A \odot B)_{ij} \end{split}$$
Since $P(A \otimes B)P$ is a positive operator, its matrix w.r.t. an ONB is positive, whence $A \odot B$ is positive, as desired.
Per the OP’s request in chat, here is a proof that, if we allow $A$ to be non-self-adjoint, then $\langle A, A \rangle_v = \langle v, A \otimes \overline{A}v \rangle = \text{Tr}(\phi(v)A\phi(v)^\ast A^\ast) \geq 0$ for all operators $A$ iff $\phi(v) = cW$ where $c \in \mathbb{C}$ and $W$ is positive.
The proof for sufficiency is essentially contained in the answer above. But basically, if $\phi(v) = cW$ where $W$ is positive,
$$\begin{split} \text{Tr}(\phi(v)A\phi(v)^\ast A^\ast) &= c\overline{c}\text{Tr}(WAWA^\ast)\\ &= |c|^2\text{Tr}(W^{1/2}AWA^\ast W^{1/2})\\ &\geq 0 \end{split}$$
For necessity, we shall call a set of the form $e^{i\beta} \mathbb{R}_{\geq 0}$, for some $\beta \in \mathbb{R}$, a ray in the complex plane. For any $\xi, \eta \in H$, let $A = \theta_{\xi, \eta}$, then,
$$\begin{split} 0 &\leq \text{Tr}(\phi(v)A\phi(v)^\ast A^\ast)\\ &= \text{Tr}(\phi(v)\theta_{\xi, \eta}\phi(v)^\ast \theta_{\eta, \xi})\\ &= \langle \phi(v)\xi, \xi \rangle \langle \phi(v)^\ast\eta, \eta \rangle\\ &= \langle \phi(v)\xi, \xi \rangle \overline{\langle \phi(v)\eta, \eta \rangle} \end{split}$$
This means $\langle \phi(v)\xi, \xi \rangle$ and $\langle \phi(v)\eta, \eta \rangle$ belong to the same ray in the complex plane. Since $\xi$ and $\eta$ are arbitrary, we have all $\langle \phi(v)\xi, \xi \rangle$ belong to a fixed ray, that is, there exists $\beta \in \mathbb{R}$ s.t. $\langle \phi(v)\xi, \xi \rangle \in e^{i\beta} \mathbb{R}_{\geq 0}$ for all $\xi$. Whence, $\langle e^{-i\beta}\phi(v)\xi, \xi \rangle \in \mathbb{R}_{\geq 0}$ for all $\xi$, so $e^{-i\beta}\phi(v) \geq 0$. Thus, $\phi(v) = e^{i\beta}(e^{-i\beta}\phi(v))$ is of the required form.