In Spectral subspace is nontrivial iff it has a non-trivial intersection with an invariant closed subspace, the original poster sets the scene as follows:
$A$ is a bounded self-adjoint operator on a Hilbert space $\mathcal{H}$ and $W$ is a closed invariant
subspace.
In his answer (https://math.stackexchange.com/a/3628794/527829) Martin Argerami points out that
$W$ is actually reducing for $T$.
The original poster then goes on to say that there is a result that $\sigma(A|_W)\subseteq\sigma(A)$
and that $f(A|_W)=f(A)|_W$. I assume this should be interpreted as
$(f|_{\sigma(A|_W)})(A|_W)=f(A)|_W$ for bounded Borel functions $f$ on $\sigma(A)$.
My questions are:
- Is it true for a bounded normal operator A with an arbitrary closed reducing subspace $W$?
- Does the result have a name? Or a reference?
- How is the functional calculus part proved? (I can prove the spectrum part myself.)
Best Answer
Thanks to the suggestion by s.harp, I can give an answer to my questions. It is amazing how long the answer is when written out fully. In order to match my handwritten notes, I am going to change the notation. So here is the original question, written as a theorem in a different notation:
Theorem. Suppose $T$ is a bounded normal operator on the (not necessarily separable) Hilbert space $H$, and suppose $H_\alpha$ is a closed reducing subspace for $T$. That is, $TH_\alpha\subseteq H_\alpha$, and $T^*H_\alpha\subseteq H_\alpha$ (or equivalently, the second condition can be replaced with $TH_\alpha^\perp\subseteq H_\alpha^\perp$). Then
Proof. When I mention the spectral theorem for bounded normal operators, I am referring to Rudin, Functional Analysis, Second Edition, 12.21-12.24. Since $H_\alpha$ is reducing for $T$, we see that $T_\alpha$ and $(T^*)_\alpha$ are bounded operators on the Hilbert space $H_\alpha$. For $x_\alpha,y_\alpha\in H_\alpha$, \begin{equation*} (x_\alpha,(T_\alpha)^*y_\alpha)=(T_\alpha x_\alpha,y_\alpha)=(Tx_\alpha,y_\alpha) =(x_\alpha,T^*y_\alpha)=(x_\alpha,(T^*)_\alpha y_\alpha), \end{equation*} so $(T_\alpha)^*=(T^*)_\alpha$ and we will write it as $T_\alpha^*$. For $x_\alpha\in H_\alpha$, \begin{equation*} T_\alpha T_\alpha^*x_\alpha=T_\alpha T^*x_\alpha=TT^*x_\alpha =T^*Tx_\alpha=T^*T_\alpha x_\alpha=T_\alpha^*T_\alpha x_\alpha, \end{equation*} so $T_\alpha$ is normal. This proves #1.
Suppose $\lambda\in\rho(T)$. Let $S=(\lambda I-T)^{-1}\in\mathscr{B}(H)$, the bounded linear operators on $H$. Let $x\in SH_\alpha$. Let $x=x_\alpha+x_\alpha^\perp$, where $x_\alpha\in H_\alpha$ and $x_\alpha^\perp\in H_\alpha^\perp$. Then $(\lambda I-T)x_\alpha\in H_\alpha$, $(\lambda I-T)x_\alpha^\perp\in H_\alpha^\perp$, and $$(\lambda I-T)x_\alpha+(\lambda I-T)x_\alpha^\perp=(\lambda I-T)x\in H_\alpha,$$ since $x\in (\lambda I-T)^{-1}H_\alpha$. Hence $(\lambda I- T)x_\alpha^\perp=0$, so $x_\alpha^\perp=0$ since $\lambda I-T$ is one-to-one. Therefore $x=x_\alpha\in H_\alpha$, so that $SH_\alpha\subseteq H_\alpha$. Conversely, if $x_\alpha\in H_\alpha$, then $y=(\lambda I-T)x_\alpha\in H_\alpha$ and $x_\alpha=Sy$ so $SH_\alpha=H_\alpha$. Define $S_\alpha=S|_{H_\alpha}$. Then $S_\alpha$ is linear, continuous, one-to-one onto $H_\alpha$, so $S_\alpha\in\mathscr{B}(H_\alpha)$, and for all $x_\alpha\in H_\alpha$, \begin{equation*} S_\alpha(\lambda I_\alpha-T_\alpha)x_\alpha =S(\lambda I-T)x_\alpha =x_\alpha =(\lambda I-T)Sx_\alpha =(\lambda I_\alpha-T_\alpha)S_\alpha x_\alpha, \end{equation*} so $\lambda I_\alpha-T_\alpha$ has an inverse $S_\alpha\in\mathscr{B}(H_\alpha)$, whence $\lambda\in\rho(T_\alpha)$. This proves #2.
Let $P_\alpha$ be the orthogonal projection on $H_\alpha$. Then for $x\in H$, write $x=x_\alpha+x_\alpha^\perp$, where $x_\alpha\in H_\alpha$ and $x_\alpha^\perp\in H_\alpha^\perp$. Then since $TH_\alpha^\perp\subseteq H_\alpha^\perp$ and $TH_\alpha\subseteq H_\alpha$, \begin{equation*} P_\alpha Tx=P_\alpha Tx_\alpha+P_\alpha Tx_\alpha^\perp=P_\alpha Tx_\alpha =Tx_\alpha=TP_\alpha x, \end{equation*} that is, $P_\alpha$ commutes with $T$, so by the spectral theorem, $P_\alpha$ commutes with $E(e)$ for every $e\in B_T$. Therefore $E(e)H_\alpha\subseteq H_\alpha$. Let $p(\sigma(T))$ be the algebra of all complex continuous functions on $\sigma(T)$ expressible as $p(\lambda,\bar{\lambda})$, where $p(\nu,\gamma)$ is a polynomial in two complex variables with complex coefficients. Then by the Stone-Weierstrass Theorem, $p(\sigma(T))$ is dense in $C(\sigma(T))$. If $f\in C(\sigma(T))$ then $f_\alpha=f|_{\sigma(T_\alpha)}\in C(\sigma(T_\alpha))$ by #2. If $f(\lambda)=\sum_{i=1}^kc_i\lambda^{p_i}\bar{\lambda}^{q_i}\in p(\sigma(T))$, then by the spectral theorem, for $x_\alpha,y_\alpha\in H_\alpha$, $f(T)=\sum_{i=1}^kc_iT^{p_i}T^{*q_i}$ and $f_\alpha(T_\alpha)=\sum_{i=1}^kc_iT_\alpha^{p_i}T_\alpha^{*q_i}$ and \begin{equation*} \begin{split} \int_{\sigma(T_\alpha)}\!f_\alpha(\lambda)\,dE'_{x_\alpha,y_\alpha}(\lambda) &=\Biggl(\biggl(\sum_{i=1}^kc_iT_\alpha^{p_i}T_\alpha^{*q_i}\biggr)x_\alpha, y_\alpha\Biggr) =\Biggl(\biggl(\sum_{i=1}^kc_iT^{p_i}T^{*q_i}\biggr)x_\alpha,y_\alpha\Biggr)\\ &=\int_{\sigma(T)}\!f(\lambda)\,dE_{x_\alpha,y_\alpha}(\lambda) \qquad\qquad\qquad\qquad\qquad(f\in p(\sigma(T))).\quad\text{(1)} \end{split} \end{equation*} If $f\in C(\sigma(T))$, let $\{f_n\}\subseteq p(\sigma(T))$ be such that $f_n(\lambda)\to f(\lambda)$ uniformly on $\sigma(T)$ (that is, in the norm of $C(\sigma(T))$). Then $f_{n,\alpha}(\lambda)\to f_\alpha(\lambda)$ uniformly on $\sigma(T_\alpha)$. By the spectral theorem \begin{equation*} \lvert((f(T)-f_n(T))x_\alpha,y_\alpha)\rvert \leq\lvert\lvert f(T)-f_n(T)\rvert\rvert\,\lvert\lvert x_\alpha\rvert\rvert\, \lvert\lvert y_\alpha\rvert\rvert\to 0 \end{equation*} as $n\to\infty$. Similarly, $\lvert((f_\alpha(T_\alpha)-f_{n,\alpha}(T_\alpha))x_\alpha,y_\alpha)\rvert\to 0$. Therefore, by (1) and the spectral theorem, \begin{equation*} \begin{split} \int_{\sigma(T_\alpha)}\!f_\alpha(\lambda)\,dE'_{x_\alpha,y_\alpha}(\lambda) &=\lim_{n\to\infty}\int_{\sigma(T_\alpha)}\! f_{n,\alpha}(\lambda)\,dE'_{x_\alpha,y_\alpha}(\lambda) =\lim_{n\to\infty}\int_{\sigma(T)}\! f_n(\lambda)\,dE_{x_\alpha,y_\alpha}(\lambda)\\ &=\int_{\sigma(T)}\!f(\lambda)\,dE_{x_\alpha,y_\alpha}(\lambda) \qquad\qquad\qquad\qquad\qquad(f\in C(\sigma(T))).\quad\text{(2)} \end{split} \end{equation*} Let $d(\lambda,S)$ be the distance from $\lambda$ to $S\subseteq\mathbb{C}$: $d(\lambda,S)=\inf\,\{\lvert\lambda-s\rvert:s\in S\}$. $d$ is continuous as a function of $\lambda\in\mathbb{C}$. Let $e$ be a closed subset of $\sigma(T)$. For $n=1,2,\dots$, let $f_n(\lambda)=\max(0,1-nd(\lambda,e))$ for $\lambda\in\sigma(T)$. Then $\{f_n\}\subseteq C(\sigma(T))$, $0\leq f_n,f_{n,\alpha}\leq 1$, $f_n(\lambda)\to\chi_e(\lambda)$ for all $\lambda\in\sigma(T)$, and $f_{n,\alpha}(\lambda)\to\chi_{e\cap\sigma(T_\alpha)}(\lambda)$ for all $\lambda\in\sigma(T_\alpha)$. Since $\chi_{\sigma(T)}\in L^1(\lvert E_{x_\alpha,y_\alpha}\rvert)$, $\chi_{\sigma(T_\alpha)}\in L^1(\lvert E'_{x_\alpha,y_\alpha}\rvert)$, and $\lvert f_n\rvert\leq\chi_{\sigma(T)}$ and $\lvert f_{n,\alpha}\rvert\leq\chi_{\sigma(T_\alpha)}$ for $n=1,2,\dots$, we have by (2) and the Dominated Convergence Theorem for complex measures and by the spectral theorem, that \begin{equation*} \begin{split} (E'(e\cap\sigma(T_\alpha))x_\alpha,y_\alpha) &=\int_{\sigma(T_\alpha)}\!\chi_{e\cap\sigma(T_\alpha)}(\lambda)\, dE'_{x_\alpha,y_\alpha}(\lambda) =\lim_{n\to\infty}\int_{\sigma(T_\alpha)}\!f_{n,\alpha}(\lambda)\, dE'_{x_\alpha,y_\alpha}(\lambda)\\ &=\lim_{n\to\infty}\int_{\sigma(T)}\!f_n(\lambda)\, dE_{x_\alpha,y_\alpha}(\lambda) =\int_{\sigma(T)}\!\chi_e(\lambda)\, dE_{x_\alpha,y_\alpha}(\lambda)\\ &=(E(e)x_\alpha,y_\alpha)\qquad\qquad\qquad\qquad\qquad \text{($e$ closed $\subseteq\sigma(T)$).}\quad(3) \end{split} \end{equation*}
Let \begin{equation*} \mathscr{M}=\{e\in B_T: (E'(e\cap\sigma(T_\alpha))x_\alpha,y_\alpha) =(E(e)x_\alpha,y_\alpha)\text{ for all }x_\alpha,y_\alpha\in H_\alpha\}. \end{equation*} Suppose $e\in\mathscr{M}$. Then \begin{equation*} \begin{split} E'(e\cap\sigma(T_\alpha))+E'(e^c\cap\sigma(T_\alpha)) =E'(\sigma(T_\alpha))&=I_\alpha\quad\text{and}\\ E(e)+E(e^c)=E(\sigma(T))&=I, \end{split} \end{equation*} so \begin{equation*} \begin{split} (E'(e\cap\sigma(T_\alpha))x_\alpha,y_\alpha) &+(E'(e^c\cap\sigma(T_\alpha))x_\alpha,y_\alpha) =(I_\alpha x_\alpha,y_\alpha)\\ &=(I x_\alpha,y_\alpha) =(E(e)x_\alpha,y_\alpha)+(E(e^c)x_\alpha,y_\alpha), \end{split} \end{equation*} hence $$(E'(e^c\cap\sigma(T_\alpha))x_\alpha,y_\alpha)=(E(e^c)x_\alpha,y_\alpha),$$ so $e^c\in\mathscr{M}$.
If $e,e'\in\mathscr{M}$, then $E'(e'\cap\sigma(T_\alpha))x_\alpha, E(e)y_\alpha\in H_\alpha$ and \begin{equation*} \begin{split} (E'((e\cap e')\cap\sigma(T_\alpha))x_\alpha,y_\alpha) &=(E'(e\cap\sigma(T_\alpha))E'(e'\cap\sigma(T_\alpha))x_\alpha,y_\alpha)\\ &=(E(e)E'(e'\cap\sigma(T_\alpha))x_\alpha,y_\alpha) =(E'(e'\cap\sigma(T_\alpha))x_\alpha,E(e)y_\alpha)\\ &=(E(e')x_\alpha,E(e)y_\alpha) =(E(e)E(e')x_\alpha,y_\alpha) =(E(e\cap e')x_\alpha,y_\alpha), \end{split} \end{equation*} so $e\cap e'\in\mathscr{M}$.
Suppose $\{e_1,e_2,\dots\}\subseteq\mathscr{M}$ are disjoint. Then \begin{equation*} (E'((\cup_n e_n)\cap\sigma(T_\alpha))x_\alpha,y_\alpha) =\sum_{n=1}^\infty(E'(e_n\cap\sigma(T_\alpha))x_\alpha,y_\alpha) =\sum_{n=1}^\infty(E(e_n)x_\alpha,y_\alpha) =(E(\cup_n e_n)x_\alpha,y_\alpha), \end{equation*} so $$e=\bigcup_{n=1}^\infty e_n\in\mathscr{M}.$$
If $\{e_1,e_2,\dots\}$ are not disjoint, let $$e_n'=e_n\cap\biggl(\bigcup_{i=1}^{n-1}e_n'\biggr)^c\qquad(n=1,2,\dots).$$ By induction, $\{e_1',e_2',\dots\}$ are disjoint and by induction along with what has been shown so far, $\{e_1',e_2',\dots\}\subseteq\mathscr{M}$. Since $$\bigcup_{n=1}^\infty e_n=\bigcup_{n=1}^\infty e_n',$$ we have that $e=\cup_n e_n\in\mathscr{M}$, so $\mathscr{M}$ is a $\sigma$-algebra containing the closed sets of $\sigma(T)$, so $\mathscr{M}=B_T$. This proves that \begin{equation*} (E'(e\cap\sigma(T_\alpha))x_\alpha,y_\alpha)=(E(e)x_\alpha,y_\alpha) \qquad(x_\alpha,y_\alpha\in H_\alpha,\,e\in B_T).\quad(4) \end{equation*} Since (4) holds for all $y_\alpha\in H_\alpha$, we have that \begin{equation*} E'(e\cap\sigma(T_\alpha))x_\alpha=E(e)x_\alpha\qquad(x_\alpha\in H_\alpha,\, e\in B_T) \end{equation*} hence $E'(e\cap\sigma(T_\alpha))=E(e)|_{H_\alpha}\quad(e\in B_T)$, which proves #3.
Suppose $f\in\mathscr{B}_T$ and $f\geq 0$. Let $\{s_n\}$ be a sequence of simple Borel measurable maps, bounded by $f$, converging uniformly on $\sigma(T)$ to $f$. Then $\{s_{n,\alpha}\}$ converges uniformly on $\sigma(T_\alpha)$ to $f_\alpha$. Say $$s_n(\lambda)=\sum_{i=1}^{k_n} c_{n,i}\chi_{e_{n,i}}(\lambda).$$ By the spectral theorem \begin{equation*} \lvert((f(T)-s_n(T))x_\alpha,y_\alpha)\rvert \leq\lvert\lvert f(T)-s_n(T)\rvert\rvert\,\lvert\lvert x_\alpha\rvert\rvert\, \lvert\lvert y_\alpha\rvert\rvert\to 0 \end{equation*} as $n\to\infty$. Similarly, $\lvert((f_\alpha(T_\alpha)-s_{n,\alpha}(T_\alpha))x_\alpha,y_\alpha)\rvert\to 0$. By the spectral theorem and by (4), \begin{equation*} \begin{split} (f_\alpha(T_\alpha)x_\alpha,y_\alpha) &=\lim_{n\to\infty}\,(s_{n,\alpha}(T_\alpha)x_\alpha,y_\alpha) =\lim_{n\to\infty}\int_{\sigma(T_\alpha)}\! s_{n,\alpha}(\lambda)\,dE'_{x_\alpha,y_\alpha}(\lambda)\\ &=\lim_{n\to\infty}\sum_{i=1}^{k_n} c_{n,i}E'_{x_\alpha,y_\alpha}(e_{n,i}\cap\sigma(T_\alpha)) =\lim_{n\to\infty}\sum_{i=1}^{k_n} c_{n,i}E_{x_\alpha,y_\alpha}(e_{n,i})\\ &=\lim_{n\to\infty} \int_{\sigma(T)}\!s_n(\lambda)\,dE_{x_\alpha,y_\alpha}(\lambda) =\lim_{n\to\infty}\,(s_n(T)x_\alpha,y_\alpha)\\ &=(f(T)x_\alpha,y_\alpha)\qquad\qquad\qquad(x_\alpha,y_\alpha\in H_\alpha). \end{split} \end{equation*} Since $P_\alpha$ commutes with $E(e)$ for every $e\in B_T$, by the spectral theorem, $P_\alpha$ also commutes with $f(T)$ for every $f\in\mathscr{B}_T$. Therefore, $f(T)H_\alpha\subseteq H_\alpha$, so $f_\alpha(T_\alpha)x_\alpha=f(T)x_\alpha$ for all $x_\alpha\in H_\alpha$, whence $f_\alpha(T_\alpha)=f(T)|_{H_\alpha}$. If $f$ is not positive, then we can decompose it into its real and imaginary positive and negative parts, all of which are positive, and finally get $f_\alpha(T_\alpha)=f(T)|_{H_\alpha}$, which proves #4 and completes the proof of the theorem.