If the spectrum is bounded, then by means of holomorphic functional calculus we can extract a projection:
$\displaystyle P = \frac{1}{2\pi i} \intop_\gamma (\lambda - T)^{-1} d\lambda$,
where $\gamma$ encloses the spectrum. Intuitively, this should be thought of as separating the "bounded part" $TP$ (which is indeed bounded, since $T\left(\lambda-T\right)^{-1}=\lambda\left(\lambda-T\right)^{-1}-1$) and the part $T-TP$, which has empty spectrum on $\mathbb{C}$ when restricted to $\ker P$, but should be thought of as having a point at infinity, since indeed its inverse is a bounded operator with spectrum $\{0\}$.
A is selfadjoint: First, it must be shown that $A$ is a closed densely-defined selfadjoint operator on its domain $\mathcal{D}(A)$. To do this it suffices to show that
- $A$ is symmetric on its domain
- For $f \in L^{2}(\mathbb{R})$ there exist solutions $g_{\pm}\in\mathcal{D}(A)$ of $(A\pm iI)g_{\pm}=f$.
A problem has been posted with solution that shows the above conditions implie that $A$ is densely-defined and selfadjoint. ( Reference Why is this operator self-adoint )
Showing $A$ is symmetric: To show that $A$ is symmetric on its domain, suppose that $f,g\in L^{2}(\mathbb{R})$ are absolutely continuous with $f',g' \in L^{2}(\mathbb{R})$. Then
$$
\left.\int_{a}^{b}(Af)\overline{g}-f(\overline{Ag})\,dx=\int_{a}^{b}f'\overline{g}+f\overline{g}'\,dt=f\overline{g}\right|_{a}^{b}
$$
The integral on the left converges because $Af$,$f$,$Ag$,$g$ are square-integrable, and this integral converges as $b\rightarrow \infty$, $a\rightarrow -\infty$. That means that the evaluation terms on the right also have such limits. The evaluation limits must be $0$ because $f\overline{g}\in L^{1}(\mathbb{R})$. Hence,
$$
(Af,g)=(f,Ag),\;\;\; f,g \in\mathcal{D}(A).
$$
Deriving the Resolvent Operator:
Suppose $\lambda\notin\mathbb{R}$ and $f\in L^{2}(\mathbb{R})$. According to the plan, we find $g\in\mathcal{D}(A)$ such that
$$
(A-\lambda I)g = f \\
g'-i\lambda g = if \\
(e^{-i\lambda t}g)'= ie^{-i\lambda t}f
$$
Now the solution breaks into two cases: $\Im\lambda > 0$ and $\Im\lambda < 0$. If $\Im\lambda > 0$, then $e^{-i\lambda t}$ is integrable on $(-\infty,x]$. So, for this case, we choose the solution
$$
e^{-i\lambda t}g(t)= i\int_{-\infty}^{t}e^{-i\lambda u}f(u)\,du,\\
g(t) = i\int_{-\infty}^{t}e^{i\lambda(t-u)}f(u)\,du.
$$
In other words, the proposed solutions $g$ are
$$
g = R(\lambda)f = i\int_{-\infty}^{t}e^{i\lambda(t-x)}f(x)\,dx,\;\; \Im\lambda > 0,\\
g = R(\lambda)f = -i\int_{t}^{\infty}e^{i\lambda(t-x)}f(x)\,dx,\;\;\Im\lambda < 0.
$$
It can be shown that $g \in L^{2}(\mathbb{R})$ by careful application of Cauchy-Schwarz. Knowing this, it is apparent that $A$ is selfadjoint because $(A\pm iI)$ are surjective, which is enough to guarantee that $\mathcal{D}(A)$ is dense in $L^{2}(\mathbb{R})$ and $A=A^{\star}$ on this domain.
Applying Stone's Formula: Now consider
$$
R(s+i\epsilon)f-R(s-i\epsilon)f=i\int_{-\infty}^{\infty}e^{-\epsilon|t-x|}e^{is(t-x)}f(x)\,dx
$$
For convenience, suppose that $f$ is compactly supported in order to avoid convergence issues. This restriction is easily removed once the final result is obtained. Dividing by $2\pi i$ and integrating the above in $s$ over $[a,b]$ gives
$$
\frac{1}{2}\{E[a,b]+E(a,b)\}f = \lim_{\epsilon\downarrow 0}\frac{1}{2\pi}\int_{a}^{b}\int_{-\infty}^{\infty}e^{-\epsilon|t-x|}e^{-isx}f(x)\,dx\,ds.
$$
The convergence of this limit is guaranteed to take place in the norm of $L^{2}(\mathbb{R})$. It is easy to see that the spectral measure has no atoms so that $E[a,b]=E(a,b)$, and
$$
E[a,b]f = \frac{1}{2\pi}\int_{a}^{b}\left[\int_{-\infty}^{\infty}e^{-isx}f(x)\,dx\right]e^{ist}\,ds.
$$
The connection between the spectral measure for $A$ and the Fourier transform with its inverse is apparent.
Theorem ($L^{2}$ Fourier Transform): Let $f\in L^{2}(\mathbb{R})$. Then the following limit exists in $L^{2}(\mathbb{R})$
$$
\mathscr{F}f=\lim_{r\rightarrow\infty}\frac{1}{\sqrt{2\pi}}\int_{-r}^{r}f(x)e^{-isx}\,dx
$$
Furthermore, $\mathscr{F}$ is a unitary operator on $L^{2}(\mathbb{R})$ whose inverse is given by
$$
\mathscr{F}^{-1} f= \lim_{r\rightarrow\infty}\frac{1}{\sqrt{2\pi}}\int_{-r}^{r}f(s)e^{ist}\,ds.
$$
The spectral measure $E$ for the operator $A$ on an interval $[a,b]$ is given by
$$
E[a,b]f = \mathscr{F}^{-1}\chi_{[a,b]}\mathscr{F}f,
$$
where $\chi_{[a,b]}$ is the characteristic function of $[a,b]$. Therefore,
$$
Af = \mathscr{F}^{-1}M \mathscr{F}f
$$
where $M$ is the multiplication operator on $L^{2}(\mathbb{R})$ defined by $(Mf)(s)=sf(s)$. Thus, $f \in \mathcal{D}(A)$ iff $s\hat{f}(s) \in L^{2}(\mathbb{R})$ where $\hat{f}(s)=\mathscr{F}f$ is the Fourier transform of $f$. In other words, a function $f\in L^{2}$ is absolutely continuous with $L^{2}$ derivative iff $s\hat{f}(s)$ is in $L^{2}$!
Proof:
Define $\mathscr{F}$ as in the theorem. For compactly supported $f\in L^{2}$, the operator $\mathscr{F}f$ is clearly defined pointwise, and it has been shown that
$$
E[a,b]f = \frac{1}{\sqrt{2\pi}}\int_{a}^{b}(\mathscr{F}f)(s)e^{ist}\,ds.
$$
The above is guaranteed to be in $L^{2}$. In fact, because $E$ is the spectral measure for $A$,
$$
\|E[a,b]f\|^{2}=(E[a,b]f,f)=\frac{1}{\sqrt{2\pi}}\int_{a}^{b}(\mathscr{F}f)(s)\int_{-\infty}^{\infty}\overline{f(t)e^{-ist}}\,dt=\int_{a}^{b}|\mathscr{F}f|^{2}\,ds.
$$
By properties of the spectral measure, $\lim_{r\rightarrow\infty}E[-r,r]f=f$, where the convergence is in $L^{2}$. Therefore, the following limit exists and is finite:
$$
\|f\|^{2} = \int_{-\infty}^{\infty}|\mathscr{F}f|^{2}\,dx.
$$
In particular, $\mathscr{F} : L^{2}\rightarrow L^{2}$ is an isometry, which is not yet known to be unitary. This is Parseval's Equality for the $L^{2}$ Fourier transform $\mathscr{F}$. Furthermore, from the first equation of the proof, it is seen that the following limit also exists in $L^{2}$:
$$
f = \lim_{r\rightarrow\infty}E[-r,r]f = \lim_{r\rightarrow\infty}\frac{1}{\sqrt{2\pi}}\int_{-r}^{r}(\mathscr{F}f)(s)e^{ist}\,ds
$$
So this is the inverse $\mathscr{F}^{-1}$ of the Fourier transform $\mathscr{F}$. By the symmetry of these expressions, it is easy to see that the two operators invert each other in either direction. So they are inverses, which means that $\mathscr{F}$ is a unitary map from $L^{2}$ to $L^{2}$, and the spectral measure of $A$ for an interval $[a,b]$ is
$$
E[a,b]f = \mathscr{F}^{-1}\chi_{[a,b]}\mathscr{F}f.
$$
So differentiation is unitarily equivalent to the multiplication operator $Mf=sf(s)$ on $L^{2}(\mathbb{R})$. The Borel functional calculus is
$$
F(A)f = \int F(\lambda)dE(\lambda)f = \mathscr{F}^{-1}M_{F}\mathscr{F}f
$$
where $M_{F}$ is multiplication by the Borel function $F$. Part of the statement of the Spectral Theorem is that the domain of $A$ consists of the set of all $f\in L^{2}$ such that
$$
\int \lambda^{2}d\|E(\lambda)f\|^{2} = \int \lambda^{2}|\mathscr{F}f|^{2}\,d\lambda < \infty.
$$
This alone is a remarkable result: A function $f \in L^{2}$ is absolutely continuous on $\mathbb{R}$ with $f'\in L^{2}$ iff $s\hat{f}(s) \in L^{2}$, where $\hat{f}$ is the Fourier transform of $f$. $\;\;\Box$
Best Answer
I think that an easy method is delivered by functional calculus.
Consider the right-hand side of your equation as $$ \lim_{ \varepsilon \downarrow 0} \frac{1}{2\pi i}f_{\varepsilon, a, b}(A)x, $$ where $$f_{\varepsilon, a, b}(t) = \int_{a}^b \left(\frac{1}{t-(u+i\varepsilon)} - \frac{1}{t-(u-i\varepsilon)}\right) \mbox{d}u.$$
We can show that (by doing calculations) $$\lim_{ \varepsilon \downarrow 0} \frac{1}{2\pi i}\int_{a}^b \left(\frac{1}{t-(u+i\varepsilon)} - \frac{1}{t-(u-i\varepsilon)}\right) \mbox{d}u = \begin{cases} 1 & \mbox{if } t \in (a,b) \\ \frac{1}{2} &\mbox{if } t = a \\ \frac{1}{2} &\mbox{if } t = b \\ 0 &\mbox{if }t\notin [a, b] \end{cases}$$
If now you apply functional calculus to the right-hand side of above experssion you will obtain excatly $$ \frac{1}{2}\left(E[a,b] + E(a,b)\right)x.$$