Say we let
$$H(x)=\begin{cases} 0, & x < 0, \\ 1, & x > 0, \end{cases}$$
and let $H(0)$ be not defined.
Say I would like to approach $0$ on this function. However, a problem arises! Looking at the plot of the function, it is clear that if one were to approach from the right hand side, the limit is $1$, whilst if one approaches from the left, the limit is $0$ and thus the two-sided limit does not exist (both sides should be approaching the same number for this limit to exist)! This can also be easily seen by plugging in numbers:
$$H(1)=1$$
$$H(.1)=1$$
$$H(.000000000001)=1$$
etc. But, doing the same thing from the left hand side, we find
$$H(-1)=0$$
$$H(-.1)=0$$
$$H-(.000000000001)=0$$
Thus we need to define a different type of limit for functions with similar discontinuities so we may approach from either side. This limit is the "one-sided limit" and is used generally when a two-sided limit does not exist, like in the above case. $\lim_{x \to x_0^+}f(x)$ represents the right handed limit of $f(x)$ to $x_0$ whilst $\lim_{x \to x_0^-}f(x)$ represents the left hand limit. So we see that $\lim_{x \to 0} H(x)$ does not exist, but
$$\lim_{x \to 0^+}H(x)=1$$
$$\lim_{x \to 0^-}H(x)=0$$
Let $F'(x)=f(x).$ Then using definitions, and the FTC we can see
$$\int_{-\infty}^\infty f(x)dx= \lim_{s\to\infty}\int_a^s f(x)dx +\lim_{t\to-\infty}\int_t^a f(x)dx$$
$$=\lim_{s\to\infty}F(s)-F(a) +\lim_{t\to-\infty}F(a)-F(t)$$
$$=\lim_{s\to\infty}F(s)-\lim_{t\to-\infty}F(t).$$
If these two limits exist, then it means that the result is finite, and in particular, it is well defined, and we don't need to evaluate these limits separately. Thus
$$\lim_{s\to\infty}F(s)-\lim_{t\to-\infty}F(t)=\lim_{s\to\infty}F(s)-F(-s).$$
It is only in this situation that the principal value is equal to the result of the improper integral (equality can only be discussed because it converges, after all!).
Best Answer
Let $f(y)$ be represented by
$$f(y)=\int_{-\infty}^\infty e^{-x^2}\cos(xy)\,dx \tag1$$
Differentiating $(1)$ under the integral reveals
$$f'(y)=-\int_{-\infty}^\infty xe^{-x^2}\sin(xy)\,dx\tag2$$
Integrating by parts the integral in $(2)$ with $u=-\sin(xy)$ and $v=-\frac12e^{-x^2}$, we obtain
$$\begin{align} f'(y)&=-\frac12y\int_{-\infty}^\infty e^{-x^2}\cos(xy)\,dx\\\\\ &=-\frac12yf(y)\tag3 \end{align}$$
From $(3)$, we see that $f(y)$ satisfies the ODE $f'(y)+\frac12yf(y)=0$, subject to $f(0)=\sqrt\pi$. The solution to this ODE is trivial and is given by
$$f(y)=\sqrt\pi e^{-y^2/4}\tag4$$
Setting $y=n$ in $(4)$ yields
$$\int_{-\infty}^\infty e^{-x^2}\cos(nx)\,dx=\sqrt\pi e^{-n^2/4}$$
Letting $n\to \infty$, we find the coveted limit is $0$.