Suppose $t>0$. (If $t \leq 0$, we can keep pushing the contour to the right and make the integral as small as we like, so the inverse transform gives $0$ there as it should). We can then change variables immediately to get the $t$-dependence out of the way: if $z=st$, the integral becomes
$$ t^p \frac{\Gamma(p+1)}{2\pi i} \int_{c-i\infty}^{c+i\infty} \frac{e^{z}}{z^{p+1}} \, dz. $$
It remains to evaluate the constant $ \frac{\Gamma(p+1)}{2\pi i} \int_{c-i\infty}^{c+i\infty} \frac{e^{z}}{z^{p+1}} \, dz$. We sweep the contour around to wrap around the branch point at $s=0$ (the integrals over the quarter-circles vanish in the limit by Jordan's lemma). The integral becomes
$$ \frac{\Gamma(p+1)}{2\pi i} \int_{\gamma} \frac{e^{z}}{z^{p+1}} \, dz, $$
where $\gamma$ is composed of a line just below the imaginary axis starting at $-\infty$ and ending at $-\varepsilon$, a circle of radius $\varepsilon$ around the branch point at $z=0$, and the line from $-\varepsilon$ back to $-\infty$ just above the imaginary axis. We could try and calculate this directly, but we'll be better off integrating by parts a few times to make the singularity at the branch point better behaved, so we can just take the limits directly. Also, integration by parts is easy with this new contour, because the exponential decays and causes the endpoint terms to vanish. We have
$$ \frac{\Gamma(p+1)}{2\pi i} \int_{\gamma} z^{-p-1}e^{z} \, dz = - \frac{\Gamma(p+1)}{2\pi i \,p} \int_{\gamma} -z^{1-p-1}e^{z} \, dz = \frac{\Gamma(p+1-1)}{2\pi i} \int_{\gamma} z^{1-p-1}e^z \, dz \\
= \dotsb = \frac{\Gamma(p+1-n)}{2\pi i} \int_{\gamma} z^{n-p-1} e^{z} \, dz $$
If $n-p>0$, the integrand no longer diverges at $z=0$ (but still has a branch point), so the integral over the circle vanishes as $\varepsilon \to 0$.
Therefore,
$$ \int_{\gamma} z^{n-p-1} e^{z} \, dz = \int_{-\infty}^0 z^{n-p-1} e^{z} \, dz + \int_0^{-\infty} z^{n-p-1} e^{z} \, dz $$
Changing variables to $z=xe^{-i\pi}$ in the first and $z=xe^{i\pi}$ in the second gives
$$ \int_{-\infty}^0 z^{n-p-1} e^{z} \, dz + \int_0^{-\infty} z^{n-p-1} e^{z} \, dz = -\int_0^{\infty} e^{-i\pi(n-p)} x^{n-p-1} e^{-x} \, dx + \int_0^{\infty} e^{i\pi(n-p)} x^{n-p-1} e^{-x} \, dx \\
= 2i\sin{(n-p)\pi} \int_0^{\infty} x^{n-p-1} e^{-x} \, dx = 2i \Gamma(n-p)\sin{(n-p)\pi}. $$
Thus the whole lot evaluates to
$$ t^p \frac{\Gamma(p+1-n)}{2\pi i} 2i \Gamma(n-p)\sin{(n-p)\pi} = \frac{\sin{(n-p)\pi}}{\pi} \Gamma(n-p)\Gamma(p+1-n) $$
But we have the reflection formula
$$ \Gamma(z)\Gamma(1-z) = \frac{\pi}{\sin{\pi z}}, $$
so the constant $ \frac{\sin{(n-p)\pi}}{\pi} \Gamma(n-p)\Gamma(p+1-n) $ is actually $1$. Therefore the inverse Laplace transform gives $t^p$, as it should.
In a very hand wavy way, what is the only number between $0^+$ and $0^-$? $0$, of course. So the only number that gets plugged in for $t$ is $0$. However, this is a terrible way to prove the Initial Value Theorem. Here is a better way. We already have that
$$\int_0^\infty f'(t) e^{-st}dt = sF(s) -f(0)$$
Then taking $s\to\infty$, and assuming $f'(t)$ was bounded, we get that
$$\int_0^\infty f'(t)\cdot 0 dt = 0 = \lim_{s\to\infty} sF(s) -f(0)$$
Best Answer
Let $g(t)=\int_0^{t}f(s)ds$. Integrate by parts to see that $s \hat {f} (s)=s^{2}\int_0^{\infty} (te^{-st}) (\frac {g(t)} t)dt$. Now $s^{2}\int_0^{\infty} (te^{-st}) (\frac {g(t)} t)dt=s^{2}\int_0^{\infty} (te^{-st}) (\frac {g(t)} t-d)dt+d$ because $s^{2}\int_0^{\infty} (te^{-st})dt=1$. Suppose $|\frac {g(t)} t-d| <\epsilon$ for $t \geq t_0$. Now split the integral into integrals over $(0,t_0)$ and $(t_0,\infty)$. Can you finish the proof now?
The integral form $t_0$ to $\infty$ is bounded by $\epsilon \int_{t_0} ^\infty s^{2}te^{-st}dt \leq \epsilon \int_{0} ^{\infty} s^{2}te^{-st}dt=\epsilon$. The intergal from $0$ to $t_0$ tends to $0$ as $ s \to 0$ because $g(t)-td$ is bounded on this interval and $s^{2} \int_{0} ^{t_0} e^{-st} dt \to 0$ by direct calculation of the integral.