I'll talk about the Laplace Transform. What I find important about the Laplace Transform in differential equations is that it makes operational calculus rigorous when dealing with DEs.
A well known engineer and mathemathician, Heaviside, considered $D = \frac{d}{dx}$ as an operator acting on $y$ to produce differential equations. He then thought about the following:
$$y' = f(t) $$
can be put as
$$Dy = f(t) $$
The if we thought about $\frac{1}{D}$ as the inverse operation of $D$ we would algebraically think about:
$$y = \frac{1}{D}f(t) $$
as the solution of our problem. But then you'd have the equivalence
$$ \int f(t) dt = \frac{1}{D}f(t) $$
So what about:
$y'+y = f(t) $
$(D+1)y = f(t) $
Our solution would now be
$$y = \frac{1}{D+1}f(t)$$
We know that if we use the integrating factor $e^t$ we have
$(e^t y)' = e^t f(t)$
Thus we finally get
$$y = e^{-t} \int {e^tf(t)dx}$$
As a consequence we might want to define
$$\frac{1}{D+1}f(t) = e^{-t} \int {e^tf(t)dt}$$
And if we think more generally, we would get
$$\frac{1}{D-m}f(t) = e^{mt} \int {e^{-mt}f(t)dt}$$
Again, what if we have a second order equation?
$$y''-y = f(t)$$
This means
$$(D^2-1)y = f(t)$$
And thus
$$ y = \frac{1}{D^2-1}f(t)$$
But what does this mean? Let's be blunt and write.
$$ \frac{1}{D^2-1} = \frac{1}{2} \left(\frac{1}{D-1}-\frac{1}{D+1}\right)$$
Thus this would mean the solution is
$$ y =\frac{1}{D^2-1}f(t) = \frac{1}{2} \left(\frac{1}{D-1}f(t)-\frac{1}{D+1}f(t)\right)$$
And from our last approach
$$ y =\frac{1}{D^2-1}f(t) = \frac{1}{2} \left(e^{t} \int {e^{-t}f(t)dt}-e^{-t} \int {e^{t}f(t)dt}\right)$$
Let's try this method with
$$y''-y = \sin t$$
Our solution would then be
$$y = \frac{1}{2}\left( {{e^t}\int {{e^{ - t}}\sin tdt} - {e^{ - t}}\int {{e^t}\sin tdt} } \right)$$
Just check this by yourself
$$\int {{e^t}\sin tdt} = \frac{1}{2}{e^t}\left( {\sin t - \cos t} \right) + {c_0}$$
$$\int {{e^{ - t}}\sin tdt} = - \frac{1}{2}{e^{ - t}}\left( {\sin t + \cos t} \right) + {c_1}$$
Then, after an algebraic manipulation you will get:
$$y = {c_1}{e^t} - {c_0}{e^{ - t}} - \frac{1}{2}\sin t$$
Which clearly satisfies the equation and $c_1$ and $c_2$ are to be determined.
This manipulations motivated a formal thoery of operational calculus that proved to be consistent by the analsis of the Laplace Transform. Note the similarities of the appereance of $$f(t) = e^t \Rightarrow F(s) = \frac{1}{s-1}$$
and the fact $e^t$ satisfies $(D-1)y = 0$. Similarily $$f(t) = e^{-t} \Rightarrow F(s) = \frac{1}{s+1}$$ and $e^{-t}$ satisfies $$(D+1)y = 0$$ And (!) you have that $$\sinh(t) = f(t) \Rightarrow F(s) = \frac{1}{s^2-1}$$ and this function satisfies $$(D^2-1)y=0$$
For more info on this check Spiegel's Applied Differential Equations (207-218), where you'll find theorems such as:
Let $\phi(D)$ be a polynomial in $D=\frac{d}{dx}$. Then
$$ \phi(D) \{ e^{at}f(t) \} = e^{at}\phi(D+a)\{f(t)\}$$
Another connection, and I guess this is the strong one is (Spiegel, p. 284)
$$\mathcal{L}^{-1}\left\{\frac{F(s)}{s-a}\right\} = e^{at} \int_0^t {e^{-au} f(u)du}$$
(Use the convolution theorem)
Rings a bell?
$$\frac{1}{D-m}f(t) = e^{mt} \int {e^{-mx}f(x)dx}$$
EDIT: The expression of $\phi(D)$ as linear factors $(D-p_1)$ can be done as long as the polynomial coefficients are constant. (This stems from the fact that $D$ isn't associative or commutative , i.e. $$(D f) g \neq D (fg)$$ and $$ D f \{\} = f' \neq f D\{\}$$ where you'd enter a function of $t$ inside $\{\}$.
I add: In the same manner the Fourier Transform makes an PDE an algebraic equation, as you state (I know almost nothing about the FT), the Laplace Transform makes an ODE an algebraic equation.
Best Answer
I don't know about circuits, but I believe you're trying to figure out the "physical meaning" of the Laplace transform. If you have any experience with the Fourier transform, you'd know that it analyzes the signal in terms of sinosoids, while the Laplace transform analyzes the signal in terms of sinousoids and exponentials. Traveling along a vertical line in the s-plane reveal frequency content of the signal weighted by exponential function with exponent defined by the constant real axe value. Traveling along a horisontal line reveal the exponential content of the signal weighted by sinousoids function with frequency defined by the constant imaginary axe value. In particular, traveling along the imaginaly axe reveal frequency content of the signal weighted by 1 which is equivalent to the Fourier transform of the signal. In general the laplace variable $s=\sigma +j\omega$ with $σ=0$ is equal to the Fourier transform of the signal. I believe that this textbook might be helpful: http://www.amazon.com/Circuits-Signals-Systems-William-Siebert/dp/0262192292