It seems obvious that the more time it passes by without an earthquake happening, the more likely it is than an earthquake will happen.
As explained in the comments, to use an exponential distribution is to assume the opposite: that whatever time passed by without an earthquake happening, still as likely as ever it is than an earthquake will happen.
So, if one thinks this rate should not be constant, how to model the time $X$ until the first earthquake happens? The hypothesis is that, for every $t\geqslant0$, when $s\to0^+$,
$$
P[X\leqslant t+s\mid X\geqslant t]=r(t)s+o(s),
$$
for some function $r$. Thus, the function $\bar F:t\mapsto P[X\geqslant t]$ is such that
$$
\bar F(t+s)=\bar F(t)\cdot(1-r(t)s+o(s)),
$$
for every $t\geqslant0$, that is,
$$
\bar F'(t)=-r(t)\bar F(t).
$$
Since $\bar F(0)=1$ by definition, one gets
$$
\bar F(t)=\exp\left(-\int_0^tr(u)\,\mathrm du\right),
$$
and finally, $X$ has PDF
$$
f(t)=\exp\left(-\int_0^tr(u)\,\mathrm du\right)r(t)\,\mathbf 1_{t\geqslant0}.
$$
One sees that every density $f$ can be written like that, since one recovers the instantaneous rate $r$ through the formula
$$
r(t)=\frac{f(t)}{\bar F(t)}=\frac{f(t)}{1-F(t)}.
$$
As A.S.'s comment indicates, both distributions relate to the same kind of process (a Poisson process), but they govern different aspects: The Poisson distribution governs how many events happen in a given period of time, and the exponential distribution governs how much time elapses between consecutive events.
By way of analogy, suppose that we have a different process, in which events occur exactly every $10$ seconds. Then the number of events that happen in a minute (i.e., $60$ seconds) is deterministically $6$, and the amount of time that elapses between consecutive events is, of course, deterministically $10$ seconds.
In contrast, in a Poisson process with a mean rate of one event every $10$ seconds (i.e., $\lambda = 1/10$), the number of events that happen in a minute is not deterministically $6$, but it has a mean of $6$. The exact distribution is given by the Poisson distribution:
$$
P_k(t) = \frac{(\lambda t)^k}{k!} e^{-\lambda t}
$$
where $t = 60$ seconds is the time window. Thus, the probability that no events occur in a minute is given by
$$
P_0(t) = \frac{(6)^0}{0!} e^{-6} = e^{-6} \doteq 0.0024788
$$
whereas the probability that $6$ events occur in a minute is given by
$$
P_6(t) = \frac{(6)^6}{6!} e^{-6} = \frac{46656}{720} e^{-6} \doteq 0.16062
$$
That is obviously much more likely, as you would expect.
Similarly, the time between events is also not deterministically $10$ seconds, but it has a mean of $10$ seconds. The actual time distribution is the exponential distribution, which can be specified using its CDF (cumulative distribution function)
$$
F_T(t) \equiv P(T < t) = 1-e^{-\lambda t}
$$
The CDF provides essentially the same information as the PDF (probability density function), whose formulation you gave in your question; in fact, the derivative of the CDF is the PDF. However, the CDF is sometimes easier to understand intuitively, so I'll explain using the CDF here.
In this case, the probability that the time between events is less than $10$ seconds is $F_T(10) = 1-e^{-1} \doteq 0.63212$, whereas the probability that the time between events is less than $60$ seconds is $F_T(60) = 1-e^{-6} \doteq 0.99752$. The probability that it is greater than $60$ seconds is $1-F_T(60) = e^{-6} \doteq 0.0024788$, and you'll notice this is equal to the probability that no events occur in a given minute. This is no coincidence; in a Poisson process, which is memoryless, the probability that the time between events is greater than a minute is naturally equal to the probability that no events occur in that minute!
Best Answer
Yes, more precisely, if the number of arrivals in an interval $t$ is Poisson $Po(\lambda)$, then the time between two consecutive arrivals is Exponential with mean $1/\lambda$.
Proof:
$$F_T(t)=P(T\leq t)=1-P(T>t)$$
Where $P(T>t)$ means: "calculate the probability that, in a size $t$ interval there are no arrivals".
In other words
$$P(T>t)=e^{-\lambda t}$$
that is
$$1-P(T>t)=1-e^{-\lambda t}$$
which is exactly the CDF of an exponential $\text{Exp}(\lambda)$