An important point is that a "pole" is actually the same thing as a removable singularity, if we think of our function as a map that takes values on the Riemann sphere (which is the complex plane with a point at $\infty$ added; the complex structure near $\infty$ comes from the map $z\mapsto 1/z$).
So a function that has a removable singularity or a pole at $z_0$ doesn't have a "real" singularity there at all; rather, we can extend the function to an analytic or meromorphic function in $z_0$. If we cannot extend the function in this way, the singularity is indeed "essential"; i.e., we cannot get rid of it. Thus the terminology is not one that is merely used for convenience or pedagogical purposes; rather, it is extremely natural.
As has already been mentioned, the magic of complex numbers results in many beautiful facts about essential singularities: functions with these singularities are very far from extending continuously.
The simplest of these facts is the Casorati-Weierstraß Theorem: The image of a neighborhood of an essential singularity is dense in the complex plane.
This is just a consequence of the removable singularities theorem. (If $f$ omitted a neighborhood of $a$, we could postcompose $f$ with a Möbius transformation that takes $a$ to infinity and see that the resulting function has a removable singularity.)
The most well-known result of this type is Picard's theorem which was already mentioned.
There are various beautiful strengthenings of Picard's theorem that arise from Nevanlinna theory, and Ahlfors's theory of covering surfaces.
So all essential singularities have some things in common, but on the other hand this should not lead us to believe that they are all the same. What they have in common is complicated behaviour, but they can be complicated in very different ways! Indeed, different transcendental entire functions (those that have an essential singularity at infinity; i.e. are not polynomials) can vary very much with respect to their behavior near infinity. Just for example, for some such functions, such as $z\mapsto e^z$, there exist curves tending to infinity on which the function is bounded, while for others this is not the case.
Note that $0$ is not an isolated singularity in 1). The classification into removable singularities, poles and essential singularities are usually only defined for isolated singularities. (If you're not convinced, what kind of singularity would $\sqrt{z}$ have at the origin?)
Best Answer
Here we have three cases: $z=0$, $e^{z^2}=1$ and singularity at infinity as $z=\infty$. The series \begin{align} f(z) &=\dfrac{1}{e^{z^2}-1}\\ &=\dfrac{1}{z^2+\frac{1}{2!}z^4+\frac{1}{3!}z^6+\cdots}\\ &=\dfrac{1}{z^2}\cdot\dfrac{1}{1+\frac{1}{2!}z^2+\frac{1}{3!}z^4+\cdots}\\ &=\dfrac{1}{z^2}\left(1-\frac{1}{2!}z^2+\frac{z^4}{12}+\cdots\right)\\ &=\dfrac{1}{z^2}-\frac{1}{2}+\frac{z^2}{12}+\cdots \end{align} shows that $z=0$ is a pole of order $2$ with residue $0$, as you found. Also the series $$f(\dfrac1z)=z^2-\dfrac12+\dfrac{1}{12z^2}+\cdots$$ shows that the infinity is an essential singularity, and in fact there is no $k$ such that $\displaystyle\lim_{z\to0}z^kf(\dfrac1z)<\infty$.