As you realize, all rings have the idempotents $0$ and $1$, so the
question is whether they have any others.
If a commutative ring has a non-trivial idempotent, then it is
isomorphic to a product of two non-trivial rings. The same
is true for a non-commutative ring, as long as the idempotent
lies in its centre.
If $n$ is a positive integer which is not a prime power then
$\mathbb{Z}/n\mathbb{Z}$ has nontrivial idempotents.
Matrix rings tend to have lots of idempotents, but not usually in their centres.
A group algebra $KG$ for a finite group $G$ over a characteristic zero
field $K$ has the
central idempotent $|G|^{-1}\sum_{g\in G}g$ and usually others.
In a unital ring $R$ of order $p^2$ the additive order of $1$ can only be $p$ or $p^2$. In the second case, $1$ generates the additive group, so it is clear that the ring itself is isomorphic to $\mathbb Z/(p^2)$. So we concentrate in the first option.
In that case all non-zero elements have additive order $p$, so the ring is in fact a $2$-dimensional $\mathbb F_p$-algebra. Pick any element $x\in R$ which is linearly independent with $1$, so that $\{1,x\}$ is a basis. We must have $x^2=a+bx$ for some $a$, $b\in\mathbb F_p$, and then $R$ is isomorphic to $\mathbb F_p[X]/(X^2-aX-b)$.
If $X^2-aX-b$ is irreducible over $\mathbb F_p$, then $R$ is a field, and there is only one field of order $p^2$: $\mathbb F_{p^2}$. If not, then it factors as $(X-\alpha)(X-\beta)$ over $\mathbb F_p$. It is easy to see now that if $\alpha\neq\beta$ we have $R\cong\mathbb F_p\times\mathbb F_p$ and if $\alpha=\beta$ then $R\cong\mathbb F_p[X]/(X^2)$.
Interestingly, the fact that $R$ is commutative plays no role here, and there are no non-commutative rings of order $p^2$.
For fun, let us suppose now that $R$ does not have a unit, and let us see what happens.
First, suppose the addititive group is cyclic, so that there is an additive generator $x\in R$ of order $p^2$. Then there is an $n\in\{0,\dots,p^2-1\}$ such that $x^2=nx$. It is clear that the isoclass of $R$ is determined by $n$.
But there is an ambiguity, as there are other generators: any other generator of the additive group is of the form $ax$ with $a$ a unit of $\mathbb Z/(p^2)$.
If instead of $x$ we had started with $y=ax$, then as $y^2=a^2x^2=a^2nx=any$, instead of $n$ we would have found $an$. It follows that the isomorphism classes of rings of this form are parametrized by the quotient of $\mathbb Z/(p^2)$ under the action of its group of units given by left multiplication.
If I did not get this too wrong, there are three orbits: the one for $0$, the one for $1$ and the one for $p$, so there are three rings of this type.
Second, let us suppose that the additive group is not cyclic, so that all non-zero elements have order $p$, and $R$ is in fact a $\mathbb F_p$-vector space.
Suppose there is a non-zero idempotent $e\in R$. Let $\lambda:a\in R\mapsto ea\in R$ and $\rho:a\in R\mapsto ae\in R$. These are two idempotent $\mathbb F_p$-linear maps which commute. Linear algebra tells us then that there is a direct sum decomposition $$R=R_{00}\oplus R_{10}\oplus R_{01}\oplus R_{11}$$
with
\begin{align}
&R_{00}=\{x\in R:ex=xe=0\},\\
&R_{10}=\{x\in R:ex=x, xe=0\},\\
&R_{01}=\{x\in R:ex=0, xe=x\}, \\
&R_{11}=\{x\in R:ex=xe=x\}.
\end{align}
At most two of these subspaces can be non-zero, and we know that $R_{11}$ contains $e$. Also, $R_{11}$ cannot be all of $R$ because we are supposing there is no unit.
If there is a non-zero element in $R_{10}$, call it $x$. Then $\{e,x\}$ is a basis, $ex=x$, $xe=0$ and $xx=x(ex)=(xe)x=0x=0$. The multiplication is therefore completely determined.
Similarly, if there is a non-zero element $x\in R_{01}$, $\{e,x\}$ is again a basis, and we have $ex=0$, $xe=x$ and $xx=0$.
Finally, if there is a non-zero element $x\in R_{00}$, we have $ex=xe=0$. Then $ex^2=(ex)x=0$ and similarly $x^2e=0$, so $x^2$ is too in $R_{00}$, and there is a scalar $a$ such that $x^2=ax$. If $a\neq0$, we let $y=a^{-1}x$, so that $y^2=a^{-2}x^2=a^{-1}x=y$, and since $\{e,y\}$ is a basis this determines the multiplication. If $a=0$, of course the multiplication is also fixed.
We are left with the case in which there are no non-zero idempotents in $R$. A classical theorem of Albert implies that all non-zero elements must be nilpotent. If there is an $x\in R$ which is non-zero and such that $x^2$ is linearly independent with $x$, then $\{x,x^2\}$ is a basis, and we must have $x^3=0$, for the map $a\in R\mapsto xa\in R$, being a nilpotent endomorphism of a vector space of dimension $2$, must have nilpotency index at most $2$. We see that the multiplication table is completely determined here too.
Finally, suppose all non-zero elements are nilpotent but for for each $x$ of them, we have that $x^2$ is a linear multiple of $x$. All elements of $R$ must therefore square to zero. Let $\{x,y\}$ be a basis, and suppose $xy=ax+by$ and $yx=cx+dy$. Then $0=(x+y)^2=(a+c)x+(b+d)y$, so $yx=-xy$. Now call $u=xy$. If $u$ is zero, then all products are zero. If $\{x,u\}$ is a basis, then we know its the multiplication, as $x^2=u^2=xu=ux=0$. If not, $\{y,u\}$ is a basis, and again all products are zero.
Best Answer
One general source of such examples is functional analysis.
One of the easiest examples to describe is the space $C_{0}{(X)}$ of functions vanishing at infinity, where $X$ is locally compact, with pointwise addition and multiplication as operations. This ring is commutative and it is unital if and only if $X$ is compact.
Another class of examples is formed by the convolution algebra $L^{1}(G)$ of a locally compact group $G$. It is unital if and only if $G$ is discrete and it is commutative if and only if $G$ is commutative. So probably the easiest examples of this kind would be $L^{1}(\mathbb{R})$ or $L^{1}(\mathbb{S^1})$. Related but a bit more complicated are the group $C^{\ast}$-algebras.
A completely different kind of (non-commutative) example would be the algebra of compact operators of an infinite-dimensional Banach space.