In the comments I claimed that the zero ring isn't a field for the same reason that $1$ isn't prime. Let me make this connection precise.
By the Artin-Wedderburn theorem, any semisimple commutative ring is uniquely the direct product of a finite collection of fields (with the usual definition of field). This theorem fails if you allow the zero ring to be a field, since the zero ring is the identity for the direct product: then for any field $F$ we would have $F \cong F \times 0$.
The "correct" replacement for the axiom that every nonzero element has an inverse is the axiom that there are exactly two ideals. The zero ring doesn't satisfy this axiom because it has one ideal. This has geometric meaning: roughly speaking it implies that the spectrum consists of a point, whereas the spectrum of the zero ring is empty.
Analogously, if you want every graph to be uniquely the disjoint union of connected graphs, you need to require that the empty graph is not connected. Here the "correct" replacement for the axiom that there is a path between every pair of vertices is the axiom that there is exactly one connected component.
(1). The additive and multiplicative identities are (obviously) $0$ and $1$ and are (obviously) unique. And $0\not \equiv 1\pmod p$. (That is necessary to say, because in a field the additive and multiplicative identities are not allowed to be equal).
(2). Additive inverses:
(2A). Existence: If $0< x<p$ then $0<p-x<p$ with $x+(p-x)\equiv 0.$ And $0+0\equiv 0.$
(2B). Uniqueness: If $x+y\equiv x+y'\equiv 0$ then $y-y'\equiv 0.$ but if $0\leq y<p$ and $0\leq y'<p$ then $|y-y'|<p.$ And if $p$ divides $|y-y'|$ with $0\leq |y-y'|<p$ then $y-y'=0 .$
(3). Multiplicative inverses:
(3A). Existence: For $1\leq x<p$ let $M=\min \{z:0<z\land \exists y\;(xy\equiv z \pmod p\}.$
First, by considering the case $y=1$ we have $M\leq x<p.$
Second, we will show that if $xy\equiv k$ with $2\leq k<p$ then there exists $y'$ and $k'$ with $0< k'<k$ and $xy'\equiv k'.$ From this we will conclude that if $2\leq k<p$ then $k\ne M.$
Thirdly, from the first and second parts above, we will have $M=1$, so $xy\equiv 1$ for some $y$.
To prove the second part: Suppose $xy\equiv k$ with $2\leq k<p.$ Let $m=\max \{m'>0: m'k<p\}.$ Then $mk<p\leq (m+1)k.$ But $p$ is prime and $m+1>1<k$ so we cannot have $(m+1)k=p.$ So $mk<p<(m+1)k.$
Let $y'=(m+1)y$ and $k'=(m+1)k-p.$ We have $0<k'<k'-mk=k.$ Now $$xy'=xy(m+1)\equiv k(m+1)\equiv k(m+1)-p=k'.$$
(3B). Uniqueness of multiplicative inverse: $$xy\equiv xy'\equiv 1\implies$$ $$\implies y\equiv y(1)\equiv y(xy')\equiv (yx)y'\equiv (1)y'\equiv y'.$$
Best Answer
Probably the most non-trivial step is showing the existence and uniqueness of the multiplicative inverse. For this, we apply Bezout's lemma: given $0 \le a < p$, there will be - because $a$ and $p$ must be co-prime - integers $m$ and $n$ such that $am + pn = 1$. Modulo $p$, this will say that $am = 1$, and then we may write $m = pk + m'$, where $0 \le m' < p$, to obtain $am' = 1$. So $a$ will have the multiplicative inverse $m'$.
If we also have $ak = 1$ for some other $k \in \mathbb{Z}_p$, then $a(k-m') = 0$, which would mean that either $a$ is zero, or else $k - m'$ is zero modulo $p$, in which case we must have $m' = k$.