You have an example of something called a local ring, this is a ring with a unique maximal ideal. You are taking the ring $\mathbb{Z}$ and you are localizing at $(p)$. (In short, localization a ring $R$ at a prime ideal $I$ is just inverting everything that is outside of $I$. The fact that $I$ is prime tells us that $R\backslash I$ is multiplicatively closed, we write this $R_I$).
It is well known that in a local ring $R_I$ the unique maximal ideal is the ideal generated by $I$ in its localization, so in your case the unique maximal ideal will be generated by $(p)$, everything of the form $p\cdot\frac{a}{b}\in R$ (fractions whose numerator is a multiple of $p$).
You could finish your problem using the following lemma:
Lemma: If $R$ is a local ring with unique maximal ideal $I$, then $R\backslash I=R^\times$.
Proof: Let $x$ be a unit. Then $x\notin I$ because the ideal generated by $x$ is $R$ and $I$ is maximal (in particular proper). Hence $x\in R\backslash I$.
Now let $x\in R\backslash I$, consider the ideal $(x)$. Assume for sake of contradiction that $x$ is not a unit, meaning that $(x)$ is proper. Hence, $(x)$ is contained in a maximal ideal, but there is only one such an ideal, so we get $(x)\subset I$, so $x\in I$, a contradiction with the fact that $x\in R\backslash I$, so we get that $(x)$ is not proper, meaning that $x$ is a unit.
Using the above lemma you could finish your problem. You already found all the units. Hence you found $R\backslash I=R^\times=\{\frac{a}{b}: p\nmid a\}$, taking complements: $I=\{\frac{a}{b}:p\mid a\}$.
Here is a reference if you want to see why localization at a prime ideal yields a local ring.
Take $R=2\mathbb Z/8\mathbb Z$. This ring has no prime ideals and the multiplication is clearly not trivial.
Since the OP took another way, I'd like to add a generalization of the example above: take $R=d\mathbb Z/n\mathbb Z$ with $d\mid n$, and $m=n/d$. The prime ideals of $R$ are of the form $pd\mathbb Z/n\mathbb Z$ with $p$ prime, $p\mid m$ and $p\not\mid d$. Now it's easy to give examples of such rings without prime ideals.
Best Answer
(a) What are the left ideals of this ring? The right ideals? Hint: Use the left action of this ring on $\mathbb{Q}^2$ to describe the left ideals, and the right action of the ring (via the transpose) to describe the right ideals.
(b) Show that I is an ideal. Then it is a kernel of some homomorphism. What is its image?