We use the complex plane because it provides a very helpful visualization of complex numbers. The reason we can use it is because each complex number $z = a + bi$ is completely determined by the real numbers $a$ and $b$. In a way, thinking about a complex number as a point in $\mathbb{R}^2$ is just a different way of notating that number. There's no danger of confusion or ambiguity, because everyone can agree that a given point $(a,b)$ corresponds to the complex number $a + bi$, and vice versa. This is explained also in Atmos's answer.
I think it's worth mentioning that there is actually more than just this correspondence between the elements of each set, however.
We can also think about different operations that can be done with complex numbers and vectors in $\mathbb{R}^2$, and we can compare these. We know that complex numbers can be added together, as can vectors in $\mathbb{R}^2$. Complex numbers and vectors in $\mathbb{R}^2$ both add in the same way. Compare the following, for $a,b, c, d \in \mathbb{R}$
$$
(a + bi) + (c + di) = (a + c) + (b + d)i \\
(a,b) + (c,d) = (a + c, b + d)
$$
You can see that to add two complex numbers you can just think of each complex number as a point in $\mathbb{R}^2$ and add these points as you normally would. The result is the point in $\mathbb{R}^2$ corresponding to the sum of the complex numbers.
We can also scale complex numbers (multiply them by a real constant), and also scale vectors in $\mathbb{R}^2$. Again, this scaling behaves in the same way in $\mathbb{C}$ as it does in $\mathbb{R}^2$. Let $a,b,k \in \mathbb{R}$ and compare again
$$
k(a + bi) = ka + (kb)i \\
k(a,b) = (ka, kb)
$$
We see that to scale a complex number by a real constant, we can just think of the complex number as a point in $\mathbb{R}^2$, scale that point as we normally would, and the result is the point in $\mathbb{R}^2$ corresponding to the scaled complex number. If you'd like, the precise way of saying all of this is to say that $\mathbb{C}$ and $\mathbb{R}^2$ are isomorphic as $\mathbb{R}$-vector spaces. So you can be confident that the things you do with vectors (i.e. scalar multiplication, and vector addition) are the same in $\mathbb{C}$ and $\mathbb{R}^2$, so we are permitted to use this helpful visualization.
Edit: Here's an explanation of some extra behavior exhibited by $\mathbb{C}$ that is not compatible with the behavior of vectors in $\mathbb{R}^2$.
Above I did not explain the whole story of the multiplication operation. I explained how scaling by a real number works, but I did not mention what happens when you scale by an arbitrary complex number. Multiplication of complex numbers has a nice geometric interpretation when thinking of the elements of $\mathbb{C}$ as points on the plane. Each complex number $z = a + bi$ has associated to it a magnitude and an angle as measured counterclockwise from the positive $x$ axis. When multiplying complex numbers $z$ and $w$, the result is a new complex number with magnitude $|z|\cdot|w|$ and an angle that is the sum of the angles of $z$ and $w$. That is, multiplication of complex numbers corresponds to scaling and rotating on the complex plane. I explained a special case of this above when I talked about scaling by a real number. A real number has an angle of zero (it is on the $x$ axis), so multiplying by a real number only scales and does no rotation.
I mentioned that this operation is not compatible with a corresponding operation on $\mathbb{R}^2$. On one hand, this is the case because we are thinking of $\mathbb{R}^2$ as a vector space. And in a vector space, multiplication of two vectors need not be defined. So, this operation of multiplication is some sort of extra property possessed by the complex numbers that is not possessed by the vector space $\mathbb{R}^2$. This doesn't really change anything that I've already said, however. We can still think of complex numbers as points in $\mathbb{R}^2$. We just have to remember this extra rule for what happens to two points when we multiply them together.
And finally, some more detail if you still happen to be interested. We can define multiplication of elements in $\mathbb{R}^2$, and the natural way to do this is to say that for $a,b,c,d \in \mathbb{R}$
$$
(a,b) \cdot (c,d) = (ac, bc)
$$
This makes $\mathbb{R}^2$ into a ring. This way of multiplying elements, however, is not the same as the way we multiply elements in $\mathbb{C}$ because in $\mathbb{C}$ we have the relation that $i^2 = -1$, which changes things slightly. The precise way to state this difference is to say that $\mathbb{R}^2$ and $\mathbb{C}$ are not isomorphic as rings. Again though, this often doesn't matter to us, because in the situations where we want to think of $\mathbb{C}$ as $\mathbb{R}^2$ we are thinking of their compatibility as vector spaces, not as rings.
Best Answer
[begin crackpot theory]
Mathematicians have committed a great blunder by thinking that a certain imaginary number is $i$ and another is $-i$, when it's really the other way around!!!!
[end crackpot theory]
If one were to take the position above, and try to rewrite all of mathematics consistently with the theory propounded above, the result would be that nothing at all would change. Which one is called $i$ and which is called $-i$ doesn't matter.
It does in some contexts make sense to pay attention to whether the real part of a complex number is positive. A Dirichlet series $\displaystyle\sum_{n=1}^\infty\frac{a_n}{n^s}$ converges if the real part of $s$ is greater than the real part of the abscissa of convergence of the series, and diverges if the real part of $s$ is less than the abscissa of convergence.