I think Alexander Duality is what you are looking for. I gather that you are a non-expert, so I will attempt to describe in fairly informal terms how Alexander duality deals with the questions that you are interested in. Consequently, I'll suppress the inevitable technicalities, since they don't enter into the very geometric situations that you are interested in.
Alexander duality deals with the following situation. Let $S^n$ denote the $n$-dimensional sphere. Note that you can think of the $n$-sphere as being standard $n$-dimensional space $\mathbb{R}^n$ with an extra point "at infinity" added in (take a look at stereographic projection if this is unfamiliar to you). The upshot of this is that working with the $n$-sphere is not too far away from the situation you are interested in. Now take some subspace $X$, like the $m$-balls you are removing. Or take anything else; a solid torus of whatever genus you like, higher-dimensional manifolds, etc. Let $Y$ denote the complement of $X$ inside $S^n$. Then, in very informal terms, Alexander duality asserts that homologically, $Y$ is exactly as complicated as $X$. Somewhat more technically, Alexander duality asserts that for all $q$, there is an isomorphism
$$
\tilde H _q(Y) \cong \tilde H^{n-q-1}(X)
$$
between the reduced homology of $Y$ and the reduced cohomology of $X$ (for whatever coefficient group we choose). If the meaning of this is somewhat unfamiliar to you, you might be more interested to know that this says that the Betti numbers of $Y$ can be computed from those of $X$. In the range $1 \le q \le n-2$, a consequence of Alexander duality is that
$$
B_q(Y) = B_{n-q-1}(X),
$$
where $B_k(Z)$ indicates the $k^{th}$ Betti number of a space $Z$. If the piece $X$ that you are removing has $p$ components, this also implies that $B_{n-1}(Y) = p-1$.
As I remarked before, Alexander duality says that the topology of a space obtained by cutting a piece out is, from the point of view of homology (which encapsulates Betti numbers) exactly as complicated as the piece being removed. It is interesting to note that from the point of view of homotopy theory, this is incredibly far from being true. A basic technique in knot theory is to study a knot by studying the topology of its complement in $\mathbb R^3$ (or $S^3$). Homologically, Alexander duality says this is very boring, but from the perspective of homotopy theory, the story is very interesting indeed.
There are formal ways to define dimension, indeed, lots of them, depending on the context. As already noted in a comment, your description of why a circle is one-dimensional is not really correct though. The one-dimensionality of a circle is not a function of the fact that a circle itself can be described by a single number (such as radius), but that to describe the points on the circle takes a single number --- e.g. the usual parameterization $(\cos\theta,\sin\theta)$ of a circle requires just one number, the angle $\theta$. A sphere has two-dimensions because it takes two parameters to describe a point on the sphere (e.g. latitude and longitude). Of course, the sphere itself is again determined by one number, its radius (but that doesn't make it one-dimensional).
To make this more precise, one has to use ideas from topology, as mentioned in
Fredrik Meyer's answer. A key point is that the map we use to parameterize the circle/sphere/whatever should be continuous (in fact even more; at least in a neighbourhood of each point it should also admit a continuous inverse); the kind of bijection you indicate between points on the $(x,y)$-plane and points on the line will not be continuous).
There are other ways to define dimension that don't use parameters, but use different topological properties.
Of these, perhaps the notion of covering dimension is the most basic. To get the idea of it, think first about a ruler: we can divide it up into inch markings, and at any point there will be at most two different inch-long intervals meeting each other. Now think about a brick wall: if the brick layer did a bad job (just stacking bricks on top of one another in columns) then there will be points where four different bricks are in contact, but even if they lay the bricks properly, there will be points where three different bricks are in contact (this is what you usually see in a brick wall, when a single brick in one layer will sit on top of two bricks in the layer below); you can't avoid having points where three bricks meet. (Here I am ignoring the mortar in between the bricks.)
Now think about making a solid pyramid out of stone blocks: even if you arrange the blocks as stably as possible, you will have points where four stone blocks meet.
So: paving a line (one-dimensional) forces some points to have to paving stones meeting; paving a plane (two-dimensional) forces some points to have three paving stones meeting; paving a solid (three-dimensional) forces some points to have four paving stones meeting; you can probably see the pattern!
This is a more topological, less analytical, definition of dimension, which is
important in theoretical investigations of dimension. (For example, it underlies the Cech definition of cohomology in topology.)
There is also the notion of Hausdorff dimension, which captures the idea of dimension in terms of the amount of volume that a figure occupies. It can give non-integral values of dimension, and so is used for discussing the dimension of fractals.
Best Answer
The concept of dimension is surprisingly subtle. The mathematics invented by Georg Cantor and his contemporaries famously showed that, contrary to intuition, it is possible to specify a point in $2$-dimensional space using only a single real number. To make this precise, we need the concept of a bijective function. What Cantor's mathematics shows is that, contrary to intuition, there exist (many) bijections $$\mathbb{R} \rightarrow \mathbb{R} \times \mathbb{R}.$$
In some sense, this is saying that the set $\mathbb{R} \times \mathbb{R}$ is (rather paradoxically) "no bigger" than $\mathbb{R}$.
To make matters worse, Giuseppe Peano (born 13 years after Cantor) managed to construct a space-filling curve; a continuous function $$[0,1] \rightarrow [0,1] \times [0,1]$$ that, rather miraculously, manages to be surjective.
What this all means is that, like I said, the concept of dimension is rather subtle. One might speculate that fundamentally, this concept actually makes no sense. The good news is that, in fact, the concept of dimension does make sense. The bad news is that the definition is pretty complicated, and needs to be built up in two stages.
In the first stage, we establish the concept of dimension in linear algebra.
We need the following notions:
vector space
basis
The definition is:
Example. The dimension of the vector space $\mathbb{R}^n$ is $n$.
Note that, in full generality, the dimension of $V$ is a cardinal number, a concept invented by Cantor to tame the chaos of infinite sets. However, for the purposes of basic geometry, we can usually assume that $V$ is finite-dimensional, in which case the dimension of $V$ will always be a natural number.
In the second stage, we establish the concept of dimension in differential geometry. This part is much more complicated.
We'll need the following concepts:
It turns out that
Example. The dimension of the manifold $\mathbb{R}^n$ is $n$.
To finally answer your question, we'll need two more concepts:
Now:
This implies that each of $A,B$ and $C$ can be viewed as manifolds in their own right. Further to this, they turn out to be connected; and, hence, they have a well-defined dimension; namely, $1,2$ and $1$ (respectively). The first two of these numbers is easy to obtain; since $A$ and $B$ are open subsets of $\mathbb{R}$ and $\mathbb{R}^2$ respectively, hence they have dimension $1$ and $2$ respectively. The dimension of $C$ is a little harder to find, because its not an open subset of $\mathbb{R}^2$. But your intuition is generally pretty trustworthy when it comes to these things; if your brain tells you that the dimension of $C$ is $1$, then the dimension of $C$ is probably $1$. Perhaps this partly explains why it took mathematics so long to give the concept of dimension a proper and rigorous treatment; perhaps its because intuition alone is usually enough to get you the right answer, even if you don't really know what that answer means in a precise, technical sense.