You can prove that
$$\mathbb R^3\setminus\{(0,0,z)\mid z\in\mathbb R\}\simeq \mathbb R^2\setminus\{(0,0)\}\simeq S^1,$$
where $\simeq$ means the spaces are homotopically equivalent. Since equivalent spaces have isomorphic fundamental groups, you're done.
Addition: Proving $A:=\Bbb R^2\setminus\{(0,0)\}\simeq S^1$: First we can define $i:S^1\to A$ to be just the inclusion map. Then take $r:A\to S^1$ to be
$$r(x)=\frac x{|x|}.$$
We want to show that $i\circ r\simeq \operatorname{id}_A$ and $r\circ i\simeq \operatorname{id}_{S^1}$. But $r\circ i$ is actually already equal to the identity function, so only the first one is left.
So, define $H:A\times I\to A$ by
$$H(x,t)=(1-t)\frac x{|x|}+tx$$
(for a fixed $x$, this is a straight line between $x/|x|$ and $x$). Note that $H$ is well defined and continuous. It is also easy to see that $H_0=i\circ r$ and $H_1=\operatorname{id}_A$.
Showing that $\Bbb R^3\setminus\{(0,0,z)\mid z\in\Bbb R\}\simeq\Bbb R^2\setminus\{(0,0)\}$ can be done in a similar way. Use the maps $r'(x,y,z)=(x,y)$ and $i'(x,y)=(x,y,0)$.
If $\pi_1(U)=\langle a,b\rangle$ and $\pi_1(V)=\langle c,d\rangle$ then (for the right choice of $a,b,c,d$ :) $i_*\gamma=aba^{-1}b^{-1}$ and $j_*\gamma=cdc^{-1}d^{-1}$, so you get a presentation
$$\pi_1(X)=\langle a,b,c,d|aba^{-1}b^{-1}= cdc^{-1}d^{-1}\rangle.$$
In general, if you have presentations $\pi_1(U)=\langle A|S\rangle$ and $\pi_1(V)=\langle B|T\rangle$, you can get a presentation $\pi_1(X)=\langle A\sqcup B|S\sqcup T\sqcup R\rangle$, where the additional relations $R$ are of the form $i_*\gamma=j_*\gamma$, where $\gamma$ runs over generators of $\pi_1(U\cap V)$.
Best Answer
I think you have the figure for the Dunce Hat wrong, see above, where all the arrows have the label $a $, say. So you have one $1$-cell, giving $S^1$, and one $2$-cell attached by a map described by $a+a-a$, which gives a group with one generator $a$ and one relation $a+a-a=a$.
Your figure would give the group with generator $a$ and relation $a^3$, as said by others.
[The figure is taken from Topology and Groupoids. ]