Do all non-conservative vector fields (in 2-space) have corresponding surfaces that are periodic or discontinuous?
No. Non-conservative vector fields can be produced through many other vector potentials. By Helmholtz decomposition, a smooth vector field $F$ can be decomposition into a conservative vector field plus a rotation of some other conservative field:
$$
F = \nabla \phi + \nabla^{\perp} \psi,
$$
where $\nabla^{\perp}$ is like embedding the the 3D curl operator for scalar function in 2D:
$$
\boldsymbol{C}^{1}(\mathbb{R}^2) \hookrightarrow \boldsymbol{C}^{1}(\mathbb{R}^3),
\\
\nabla^{\perp} \psi(x,y) : = \left(\frac{\partial \psi}{\partial y},-\frac{\partial \psi}{\partial x}\right)\mapsto \left(\frac{\partial \psi}{\partial y},-\frac{\partial \psi}{\partial x},0\right) = \nabla\times (0,0,\psi).
$$
Ignoring the conservative part of $F$, we can produce all sorts of non-conservative part of $F$ in $\mathbb{R}^2$ using very "smooth" potential $\psi$, neither periodic nor discontinuous. For example: let $\psi = e^{-x^2-y^2}/2$
$$
F = \nabla^{\perp}\psi = (- y\psi, x\psi).
$$
You can easily check the field you gave is $\nabla^{\perp} xy$, a rotation of the conservative vector field $(x,y)$.
In fact, a $90^{\circ}$ degree rotation of any conservative vector field in $\mathbb{R}^2$ will make it non-conservative.
The surface corresponding to the vector field $F= (y,-x)$ is continuous but periodic, spiraling along the $z$-axis.
As joriki pointed out in the comments, the vector field generated by the spiral you gave is similar to "the gradient field of the polar angle"
$$
F = \nabla \arctan \left(\frac{y}{x}\right) = \left(\frac{-y}{x^2+y^2},\frac{x}{x^2+y^2}\right). \tag{1}
$$
If the domain contains a curve winding around the origin, then this is not conservative. Otherwise, it is conservative indeed. Like you did there, we let $z$ be parametrized so that we glue different branches of $\arg (x+iy)$ together, the gradient flow is non-conservative. For a more detailed discussion you could refer to my answer here. Roughly the summary is:
$$
\text{zero curl} + \text{simply-connectedness of the domain} \implies \text{conservative}
\\
\text{gradient} + \text{no singularities in the domain} \implies \text{conservative}
$$
Notice "curl zero" means $0$ everywhere, not like (1), if you include $\{0\}$ to make the domain simply-connected, then the curl is zero except this very point.
Lastly, to address your question again, the non-conservative field you found using that potential ("spiral") is actually a special kind among other non-conservative fields. It is a gradient, but it is not conservative (integral around a close path is non-zero).
If all you want is an inverse-square vector field in a topologically trivial region, then the Dirac string vector potential does the trick:
$$
\vec{A} = \frac{1 - \cos \theta}{r \sin \theta} \hat{\phi} = \frac{\tan (\theta/2)}{r} \hat{\phi}.
$$
It is not too hard to show that the curl of this vector field is
$$
\vec{\nabla} \times \vec{A} = \frac{1}{r^2} \hat{r}.
$$
However, $\vec{A}$ is ill-defined when $\theta = \pi$, which means that it can't be extended to all of space. As noted in my previous answer, there are topological obstructions to extending any such vector field over all of space.
Best Answer
As already noted in an old answer, the inverse of the curl operator (up to possible problems with the domain not having a star shape) can be written as $$\vec A(\vec r)= \int_0^1 [\vec B(t \vec r) \times (t\vec r) ]\, dt.$$
For concreteness, let us assume that the cylinder has radius $R$ and we use cylindrical coordinates $\vec r=(\rho \cos\theta,\rho \sin \theta, z)$. Then we have $$A_x(\vec r)= -y B_0 \int_0^1 t \mathop H(R-t\rho)\,dt$$ with $H$ the Heaviside-step function. Similarly, $$A_y(\vec r)= x B_0 \int_0^1 t \mathop H(R-t\rho)\,dt$$ and $$A_z(\vec r)=0.$$
To find an explicit form of $\vec A(\vec r)$, we need to perform the integral ($\rho= \sqrt{x^2 + y^2}$) $$\int_0^1 t \mathop H(R-t\rho)\,dt = \int_0^{\mathop{\rm min}(R/\rho,1)}\!\!\!t\,dt = \tfrac12\mathop{\rm min}(R/\rho,1)^2 .$$
Edit: Putting everything together, we have $$\vec{A}(\vec r) = \tfrac12 B_0\mathop{\rm min}(R/\rho,1)^2 \begin{pmatrix}-y\\x\\0\end{pmatrix} =\tfrac12 B_0\mathop{\rm min}\left[\frac{R^2}{x^2+y^2},1\right] \begin{pmatrix}-y\\x\\0\end{pmatrix}. $$