As the spiral is winding around, we have four $90°$ turns before completing a full rotation. Notice that each rotation starts and ends with an odd square.
This is because, starting at $r^2-1$ and adding the four sides $4(r+1)$ to complete a rotation, gives $r^2+4r+3=(r+2)^2-1$, reaching the next odd square.
Therefore, odd squares $n=(2k+1)^2$ are mapped to $(y,x)=(-k, k+1)$.
Hence, given $n$ we start by computing $k=\frac12(\sqrt{n}-1)$ to determine in which rotation the given number is located in. That is, it is located between two odd squares $(2\lfloor k\rfloor+1)^2$ and $(2\lceil k\rceil+1)^2$ that determine the start and end of the rotation.
Now we simply backtrack from the end to the start of the rotation.
Given $n$, end of rotation is at $(-K, K+1)$, where $K=\left\lceil\frac12(\sqrt{n}-1)\right\rceil$.
Given $n$, distance from the end of the rotation is $d=(2K+1)^2 - n$.
This gives $U(n)=(y,x)$ as:
$$
U(n)=
\begin{cases}
(-K&,+K+1-d&), & 0K+0\le d\le 2K+1\\
(-3K-1+d&,-K&), & 2K+1\lt d\le 4K+1\\
(+K&,-5K-1+d&), & 4K+1\lt d\le 6K+1\\
(+7K+1-d&,+K&), & 6K+1\lt d\lt 8K+1\\
\end{cases}
$$
which is as simple as it gets.
Verifying the formula in python, gives the expected result as in the picture:
36 35 34 33 32 31 30
37 16 15 14 13 12 29
38 17 4 3 2 11 28
39 18 5 0 1 10 27
40 19 6 7 8 9 26
41 20 21 22 23 24 25
42 43 44 45 46 47 48 49
Neither of your sets is written correctly. Mathematical notation is not a shorthand for English text and therefore, different rules apply:
“$n, h \in \mathbb Z \geq 0$” does not make sense. When a statement contains several relation symbols (like $\in$ and $\geq$ in this case), the normal way to interpret this is as two separate statements like this: $n, h \in \mathbb Z$ (which makes sense) and $\mathbb Z \geq 0$ (which does not). [Technically, even something like $a < b < c$ is abuse of notation. In this case, however, it stands for $a < b$ and $b < c$ which are both sensible on their own and together they imply $a < c$, so there really isn’t any room for confusion.]
Writing “$h \to +\infty$” makes no sense. This $\to$-notation always needs a partner, i.e. “$h \to +\infty$ as $x \to 0$” (where presumably $h$ would depend on $x$ in some way). A variable cannot go off to infinity on its own, it always needs to do so in response to some other change. If you want to include a formal symbol “$\infty$” in your set, that is fine.
Writing “$h \in \mathbb Z \geq 0 \vee {}\to + \infty$” makes even less sense. I think that you intend to say that $h$ is an integer greater than or equal $0$ or goes to infinity; however, unlike in English, “or” (i.e. “$\vee$”) can only connect complete statements which “goes to infinity” is not. In English, we essentially mentally insert a copy of the subject of the sentence (here “$h$”) after the “or”, in mathematical notation you need to be explicit. [There are only very few cases where something like this is allowed, most importantly when writing $n, h \in \mathbb Z$ instead of $n \in \mathbb Z \wedge h \in \mathbb Z$.]
You seem to want to use set-builder notation which always requires you to have two parts between $\{$ and $\}$: First, the variable you want to use in your description and second (after a “$|$” or “$:$”, or rarer, “$;$” or “$,$”), the conditions the elements need to fulfill. [For set-theoretic reasons that I don’t want to go into right now, the base set your objects come from is usually placed in the first part but not doing so is generally acceptable.] It is also okay to “destructure” your elements in the first part, so if you want to have a set of pairs, you can write $\{ (a, b) | \dots \}$ instead of $\{ x | x = (a, b), \dots \}$.
Putting all of these corrections together, we get
$$
A = \{ (n, h) | n \in \mathbb Z \wedge (h \in \mathbb Z \vee h = +\infty) \}
$$
or maybe
$$
A = \mathbb Z^2 \cup \{ (n, +\infty) | n \in \mathbb Z \}.
$$
Best Answer
Strictly speaking, it is mostly subjective; $h\pi\equiv\pi h$, so neither of them are wrong. But, some things just look nicer.
Of course, nobody is going to write $x17\ (17x)$, $e^x2\ (2e^x)$, or $\pi3\ (3\pi)$. But, it seems perfectly fine (to my eyes) to write $e^xy$ or $\pi\sqrt6$.
To make this slightly less biased, I did some tests on a Casio fx-82AU PLUS II. In the following table, I enter the expressions exactly as I entered them into the calculator. "N" means that the expression returned a syntax error and "Y" means that it evaluated. $$ \begin{array} {|r|r|} \hline \text{Expression} & \text{Evaluated?} \\ \hline 3\pi & Y \\ \hline \pi3 & N \\ \hline (\pi)(3) & Y \\ \hline (\pi)3 & N \\ \hline \pi(3) & Y \\ \hline \pi\log6 &Y \\ \hline \log(6)\pi &Y \\ \hline \pi\sqrt6 &Y \\ \hline \sqrt6\pi & Y \\ \hline \frac23\pi & Y \\ \hline \pi\frac23 & Y^1 \\ \hline |3|2 & N \\ \hline |3|(2) & Y \\ \hline \pi|3|\pi & Y \\ \hline \pi|3|\pi\sqrt3e & Y \\ \hline \pi|3|\pi\sqrt3e\frac23 & Y^1 \\ \hline \end{array} $$ $^1$The calculator automatically added brackets around the fraction when I pressed $=$, for example the screen said $\pi\left(\frac23\right)$.
$3\pi$ looks better than $\pi3$, but $\pi r^2$ looks better than $r^2\pi$. $a+bi$ is the usual form for a complex number, but you'll almost always see $e^{i\pi}+1=0$, not $e^{\pi i}+1=0$. But (as mentioned in the comments), $e^{2\pi i}$ is preferred over $e^{i2\pi}$, although we write $e^{i\theta}$ not $e^{\theta i}$.
Basically, there is no precise set of rules to follow when ordering things. (Assuming the operation is commutative, that is.) The way I usually see things done is: if you're dealing with the Latin alphabet, write in alphabetical order. Usually, integers or fractions (put the integers in the fractions (multiply) if there are fractions) go first, numbers with "operations" ($\sqrt,\ \log,\ \tan$, etc.) go next, and variables go last.
Of course, this won't always work, just write whatever looks the neatest and conveys what you're trying to say the clearest. To avoid any possibility of ambiguity, use brackets; brackets also seem to "soften the blow" of bad ordering; $x17$ just looks disgusting, whereas $(x)(17)$ is acceptable.