This is likely piling one mystery on another, but …
I was exploring a function $f(n): \mathbb{N} \mapsto \mathbb{N}$ defined as follows:
$$
f(n) =
\begin{cases}
n^2 & \text{if} \;n \;\text{is prime} \\
\lfloor n/2 \rfloor & \text{if} \;n \;\text{is composite}
\end{cases}
$$
For example,
$$
\begin{eqnarray*}
f(11) &=& 121 \\
f(121) &=& 60 \\
f(60) &=& 30 \\
f(30) &=& 15 \\
f(15) &=& 7 \\
f(7) &=& 49 \\
f(49) &=& 24 \\
f(24) &=& 12 \\
f(12) &=& 6 \\
f(6) &=& 3 \\
f(3) &=& 9 \\
f(9) &=& 4 \\
f(4) &=& 2 \\
\end{eqnarray*}
$$
and now we are in a $2/4$ cycle.
For all $n=2,\ldots,228$, repeated applications of $f(n)$ leads to a $2/4$ cycle.
For $n=229$ (a prime), $f(229)$ seems to shoot off, in spurts, to large values:
$$229, 52441, 26220, 13110, 6555, 3277, 1638, 819, 409, 167281, 83640,
41820, 20910, 10455, 5227, 27321529, \ldots $$
For $n=2,\ldots,10^4$, about $87$% end in a $2/4$ cycle, and $13$% seem to shoot off
into the realm of large numbers.
Q0. Have iterates of this function $f(n)$ (or its close analogs) been studied before?
Q1. Is there any hope of explaining the behavior of iterates of $f(n)$?
Q2. Can the convergence of iterates of $f(n)$ be proved for selected classes of values of $n$?
For example, $n=2^k$ always ends in the $2/4$ cycle.
Q3. For which values of $n$ can it be proved that $f^k(n) \to \infty$?
(Added 8Aug15): Here are the trajectories of the first 20 primes (in red),
all heading toward the $2/4$ cycle (green). It remains unsettled if (a) any integer starting
value diverges to $\infty$, and (b) whether the only cycle is the $2/4$ cycle.
Best Answer
Let us use a rough estimate for the probability of a large integer $n$ being prime as $\frac{1}{\ln n}$ (as suggested by PNT). Then the expectation of $\ln f(n)$ can be estimated as $$\frac{1}{\ln n}\cdot \ln n^2 + \left(1-\frac{1}{\ln n}\right)\cdot \ln \lfloor \frac{n}{2}\rfloor = \ln n + 1 - \ln 2 + O(\frac{1}{\ln n}).$$ Since $1-\ln 2>0$, we have that the expectation of $\ln f(n)$ is larger than $\ln n$, which suggests that iterations of $f()$ may diverge for some large $n$.
This approach also suggests that to eliminate divergence, instead of the divisor 2 in the composite case one needs to take a divisor larger than $e\approx 2.718$ (e.g., 3).
P.S. Unavoidable divisions after squaring do not change the picture. E.g., if we have $\lfloor\frac{n^2}{2^5}\rfloor$ instead of $n^2$, the expectation of $\ln f(n)$ is still estimated as $\ln n + 1 - \ln 2 + O(\frac{1}{\ln n})$ for large $n$.