No solution, just some heuristics:
Assume we have $0<n<10^k$ such that $n^2$ ends in $k$ digits $\{0,1\}$.
Then for the numbers $n+10^kd$, $d\in\{0,\ldots,9\}$, we have $(n+10^kd)^2=n^2+2\cdot d\cdot 10^k+10^{2k}d^2$, which may or may not end in $k+1$ digits $\{0,1\}$. Success happens for exactly two choices of $d$, which only depend on the "next" digit of $n^2$. The two choices of $d$ differ by $5$ and produce the same next 0-1-digit in the square. This way, we obtain a binary tree of longer and longer possible ending sequences, starting from $1$:
$$\begin{align} 1&\\&\to 01,51\\&\to 001,501,251,751\\&\to 0001,5001,0501,5501,4251,9251,3751,8751\\&\to\ldots\end{align}$$
[Another way to state this, is to note that $n^2\equiv a\pmod{10^k}$ has four solutions for any 0-1 remainder $a$ ending in $001$]
The question is if at some stage we are "lucky" and the next $k$ digits happen to be $0$s and $1$s. Very heuristically, the probaility for such an event is $5^{-k}$ and in each "generation" we have $2^k$ candidates. Thus each generation contributes $(\frac25)^k$ solutions on average, hence all generations together contribute $1+\frac25+\frac4{25}+\ldots=\frac53$ solutions, one of which is $1^2=1$. Thus we might expect that at most one extra solution might be found, and that relatively early ...
If you factor a perfect square into primes, every prime has an even exponent. The argument made says that for all $n \gt 5,$ there are at least two factors of $2$ in $n!$. In fact, this is true for all $n \ge 4$, but they covered $4$ and $5$ in the casework. Once $n!$ is a multiple of $4$, $n!+10\equiv 2 \pmod 4$ has an odd exponent of $2$, so it cannot be a perfect square.
Best Answer
Dividing by an appropriate power of $100$ we can ensure that the final two digits are not both $0$. But a simple search (or congruence argument) shows that none of $2,20,22$ are squares $\pmod {100}$.