Let $a,b$ be positive integers $>1$ such that $(a^n-1)(b^n-1)$ is a square for all $n\ge 1$. Prove that $ab$ is a perfect square.
I'm not asking for a solution to this problem because I already know one. What I'm asking is would the following approach work? or rather I want a particular limit to equal zero.
Assume $(a^n-1)(b^n-1)$ is a perfect square $\forall n\ge1$ and let $x_n=\sqrt{(a^n-1)(b^n-1)}$. Note that $x_n\in \mathbb N $ Now $$x_{n+2}=\sqrt{(a^{n+2}-1)(b^{n+2}-1)}=ab\sqrt{\left(a^{n}-\frac{1}{a^2}\right)\left(b^{n}-\frac{1}{b^2}\right)}$$
and $ab\cdot x_n=ab\sqrt{(a^n-1)(b^n-1)}$. Define $d_n=abx_n-x_{n+2}$ and let us assume for the moment $$\lim_{n\to \infty}d_n=\lim_{n\to \infty}abx_n-x_{n+2}=0$$
Therefore $(d_n)$ converges to $0$, but $d_n\in \mathbb N $ hence $d_n$ is eventually $0$. Meaning for big enough $N$, we have $\forall n\ge N$ $$d_n=abx_n-x_{n+2}=0$$
in other words $abx_n=x_{n+2}$. From this I think there is a lot of ways to get a contradiction but this is what I did,
$$\forall n\ge N' \quad ab\mid x_{n}\mid x_n^2=(a^n-1)(b^n-1)=(ab)^n-a^n-b^n+1$$
which means $a\mid ab\mid a^n+b^n-1\implies a\mid b^n-1$. In particular $$\forall p\ge N' \quad b^p\equiv 1\pmod a$$
Where $p$ is a prime. But this implies $\operatorname{ord}_ab\mid p$ hence $r=\operatorname{ord}_ab=1$ or $p$. Using the very same method we can see $s=\operatorname{ord}_ba=1$ or $p$ both gives a contradiction. If one of $r$ or $s$ is $p$ the contradiction is immediate since $p\mid \phi(a)$ or $\phi(b)$ but clearly this not okay for large enough $p$. Thus both of $r$ and $s$ is $1$. But this means $$a\mid b-1 \text { and } b\mid a-1$$ Hence $a+b\le a+b-2\implies 0\le -2$ a contradiction.
I think that the solution doesn't contain errors (If it did tell me). The only thing that I'm not sure of is $\lim d_n=0$ because I just counldn't evaluate $$\lim_{n\to \infty}\sqrt{(a^n-1)(b^n-1)}-\sqrt{\left(a^{n}-\frac{1}{a^2}\right)\left(b^{n}-\frac{1}{b^2}\right)}$$
You can clearly multiply by the conjugate but after expanding you get $-\infty /\infty$.
Best Answer
The limit of $d_n$ is not zero except if $a=1$ or $b=1$. Assume that $b \ge a \ge 2$. Then,
\begin{align*} \sqrt{\Big(a^n-\frac{1}{a^2}\Big)\Big(b^n-\frac{1}{b^2}\Big)} & \ge \sqrt{\Big(a^n-\frac{1}{4}\Big)\Big(b^n-1\Big)} \\ & \ge \sqrt{(a^n-1)(b^n-1) + \frac{3}{4}(b^n-1)} \\ & \ge \sqrt{(a^n-1)(b^n-1)} + \frac{3(b^n-1)}{8\sqrt{(a^n-1)(b^n-1) + \frac{3}{4}(b^n-1)}}\\ & \ge \sqrt{(a^n-1)(b^n-1)} + \frac{3(b^n-1)}{8\sqrt{\frac{7}{4}(b^n-1)(b^n-1)}} \mbox{ as $b \ge a$}\\ & \ge \sqrt{(a^n-1)(b^n-1)} + \frac{3}{4\sqrt{7}} \end{align*}
where the third inequality comes from: $\sqrt{x+y} - \sqrt{x} = \frac{y}{\sqrt{x+y}+\sqrt{x}}$ so $\sqrt{x+y} \ge \sqrt{x} + \frac{y}{2\sqrt{x+y}}$.