Note that $80 = \mathrm{lcm}(2,10,16)$. So you can write $a^{80} = (a^2)^{40} = (a^{10})^{8} = (a^{16})^5$. So,
\begin{align*}
a^{80}= (a^2)^{40} &\equiv 1^{40} = 1\pmod{3},\\
a^{80}= (a^{10})^{8} &\equiv 1^8 = 1 \pmod{11},\\
a^{80}= (a^{16})^5 &\equiv 1^5 = 1\pmod{17}.
\end{align*}
By the Chinese Remainder Theorem, the system of congruences
\begin{align*}
x&\equiv 1\pmod{3}\\
x&\equiv 1\pmod{11}\\
x&\equiv 1\pmod{17}
\end{align*}
has a unique solution modulo $3\times 11\times 17 = 561$. But both $x=1$ and $x=a^{80}$ are solutions. Since the solution is unique modulo $561$, then
the two solutions we found must be congruent. That is,
$$a^{80}\equiv 1\pmod{561}.$$
(Added. Or, more simply, as Andres points out, since $3$, $11$, and $17$ each divide $a^{80}-1$, and are pairwise relatively prime, then their product divides $a^{80}-1$).
Once you have that $a^{80}\equiv 1\pmod{561}$, then any power of $a^{80}$ is also congruent to $1$ modulo $561$. In particular,
$$a^{560} = (a^{80})^{7} \equiv 1^7 = 1 \pmod{561}$$
as desired.
There is a minor issue. Not every prime divisor of your $n$ is necessarily of the form $6k+1$. (Because, as you noted, the theorem with the Légendre-symbol is only valid for $p\geqslant5$.) You would have to argue that $2$ and $3$ do not divide $n$.
This can be fixed by letting $n=(2p_1,\cdots,p_k)^2+3$ as you noted, or alternatively by observing that $x^2\equiv1\pmod8$ for odd $x$, which means your $n$ cannot be a power of $2$, hence has an odd prime divisor.
Best Answer
Actually much stronger is true. We have that
The proof is as follows