For the sake of simplicity, I will consider positive integers $a,b$.
First, let's consider the case of $4a^2-b^{b+1}=0$. You've already noticed that $b$ cannot be odd, and so we shall consider only the case of even $b$. If $b$ is even, then $b+1$ is odd and so $b^{b+1}$ is a perfect square iff $b$ is a perfect square, so we have that $b$ is a perfect square. Since it is an even perfect square, we may write $b=4c^2$ and $b^{b+1}=4\cdot 4^{4c^2}\cdot c^{8c^2+2}$ and we may let $a=4^{2c^2}\cdot c^{4c^2+1}$ for a solution. Thus, we have found one possible solution set:
$$(a,b)=(4^{2c^2}\cdot c^{4c^2+1},4c^2)$$
Now suppose that $4a^2-b^{b+1}=1$. Then $(2a+1)(2a-1)=b^{b+1}$, and so $b$ must be odd, meaning that $b+1$ is even and $b^{b+1}$ is a perfect square. But $4a^2=b^{b+1}+1$ is also a perfect square, giving a contradiction. So $4a^2-b^{b+1}\ne 1$.
Similarly, suppose that $4a^2-b^{b+1}=-1$, or $4a^2=b^{b+1}-1$. We have again that $b$ must be odd, and so $b+1$ is even and $b^{b+1}$ is a perfect square, which is a contradiction since $b^{b+1}-1$ is a perfect square.
Now suppose that $4a^2-b^{b+1}=2$, or $4a^2=b^{b+1}+2$. Then $b$ must be even, and we have that the $2$-adic valuation of $b^{b+1}+2$ is equal to $1$. But the $2$-adic valuation of $4a^2$ is at least $2$, giving a contradiction.
Use the same reasoning for the case of $4a^2-b^{b+1}=-2$.
Suppose that $4a^2-b^{b+1}=3$, or $4a^2=b^{b+1}+3$. Then $b$ must be odd and $b+1$ must be even, so $b^{b+1}$ is a perfect square. However, $b^{b+1}+3$ is also a perfect square, which is only possible if $b^{b+1}=1$. This gives the solution pair
$$(a,b)=(1,1)$$
and no others.
Finally, suppose that $4a^2-b^{b+1}=-3$ or $4a^2=b^{b+1}-3$. Then $b$ must be odd and $b+1$ must be even, so $b^{b+1}$ is a perfect square. However, $b^{b+1}-3$ is also a perfect square, which can only occur if $b^{b+1}=4$, but this never happens for positive integers $b$.
We are done! We have only the solutions $(1,1)$ and
$$(4^{2c^2}\cdot c^{4c^2+1},4c^2)$$
for $c\in\mathbb N$.
Lets look for $2^n$ formula that might work:
$(2^k)^5+(2^l)^7=(2^n)^9$
so we know $2^t+2^t=2^{t+1}$
Than we look for $5 k=7l=9n-1$
$k=\frac{9n-1}{5}, l=\frac{9n-1}{7}$
we are looking for int, so you want the fractions to be integers.
It means $7|9n-1$ and $5|9n-1$ which is $35|9n-1$
Just from a look, you can tell $n=4$ will work
So $x=2^7, y=2^5, z=2^4$
Best Answer
Just follow your nose; let $d:=\gcd(a,b)$ so that $a=du$ and $b=dv$ with $u$ and $v$ coprime. Then $$b^a=(dv)^{du}=((dv)^u)^d \qquad\text{ and }\qquad a^{b^2}=(du)^{d^2v^2}=((du)^{dv^2})^d,$$ from which it follows that $(dv)^u=(du)^{dv^2}$. Because $u$ and $v$ are coprime we either have $u=1$ or $v=1$.
If $u=1$ then $dv=d^{dv^2}$, and so $v=d^{dv^2-1}$ from which it quickly follows that also $v=d=1$ and hence $a=b=1$.
If $v=1$ then $d^u=(du)^d$ from which it follows that $u^d=d^{u-d}$, and in particular $u\geq d$. Let $c:=\gcd(d,u)$ so that $d=ce$ and $u=cw$ with $e$ and $w$ coprime and $w\geq e$. Then $$u^d=(cw)^{ce}=((cw)^e)^c \qquad\text{ and }\qquad d^{u-d}=(ce)^{cw-ce}=((ce)^{w-e})^c,$$ from which it follows that $(cw)^e=(ce)^{w-e}$. As $e$ and $w$ are coprime and $w\geq e$ it follows that $e=1$, so $$cw=c^{w-1},$$ and hence $w=c^{w-2}$, from which it quickly follows that $w\leq4$. We check these few cases: