The underlying question here, I beleive, is what's called monotony, which basically means increasing or decreasing. Consider a function a real-valued function $f$, that takes real variables, with the property that: $$\text{For real numbers $x,y$ of a subset A} \\ x<y \Rightarrow f(x) < f(y)$$
Then we say that $f$ is increasing (in $\text{A}$) This happens, for example, with $f\mid f(x)=2x$. On the other hand, take $g \mid g(x)=x^2$, and the numbers $-3,2$. Obviously $-3<-2$, but $g(-3) \not< g(2)$. Actually if $x<0<y$, then $g(x)\not <g(y)$. This is why $$\sqrt{8x^{2}+22x+15} > 4x+3\not\Rightarrow \ \big( \sqrt{8x^{2}+22x+15} \big) ^{2} > \big(4x+3\big) ^{2},
$$ because $4x+3$ can be negative for, say $x=-2$, which is in the domain you have found that the square root is real (and therefore the expression makes sense). In fact, $4x+3$ is negative if and only if $x<\large\frac{-3}{4}$. Also notice that $x>\large\frac{-3}{4}\Rightarrow $ $x \ge \large\frac{-5}{4}$, and so, for all $x>\large\frac{-3}{4}$ $$\big( \sqrt{8x^{2}+22x+15} \big) ^{2} > \big(4x+3\big) ^{2}
$$
You can continue as before, taking into account that the result applies for the mentioned $x$'s. Say you obtain $x\in P\subset\mathbb R$ ($P$ some subset of real numbers).
For $x < \frac{-3}{4}$ (for $x < \frac{-3}{2}$ to be precise, because the expression is only defined here), the expression holds always. There the original inequality implies one of two situations: $$\left\{\begin{align} x &<\frac{-3}{2} \\ &\mathrm {or} \\ x&\in P\subset\mathbb R\end{align}\right.$$
This should be the final result, correct me if I'm wrong!
As for roots and exponents in general, or any function we "apply" to an inequality (I use quotations because really we are just taking into account the definition of increasing or decreasing, rather than "applying" the function), we just have to find the subsets where the function increases or decreases (decreasing has the same definition, replacing the second "$<$" with "$>$"). Then the procedure will yield, for each subset, a different subset of real numbers in which the inequality holds (could be empty sets). For example, for natural $n$, $h\mid h(x)=x^n$ is increasing in all of $\mathbb R$ if and only if $n$ is odd, and behaves as $x^2$ for even $n$. Try figuring out similar properties for positive rational exponents and positive $x$ (hint: write $x^{p/q}$ as $(x^p)^{\frac1q}$ and use the rules for natural exponents shown before, and think whether the $q$-th root of a number is increasing or decreasing (the function that assigns the $q$-th root). Hope I helped out!
Let $x=(y^2-a)^2$ with $y^2\ge a$ and $y\ge 0$ then
$$x+\sqrt{a+\sqrt{x}}=a \iff (y^2-a)^2+y=a \iff y^4-2ay^2+y-a+a^2=0$$
and since the $y^3$ term is equal to zero we can guess
$$y^4-2ay^2+y-a+a^2=(y^2+y+A)(y^2-y+B)=$$
$$=y^4+(A+B-1)y^2+(B-A)y+AB$$
which leads to $A+B-1=-2a$, $B-A=1$, $AB=a^2-a$ that is
$$y^4-2ay^2+y-a+a^2=( y^2 + y-a) ( y^2 - y + 1-a) = 0$$
Answer completed with a full solution here.
Best Answer
Setting $p=x+y$ and $q=\sqrt{xy}$ (almost as suggested by Alexey in the comments, but $\sqrt{xy}$ looks like it will be more symmetric) and expanding gives \begin{align} 3p^2q + 4q^3 &= 14 \\ p^3 + 12pq^2 &= 36 \end{align} From here, a lucky coincidence is that $$ (p + 2q)^3 = p^3 + 6p^2q + 12pq^2 + 8q^3 = 36 + 2 \cdot 14 = 64 $$ and $$ (p - 2q)^3 = p^3 - 6p^2q + 12pq^2 - 8q^3 = 36 - 2 \cdot 14 = 8 $$ which gives us $p+2q = 4$ and $p-2q = 2$. Therefore $p=3$ and $q = \frac12$, giving us $x,y = \frac{3 \pm 2\sqrt2}{2}$.