Proceeding in the usual way, find critical points of the function by setting its partial derivatives to zero and solving. You’re using $f$ for both the function and one of the coefficients in its definition, so I’ll call the function $F$ here. To reduce clutter, let $G(x,y)=dx^2+ey^2+fxy+gx+hy+i$, so that $F(x,y) = (ax+by+c) G(x,y)^{-1/2}$. Differentiating, we have
$$\begin{align} F_x(x,y) &= (a G(x,y) - \frac12 (ax+by+c) (2dx+fy+g))G(x,y)^{-3/2} = 0 \\
F_y(x,y) &= (b G(x,y) - \frac12(ax+by+c)(fx+2ey+h))G(x,y)^{-3/2} = 0, \end{align}$$ so the problem becomes that of finding the intersections of a pair of conics. A general strategy for doing this is to find a degenerate linear combination of the two conics, split it into linear factors and then solve the simpler line-conic intersections. In this case, we’re already most of the way there: subtract $2a$ times the second equation from $2b$ times the first to get $$(ax+by+c)\left[ (af-2bd)x + (2ae-bf)y + (ah-bg)\right] = 0,$$ which is already split into linear terms. The intersection points of the two conics are thus the intersections of the two lines defined by these factors with either of the two original conics.
Turning to the first line, setting $ax+by+c=0$ reduces the partial derivative equations to $aG(x,y)=0$ and $bG(x,y)=0$. Unless $a=b=0$, this means that $G(x,y)=0$, but $F$ is undefined in this region, so there can be no critical points there. That leaves the intersections of the line $$(af-2bd)x + (2ae-bf)y + (ah-bg) = 0$$ with either of the other conics, which I’ll leave to you to work out.
If you’re interested in exploring the qualitative behavior of this family of functions, I’d suggest examining the simpler cases in which $G(x,y)=0$ is one of the canonical examples of the different types of conics. All of the functions in this family can be obtained via an affine coordinate transformation from one of these canonical equations, which will move the critical points in a well-defined way.
When $ 0\le x,y \le 1$, then
$$\frac{x}{y+1} \le \frac{x}{x+y}$$
$$\frac{y}{x+1} \le \frac{y}{x+y}$$
Adding them we have
$$\frac{x}{y+1}+\frac{y}{x+1} \le \frac{x+y}{x+y}=1.$$
Equality holds when $x=0$ and $y=1$ or $x=1$ and $y=0$ and the maximum of $1$ is attained.
Best Answer
By AM-GM, $$\frac{x^2+1}{3}=\frac{x^2+\tfrac{1}{2}+\tfrac{1}{2}}{3}\ge\sqrt[3]{x^2/4}$$ so $$\frac{x}{(x^2+1)^{3/2}}\le\frac{2}{\sqrt{3^3}}$$ with equality when $x^2=1/2$, that is $x=1/\sqrt2$.