The second is neither convex nor concave - that's easy to determine simply by looking at it. Along the line $y=x$, it is convex as a 1D function; along the line $y=-x$ it is concave. It is neither quasi-convex nor quasi-concave: to show not quasi-concave, consider the points $x = (0, 1)$, $y = (-1, 0)$, so $f(x) = f(y) = 0$. Parametrise the function along that line segment by $\lambda$; then $f(\lambda) = \lambda (\lambda - 1) < 0 = \min \{ f(x), f(y) \}$. You can rotate to get non-quasi-convexity.
The first is convex but not concave, and it's not quasi-concave. Examine the value of $f$ at the points $x=1/3, x=10, x=1$ to see that it's not quasi-concave. It's convex again by inspection or by showing that its second derivative is strictly positive.
You forgot a term in $f_{yy}$. With $f_{yy}=\left(4b^2y^2+2b\right)f$, the sign of the Hessian is the sign of $b$. You can also use the trace of the Hessian, which is $\left(a^2+4b^2y^2+2b\right)f$. The determinant and the trace are the product and the sum, respectively, of the eigenvalues of the Hessian.
For $b\gt0$, the product and sum are both positive, so both eigenvalues are positive and $f$ is strictly convex.
For $b=0$ and $a=0$, the product and sum are both $0$, so both eigenvalues are $0$ (not surprisingly, since in this case $f$ is constant). In this case $f$ is both convex and concave, but neither strictly convex nor strictly concave.
For $b=0$ and $a\ne0$, the product is $0$ and the sum is positive, so one eigenvalue is $0$ and the other is positive; $f$ is convex but not strictly convex.
For $b\lt0$, the product is negative, so $f$ is neither convex nor concave.
Best Answer
This is how you transform a line into a curve on a graph:
Let $f(x)=$ some linear expression obtained by plotting the data pairs $(x_1,y_1), (x_2,y_2)$ on a horizontal $x$-axis and vertical $y$-axis.
Let the parameter $m=$ the contour of the curve.
$m=(0,\infty)$
$g(x)= \begin{cases} f(x), & \text{if $m=1$} \\ (y_2-y_1)\frac{m^{x-x_1}-1}{m^{x_2-x_1}-1}+y_1, & \text{if $m\neq1$} \end{cases}$
when $m=1$, curve is linear
when $0<m<1$, curve is convex
when $m>1$, curve is concave