You don't say so explicitly, but your formulas imply that the
symbol $r$ represents the radius of the cylinder and $h$ is the height
of the cylinder.
There are many differently-shaped cylinders that can fit inside a
given sphere. If the radius of the sphere is $6$,
near one extreme, you have a very skinny cylinder whose
height is very close to $12$ and whose radius is close to zero.
Near the other extreme, you have a cylinder that is almost a flat disk
with a very small "height" (or thickness) $h$ and a radius that is
barely less than $6$.
We can even take the cylinder all the way to the extreme where it is
only a degenerate cylinder: $h=12$, $r=0$ describes a line segment that
just exactly fits in the sphere, and $h=0$, $r=6$ describes a flat disk
that also exactly fits in the sphere.
There's not much point in considering $r < 0$, because what is a cylinder
with a negative radius? (Actually there is a way to interpret such a thing,
but if you allow, for example, a cylinder of radius $-2$, it's identical to
a cylinder of radius $2$; so you have found no new cylinders but you have
to rewrite formulas such as $A(r) = 2\pi rh$ because the lateral area
of a cylinder is not less than zero. It's easier, and we don't miss
any answers, if we consider only cylinders such that $r \geq 0$.)
There is certainly no point in considering a cylinder with radius $r > 6$
if the radius of your sphere is $6$, because there is no way such
a cylinder could possibly fit inside the sphere.
You could also come to these conclusions by examining the formula
you found,
$$A(r)=2\pi r \cdot 2\sqrt{36-r^2},$$
because $r < 0$ gives a negative area (which can't be correct)
and $r > 6$ asks you to take the square root of a negative number,
which doesn't work.
But I think it's a good idea to have reasons based in the original
problem statement that say why a variable $r$ should have only a certain
possible set of values, because there's no guarantee that every
restriction that the problem statement requires will also be
enforced by whatever formulas we derive.
So we are OK with $r=0$ (if we accept a line segment as a degenerate cylinder of radius zero), we are OK with $r=6$ (if we accept a circle as a degenerate cylinder of height zero), and we are OK with
anything in between--just set $h= 2\sqrt{36-r^2}$, as you discovered,
and the cylinder will fit perfectly.
But $r < 0$ is useless to consider and $r > 6$ is impossible.
So $0 \leq r \leq 6$ describes every cylinder that we could ever want
to inscribe in a sphere of radius $6$.
Best Answer
More than a hint...If $R$ is the radius of the sphere and $r$ is the radius of the cylinder, with $h$ the height of the cylinder, then by Pythagoras we have $$\frac{h^2}{4}=R^2-r^2$$
The volume of the cylinder is then $$V=2\pi r^2\sqrt{R^2-r^2}$$ $$\implies V^2=4\pi^2(r^4R^2-r^6)$$
You can differentiate this to get $$2VV'=4\pi^2(4r^3R^2-6r^5)=0$$ So the maximum $V$ which is when $r=R\sqrt{\frac 23},$ and $$V_{max}=\frac{4\pi R^3}{3\sqrt{3}}$$