Pick a point $A = (x,y,z)$ on the plane so that $x + 3y + 6z = 18$.
The volume $V$ of the box is $V = xyz$. So we find $max(V)$ with conditions $x + 3y + 6z = 18$ and $x, y, z \in \mathbb{R}^{\text{nonneg}}$.
Use $g(x,y,z) = x + 3y + 6z = 18$. So
$$V'(x) = yz = rg'(x) = r$$
$$V'(y) = xz = rg'(y) = 3r$$
$$V'(z) = xy = rg'(z) = 6r$$
This means $xz = 3yz , xy = 6yz \Rightarrow z(x - 3y) = 0, y(x - 6z) = 0$.
If $z = 0$ or $y = 0$, then $V = 0$ and is not a max. So $$x - 3y = 0 = x - 6z \Rightarrow y = \frac{x}{3}, z = \frac{x}{6}$$
$$\Rightarrow x + 3\left(\frac{x}{3}\right) + 6\left(\frac{x}{6}\right) = 18$$
$$\Rightarrow 3x = 18 \Rightarrow x = 6, y = 2, z = 1$$ So $max(V) = 6*2*1 = 12$.
We can formulate the problem as a constrained optimization problem and apply Lagrange multiplier. Express the plane as
$$\frac xa+\frac yb+\frac zc=1$$
It simply means that the normal direction is $\vec { n } =\left<\dfrac 1a,\dfrac 1b,\dfrac 1c\right>.$ This form of the plane is convenient for this problem since it readily shows that the intercepts with $x,y,z$ axis are $a,b,c$ respectively. Therefore the volume of the cutoff tetrahedron is $V(a,b,c)=\dfrac{abc}{6}.$ Now we need to satisfy the constraint
that the plane passes a giving point. Plugging the point $(2,2,1)$ into the plane we get $\dfrac 2a+\dfrac2b+\dfrac1c=1$
So the mathematical model is a constraint optimization problem
$$\begin{cases}min_{a,b,c>0}f(a,b,c)=6V(a,b,c)=abc\\
s.t \mbox{ }g(a,b,c)=\dfrac1a+\dfrac1b+\dfrac1c-1=0\\
\end{cases}$$
By Lagrange multiplier, taking partial derivatives w.r.t. $a, b, c$ as in
$$\nabla f=\lambda\nabla g$$
We get
$$bc=-2\lambda a^{-2}$$
$$ac=-2\lambda b^{-2}$$
$$ab=\lambda c^{-2}$$
Multiplying respectively $a, b, c$ on both sides of the above three equations, we get
$$abc=-\lambda\frac2a=-\lambda\frac2b=-\lambda\frac1c$$
Since $a, b, c$ are positive, we have $\lambda\ne0.$ So we must have
$$a=b, a=2c$$
Plugging the above into the constraint $\dfrac2a+\dfrac2b+\dfrac1c=1$ we get $a=6,b=6,c=3.$ There for the plane that cuts off the smallest volume is
$$\frac x6+\frac y6+\frac z3=1\mbox{ or } x+y+2z=1$$
And the smallest cutoff volume is $V(6,6,3)=\dfrac16\times6\times6\times3=18$
Best Answer
Since $V=xy-\frac{1}{3}x^2y-\frac{2}{3}xy^2$,
$V_{x}=y-\frac{2}{3}xy-\frac{2}{3}y^2=0$ and $V_{y}=x-\frac{1}{3}x^2-\frac{4}{3}xy=0$.
Multiplying by 3 in each equation and dividing by $y$ in the first equation and by $x$ in the second equation (since $x>0$ and $y>0$) gives
$2x+2y=3$ and $x+4y=3$. (Now solve for x and y.)