I hesitate to give you a checklist, because I fear you'll interpret this as "something to memorize" and will miss the general principle; it's not a recipe, it's a set of steps you may want to do in order to organize your approach to attacking optimization problems.
This is my checklist for Optimization Problem:
Read the problem. Carefully.
Yes, it's incredibly silly to say it, but you'd be surprised how many people try to solve problems without reading them carefully, and end up solving something else, or not being able to solve it because they can't figure out what the problem is.
Identify what quantity you are trying to optimize, and whether you want to maximize it or minimize it. Give it a name (a single letter that will make it clear).
Draw a picture. In that picture, label any quantity that will play a role in computing the quantity you have in part 2.
Identify any constraints you have, and write them in terms of the labels you introduced in 3.
Write down a function for the quantity you want to optimize, in terms of the quantities in 3.
Use the constraints you have to change the function from 5 into a function of a single variable. Identify its domain in terms of the original problem.
Optimize the function you found in 6 the "usual way": if the domain is a finite closed interval, find the critical points, and evaluate at the critical points and the end points of the interval; largest value is the maximum, smallest value is the minimum. If the domain is not a finite closed interval, use other techniques you have for determining whether it has an absolute extreme of the kind you want and to find it.
Check that everything makes sense and checks out (that is, that your solution meets the constraints). Not a bad idea.
Translate your answer back into the setting of the problem. That is:
A "Word Problem" should have a "Word Answer." Always answer a 'word problem' with a sentence that describes, in words, the solution to the problem, including units (if applicable), etc.
So let's look at your problem:
If $1200\ \mathrm{cm}^2$ of material is available to make a box with a square base and an open top, find the largest possible volume of the box.
The quantity we want to optimize is the volume of the box.
Let $V$ be the volume of the box. We want to find the maximum value of $V$.
Now, let's draw a picture: we have a box with a square bottom, no top, arbitrary height. The two bottom sides are the same; let's call the length of the bottom $b$; let's call the height of the box $h$.
b
------------
b / / |
/ b / | b = length of the side of the base, in cm.
---------- | h = height of the box, in cm.
| | | h
| | |
h| h| |
| | /
| | /b
-----------
The constraints we have are that the total material can only be 1200 $\mathrm{cm}^2$. How much material is in the box with the open top? A base, and four sides.
The base uses $b^2\ \mathrm{cm}^2$ of material. Each of the sides uses $hb\ \mathrm{cm}^2$ of material. So the total material used is $b^2 + 4bh$. That is:
Constraint: $b^2 + 4bh = 1200$.
The volume is the area of the base times the height. That is,
$V = b^2h$. We want to find the maximum.
The volume depends on both $b$ and $h$. We want to express it as a function of a single quantity. In order to do that, we have to either write $h$ in terms of $b$, or write $b$ in terms of $h$. We will do this by using the constraint. Because in the constraint there is only one $h$, it is easier to "solve for $h$" than to "solve for $b$". We solve for $h$ so that we can "plug in" into $V$.
$$\begin{align*}
b^2 + 4bh &= 1200\\
4bh &= 1200-b^2\\
h &= \frac{1200 - b^2}{4b}.
\end{align*}$$
Now we use this expression and plug it into $V$:
$$\begin{align*}
V &= b^2h\\
V &= b^2\left(\frac{1200-b^2}{4b}\right)\\
V &= \frac{b^2(1200-b^2)}{4b}\\
V &= \frac{b(1200-b^2)}{4}.
\end{align*}$$
Now we can think of $V$ as a function of the length of the base:
$$V(b) = \frac{1}{4}b(1200-b^2) = \frac{1}{4}(1200b-b^3).$$
What is the domain of our $V$? The largest that the base could possibly be is a square that uses up the entire $1200\ \mathrm{cm}^2$ of material (not much a box, but possible). This will happen if $b^2 = 1200$, or if $b=\sqrt{1200} = 10\sqrt{12} = 20\sqrt{3}$. That's the largest that $b$ can be.
What's the smallest that $b$ can be? In principle, it can be any number greater than $0$: you just make a box that is really tall. It shouldn't be allowed to be $0$, because then you don't really have a box; but then you don't really have a box if $b=20\sqrt{3}$ either, so we may as well "throw in" $b=0$ into the domain as well. So the domain of the function is $[0,20\sqrt{3}]$.
That's perfect: we have a continuous function, $V(b) = \frac{1}{4}(1200b-b^3)$, on a finite closed interval $[0,20\sqrt{3}]$. We know that the absolute maximum (which is what we are trying to find) is either at a critical point, or at an endpoint.
So we first find the critical points: the points where the derivative does not exist, or is zero. We have:
$$\begin{align*}
V(b) &= \frac{1}{4}(1200b - b^3)\\
V'(b) &= \left(\frac{1}{4}(1200b - b^3)\right)'\\
&= \frac{1}{4}(1200b - b^3)'\\
&= \frac{1}{4}(1200 - 3b^2).
\end{align*}$$
So $V'(b)$ is defined everywhere. The critical points are then only where the derivative is $0$, which is where $1200-3b^2 = 0$.
$$\begin{align*}
1200-3b^2 &= 0\\
1200 &= 3b^2\\
400 &= b^2\\
\sqrt{400} &= |b| &\text{(but }b\text{ has to be positive: it's a length)}\\
\sqrt{400} &= b\\
20 &= b.
\end{align*}$$
So, the only critical point is $b=20$. The maximum has to be either at the critical point, or at an endpoint of the domain. So we just evaluate $V$ at these three points and determine which one is the largest value; that's the maximum:
$$\begin{align*}
V(0) &= \frac{1}{4}(1200(0) - (0)^3) = 0;\\
V(20) &= \frac{1}{4}(1200(20) - (20)^3) = \frac{1}{4}(24000 - 8000) = 4000;\\
V(20\sqrt{3}) &= \frac{1}{4}\left(1200(20\sqrt{3}) - \left(20\sqrt{3}\right)^3\right)\\
&= \frac{1}{4}\left( 24000\sqrt{3} - 24000\sqrt{3}\right)\\
&= 0.
\end{align*}$$
So the largest value occurs at $b=20$.
What does that mean in terms of the original problem? Our box will have a base that is $20$ centimeters on each side, and height will be
$$h = \frac{1200-b^2}{4b} = \frac{1200 - (20)^2}{4(20)} = \frac{1200 - 400}{80} = 10\ \text{cm}.$$
Our "optimal box" will have a base that is 20 cm by 20 cm, height 10 cm, and volume 4000 cubic centimeters.
Let's do a check: if the base is $20\times 20\ \text{cm}^2$ and the height is $10$ cm, how much material have we used? The base is $20\times 20 = 400\ \text{cm}^2$. Each side is $20\times 10 = 200\ \text{cm}^2$, and we have four sides. So the total amount of material used on the sides is $4\times 200 = 800\ \text{cm}^2$. Base and sides together do amount to $1200\ \text{cm}^2$ of material. Good.
Now, let's go back and read the question: what are they asking us? The volume of the largest box under the given constraints. So:
Answer. The largest possible volume for a box with a square bottom and no top that is constructed out of 1200 $\mathrm{cm}^2$ of material is 4000 $\mathrm{cm}^3$.
You may also want to add:
This is achieved by the box of height $10$ cm and base $20$ cm by $20$ cm.
When you write down a formula, you must write down what the letters stand for.
When you write down $4lw+w^2=1200$, you must add, "where $l$ is the length of the base of the box, and $w$ is the width of the base of the box".
If you do that, you might see right away where you've messed up. You've written down a formula for the area which doesn't include the height of the box---that can't possibly be right, can it?
In fact, the formula you have written down only makes sense if $l$ is the height of the box, right?
So go back and identify the variables explicitly and then write down formulas that make sense.
Best Answer
Looks good to me. What did your professor say the correct answer was? Can your professor explain their work? How many other questions has the professor messed up?
The only thing I would have done differently, and this is a personal preference in this case, is that I would have expanded $x(1200-x^2)$ to $1200x - x^3$ before differentiating, so I would find the root of $1200-3x^2=0$. The result is the same, of course.