Now that I understand your "layout", yes, the formulae you created will work just fine, remembering that little "y" is the dimension of the length of a smaller rectangle, and $Y = 3y$ the dimension of the length of the large rectangle encompassing the whole.
$$ 4x+3y = 1080,~~~\text{Area} = 3xy.$$
We can then express Area as a function of $x$, first solving for $y$ in the first equation, and substituting this expression for $y$ into the Area equation:
$$4x + 3y = 1080 \iff 3y = 1080 - 4x \iff y = 360 - \frac 43 x$$
$$\text{Area}\;= 3x(360 - (4/3)x) = x(1080 - 4x) = = 1080x - 4x^2$$
To maximize area, we calculate $A'$ and set $A' = 0$ to determine the function's critical point:
$$A' = 1080 - 8x,\qquad A' = 0 \implies 1080 - 8x = 0 \implies x = 135. $$ We can easily show that the solution to $A' = 0 \iff x = 135$ ft. does in fact give the maximum value for $A$.
Now we have $x$, and will need only to compute $y = 360 - 4/3(135) = 360 - 180 = 180$ ft.
Then $Y = 3 \times 180 = 540$ ft, and so we've determined the larger rectangle's dimensions for maximized area: $ Y$ ft $ \times x$ ft $= 540 \text{ ft} \times 135 \text{ ft}$, with smaller rectangles each $180 \text{ ft} \times 135\text{ ft}$
Let $L$ be the length and $W$ be the width of the field. The required area gives you a relation between $L$ and $W$. Let the length be the direction that has the cheap fences on both sides (because you guess it will be longer, but if it comes out the other way that is OK.) What is the cost of the fence? Use the relation from the area to get cost as a function of (say) width. Differentiate, set to zero.....
Best Answer
What got you into difficulty is that you did not take into account that one side of the rectangular region is bounded by the river. Thus, the perimeter of the fence includes just three sides of the rectangle.
The area of the rectangular region is $1800 = lw$, so
$$w = \frac{1800~\text{ft.}^2}{l}$$
The perimeter of the fence is $P = l + 2w$. Substituting for $w$ yields
\begin{align*} P(l) & = l + 2\left(\frac{1800~\text{ft.}^2}{l}\right)\\ & = l + \frac{3600~\text{ft.}^2}{l} \end{align*}
Differentiating yields
$$P'(l) = 1 - \frac{3600~\text{ft.}^2}{l^2}$$
Setting the derivative equal to zero yields $l = 60~\text{ft.}$ The derivative is negative if $l < 60~\text{ft.}$ and positive if $l > 60~\text{ft.}$ Thus, by the First Derivative Test, the perimeter is minimized if $l = 60~\text{ft.}$
If $l = 60~\text{ft.}$, then the width of the rectangular region is
$$w = \frac{1800~\text{ft.}^2}{l} = \frac{1800~\text{ft.}^2}{60~\text{ft.}} = 30~\text{ft.}$$
so the fence has perimeter
$$P = l + 2w = 60~\text{ft.} + 2(30~\text{ft.}) = 120~\text{ft.}$$