I'm guessing C or D because odd degree polynomials which aren't linear extend one way to infinity and the other way to negative infinity?
So what's the relevance of the a>0? As x approaches infinity we should have infinity so the answer is C?
[Math] 5th degree polynomial with positive leading coefficient
algebra-precalculusgraphing-functionspolynomials
Related Solutions
If you know for a fact that your points lie on a polynomial of degree 5, you can use finite differences. This is especially easy when the $x$-values are in arithmetic progression, as in your case. Write the $y$-values:
175 125 100 125 0 -125 -100 -125 -175
Then write the difference between each number and the next:
-50 -25 25 -125 -125 25 -25 -50
Again:
25 50 -150 0 150 -50 -25
Again:
25 -200 150 150 -200 25
Again:
-225 350 0 -350 225
And once more:
575 -350 -350 575
Now if your points were really from a polynomial of degree 5, that last line would have been constant, but it's not, so they're not. But after all, you said they were estimated points - they still might be close to some polynomial of degree 5. To find the polynomial of degree 5 that comes closest to your points, there is a method called least squares, and there are many expositions of it on the web.
EDIT: Here's a start on least squares:
You want a polynomial $p(x)=ax^5+bx^4+cx^3+dx^2+ex+f$ such that $p(800)=175$, $p(600)=125$, etc. We already know that's impossible, so we settle for making all the numbers $p(800)-175$, $p(600)-125$, etc., small. In fact, we form the quantity $$(p(800)-175)^2+(p(600)-125)^2+\cdots+(p(-800)-175)^2$$ and we try to minimize it. This quantity is a function of the 6 variables $a,b,c,d,e,f$, and there are standard calculus techniques for minimizing such a function. It gets very messy, but fortunately you don't actually have to do it; someone has done it for you, in the most general case, and found that you can write down a very simple matrix equation which you can solve for the unknowns $a,b,c,d,e,f$. And that's what you'll find if you search for least squares polynomial fit.
If you ask me, you did the good thing ignoring the hint and solving it your way. However, there is an "algorithm" for solving these type of problems that the hint is probably referring to. Notice that sum of coefficients is $0$, so $x = 1$ is root. Dividing by $x - 1$ leaves $$6x^4 + 11x^3 - 40x^2 +11x +6 = 0$$ and noticing that $0$ isn't a root and dividing by $x^2$ you get $$6(x^2 + \frac{1}{x^2}) + 11(x + \frac{1}{x}) - 40 = 0$$ Now substitute $y = x + \frac{1}{x}$, and notice $x^2 + \frac{1}{x^2} = y^2 - 2$, you are left with two quadratic equations to solve.
Best Answer
The answer is $C$. Btw, notice there's a '$3x$'. This is just a distraction. All we need to know to determine the answer is:
$1)$ The polynomial is $5th$ degree
$2)$ $a>0$
This tells us that as $x$ approaches $\infty$ $f(x)$ should approach $\infty$. As $x$ approaches $-\infty$, $f(x)$ should approach $-\infty$. $C$ is the only choice that satisfies these requirements.
My favorite way of remembering the end behavior of polynomials is by just thinking about a line for an odd degree polynomial, and a quadratic for an even degree polynomial. The end behavior of both of these should already be very familiar to you. All functions of odd degree will have the same end behavior as lines (with the respective positive or negative leading coefficient) and functions of even degree will have the same behavior as parabolas.