Energy conservation dictates
$$ E = \frac{1}{2}m\dot{x}^2 + V(x) = \text{const}$$
With some arithmetic it follows
$$ \dot{x} = \frac{dx}{dt} = \sqrt{2m^{-1}(E-V(x))}$$
This ODE can be solved via separation of variables,
yielding
$$ \int_{t_1}^{t_2}dt = \int_{x_1}^{x_2} \frac{dx}{\sqrt{2m^{-1}(E-V(x))}}$$
The integral on the left hand side can be evaluated immediately, where $t_1$ and $t_2$ are understood as the times when the particle is at $x_1$ or $x_2$ respectively. So it is simply half the period.
Observe that the integral on the right diverges when $x$ approaches the turning point $E=V(x)$.
This method of solving Newtons equations in a 1d potential should be treated in any textbook on mechanics.
For a general potential it is in general hard or impossible to find the turning points in closed form. Here however, you can substitute $y=\exp(-ax)$ and solve the corresponding quadratic equation. I'll leave the explicit calculation to you.
I suppose the simplest way to deal with this is in the Lagrangian formalism, assuming the Lagrangian is of the form
$$
L=\frac{1}{2}m\dot{x}^2 - V(x)
$$
The critical points of the Lagrangian are those for which
$$
\left(\frac{\partial L}{\partial \dot{x}}, \frac{\partial L}{\partial x}\right)=0
=\left(m\dot{x},-\frac{\partial V}{\partial x}\right)\, ,
$$
which immediately imply the equilibrium positions are at $\dot{x}=0$ and at the critical points of $V$.
Let $x_0$ be such that $V'(x_0)=0$ and expand $V(x)$ near $x_0$ by writing
$x=x_0+ \delta x$. This yields $\dot{x}=\dot{(\delta x)}$
$$
V(x)\approx V(x_0)+(\delta x)^2 \frac{1}{2}V''(x_0) \tag{1}
$$
since by definition $V'(x_0)=0$. The approximate Lagrangian is thus
$$
L=\frac{1}{2}m(\dot{\delta x})^2 - V(x_0)-(\delta x)^2 \frac{1}{2}V''(x_0)
$$
and the resulting EOM is the so-called linearized equation of motion:
$$
m\ddot{(\delta x)}= -V''(x_0)\delta x\, .
$$
Thus, if $V''(x_0)>0$, the solution is a harmonic oscillation of the type
$$
\delta x(t)= A\cos(\omega t+\varphi) \tag{2}
$$
This is the stable case in the sense that, if the particle starts "close" to $x_0$, it will always remain "close" to $x_0$.
If $V''(x_0)<0$, then the solution is exponential:
$$
\delta x(t) =Ae^{\lambda t} + Be^{-\lambda t}\, . \tag{3}
$$
This is unstable as a particle starting near $x_0$ will runaway exponentially fast never to return close to $x_0$.
If $V''(x_0)=0$, then we cannot decide: this is the neutral case and we must expand $V(x)$ beyond the second order term in $\delta$ in (1).
The same argument holds if $V=V(x,y)$. If $(x_0,y_0)$ is a critical point of $V$, then expand $V$ to second order. The resulting equations of motions for $\delta x$ and $\delta y$ are typically linearly coupled, i.e. they are of the form
$$
\left(\begin{array}{c}
\ddot{\delta x}\\
\ddot{\delta y}\end{array}\right)=
\left(\begin{array}{cc}
\alpha & \beta \\
\beta &\gamma \end{array}\right)\left(\begin{array}{c}
{\delta x}\\
{\delta y}\end{array}\right) = M\left(\begin{array}{c}
{\delta x}\\
{\delta y}\end{array}\right) \, . \tag{4}
$$
The numbers $\alpha,\beta,\gamma$ are related $V_{xx}(x_0,y_0), V_{xy}(x_0,y_0)$ and $V_{yy}(x_0,y_0)$.
To uncouple these linear equations, one finds the linear combinations of $\delta x$ and $\delta y$ that make the matrix $M$ diagonal. This amounts to find eigenvectors $\delta X$ and $\delta Y$ in terms of $\delta x$ and $\delta y$. If $D_M$ is the diagonal form of the matrix $M$, the uncoupled equations of motion are of the form
$$
\left(\begin{array}{c}
\ddot{\delta X}\\
\ddot{\delta Y}\end{array}\right)=
\left(\begin{array}{cc}
A& 0 \\
0 &G \end{array}\right)\left(\begin{array}{c}
{\delta X}\\
{\delta Y}\end{array}\right) = D_M\left(\begin{array}{c}
{\delta X}\\
{\delta Y}\end{array}\right) \, . \tag{5}
$$
where $A$ and $G$ are the eigenvalues of $M$. The stability character of the motion for $\delta X$ and $\delta Y$ can be studied as per the 1d motion example.
Clearly, if the eigenvalues $A$ and $G$ are both negative, $\delta X$ and $\delta Y$ will have the form of (3) and the particle will remain always close to its initial position: the critical point is stable.
If one or both of the eigenvalues is positive, then one or both of $\delta X$ and $\delta Y$ will have an exponential solution and the particle will NOT remain close to its initial position: it will run away exponentially in at least one direction.
If one or both eigenvalues are $0$, we can't decide since there is no guarantee that the particle will remain close to its initial position.
The case of $3$ or more degrees of freedom is treated by immediate generalization of the 2d case.
Best Answer
You gave an expression for the force, whereas Prathyush's answer treats it as the potential. If that expression really is the force, then the answer is different.
You don't really need to do any calculation in this case to see what is going on. Clearly $F(0) = 0$, so this is indeed a point of equilibrium (the question should probably read something like "the particle is placed at rest at the point $x=0$...", to be unambiguous). To check stability, think about displacing the particle slightly in one direction or the other. If $x$ is very close to zero, then $|x^4| \ll |x^3|$, so we can safely ignore the second term, and consider $F \sim -a x^3$. So if $x$ is small and positive, the force pushes the particle in the negative direction, and vice-versa; $x=0$ is therefore stable.