For forces that change along the way, displacement is not the thing to calculate work with. Let $\gamma : [0,1] \rightarrow \mathbb{R}^3$ be the (closed or open) path that the particle the force is exerted on follows. Then, the work done along that path is
$$ W[\gamma,F] = \oint_\gamma \vec{F}(\vec{x})\cdot \mathrm{d}\vec{x}$$
which is a line integral. If $\vec{F}$ is conservative, there is a function $V(\vec{x})$ such that $\nabla V(\vec{x}) = \vec{F}(\vec{x})$, then we can apply Stokes' theorem (or, less fancy, the fundamental theorem of calculus) to calculate the work by
$$\oint_\gamma \vec{F}(\vec{x})\cdot \mathrm{d}\vec{x} = \int_{\partial\gamma} V(\vec{x}) = V(\gamma(1)) - V(\gamma(0))$$
For closed paths, $\gamma(1) = \gamma(0)$, so this is zero. If there is no potential with $\nabla V = \vec{F}$, we cannot apply this argument and have to actually calculate the line integral, which may be anything, especially not zero.
I'd be much obliged if someone could give me an example of such a
field
Consider a vector field $\vec F$ with non-zero curl in the $z$ direction only:
$$\nabla \times \vec F = \left(\frac{\partial F_y}{\partial x} - \frac{\partial F_x}{\partial y}\right)\hat z $$
An example of such a field is
$$\vec F = -y \hat x + x \hat y$$
which has curl
$$\nabla \times \vec F = 2\hat z$$
so $\vec F$ is non-conservative. By Stoke's theorem, for a closed contour in the $z=z_0$ plane enclosing an area $A$, the line integral along that contour will have the value of $2A$.
Clearly, for plane areas parallel to the $z$ axis, there is zero flux of $\nabla \times \vec F$ through, thus, the closed contour integral of $\vec F$ in such a plane will be zero by Stoke's theorem.
To verify, pick a contour within the plane defined by, e.g., $y = 1$.
Integrate the field along the path from $(0,1,0)$ to $(1,1,0)$ to $(1,1,1)$ to $(0,1,1)$ to $(0,1,0)$
There is no component of $\vec F$ in the $z$ direction so the only contributions are the integrals along the $x$ direction.
$$\int_0^1 (-1)\hat x \cdot \hat x dx + \int_1^0 (-1)\hat x \cdot \hat x dx = -1 + 1 = 0$$
So, this is a simple example of a non-conservative vector field and a closed contour integral in that field that is zero.
Best Answer
You're thinking that an integral over a closed path for any field $\bf F$ will always evaluate to zero. This is incorrect.
However, and by definition, for a force $\bf F$ to be conservative, the line integral $$\oint \bf F\cdot dl= 0$$ If this line integral is not zero, then the force cannot be conservative.
You can also verify that the force is non-conservative by computing the curl$^1$. That is, $$\nabla\times {\bf F}=\nabla\times (0,-x,0)= \begin{vmatrix} \hat i & \hat j & \hat k\\ \frac{\partial}{\partial x} & \frac{\partial}{\partial y} & \frac{\partial}{\partial z}\\ 0 & -x & 0 \end{vmatrix} =-\hat k$$
Any field with a nonzero curl is not conservative. This force is not conservative, hence why you do not get a zero value for the integral.
$^1$ Note that from Stoke's theorem, $$\bf\iint_S(\nabla\times F)\cdot dS=\oint_l F\cdot dl$$ where $S$ is the surface enclosed by the closed path $l$.