A uncharged capacitor C is connected to a battery with potential V
Since the voltage across a capacitor must be continuous (for finite current), one cannot specify this and expect to get valid answers.
Remember, for the ideal capacitor
$$i_C = C \dfrac{dv_C}{dt}$$
But specifying that an "uncharged capacitor C is connected to a battery with potential V" is equivalent to specifying
$$v_C(t) = Vu(t)$$
But this is equivalent to specifying
$$i_C(t) = CV\delta(t)$$
which certainly ejects this problem outside of the realm of ideal circuit theory.
Now, there is an easy remedy for this - place a resistor in series with the capacitor and then the capacitor voltage becomes
$$v_C(t) = V(1 - e^{-\frac{t}{RC}})u(t)$$
and the current is finite
$$i_C(t) = \frac{V}{R}e^{-\frac{t}{RC}}u(t)$$
Now, it is straightforward to show that the energy stored in the fully charged capacitor is less than the energy supplied by the battery with the difference being the energy dissipated by the series resistor.
Finally, $R$ cannot be reduced to zero - one can show that there is a fundamental radiation resistance associated with the circuit such that energy from the battery can be radiated away during charging.
My question is: What is the work done by the the battery?
When the polarity is reversed, the capacitor will initially discharge, doing work on the battery, until fully discharged and then the battery will again begin doing work on the capacitor.
Since there is loss during the charging / discharging process, one cannot equate the work done by the battery to the work done on the capacitor.
Perhaps the problem is to be solved assuming that the capacitor
connected to a battery gets approximately a charge CV(in negligible
time).
Including a series resistor allows one to compute the energy dissipated by the resistor which turns out to be independent of the resistance and equal to
$$W_R = \frac{CV^2}{2}$$
As an answer to your second question.
If you released the dielectric it would reach the equilibrium position and then overshoot as it would have kinetic energy eventually stopping on the other side and then returning.
Compare this with the oscillation of a spring-mass system.
So the dielectric would undergo oscillatory motion and if damped it would eventually stop at the equilibrium position with some of the energy supplied by the battery dissipated as heat and the rest stored in the capacitor.
Best Answer
Suppose that you double the separation of the plates of a parallel plate capacitor of initial capacitance $C$ which is connected to a battery of emf $V$.
The capacitance of the capacitor become $\frac C 2$ and the energy stored in the capacitor changes from $\frac 12 CV^2$ to $\frac 1 2 \frac C 2 V^2$.
There is a decrease in the energy stored in the capacitor of $\frac 1 4 CV^2$.
Originally the capacitor had a charge $Q= CV$ but after increasing the separation of the plates at constant $V$ the charge stored is $\frac Q 2$.
The work done in moving the charge $\frac Q 2$ through the battery is $\frac Q 2 V = \frac 1 2 CV^2$.
The missing $\frac 1 2 CV^2 - \frac 1 4 CV^2 = \frac 1 4 CV^2$ is the work done by an external force (you?) in separating the capacitor plates.
If there was resistance in the circuit then the external force would have to do more work to move the charge $\frac Q 2$ around the circuit which would manifest itself as the resistor heating up.