Imagine a free-standing battery (not connected to any wires) and take a closed loop through the battery, out one terminal, and back in the other terminal. The total work done in moving a test charge around that loop must vanish. For this to happen, the change in electric potential outside of the battery must equal the negative of the EMF change within the battery. $$\int_\mathrm{outside}\vec{E}\cdot{\mathrm{d} \vec{\ell}} + \mathcal{E} = 0$$
Update after comments
Work is well defined as the integral of force over distance. The relationship between work and energy is more subtle. One needs to carefully define what the system in question is. We also have to recognize that potential energy is the energy associated with the configuration of a system of interacting entities. One is on thin ice if one refers to "the potential energy of a particle". Particles do not have potential energy. The system comprising the particle and something with which it interacts has potential energy. (A ball does not have potential energy. The earth-ball system has potential energy.) I'll review the background on this, with apologies if the background is already well-understood.
Once a system is defined, energy can be added to the system by an external force which can do external work on the system. Work is one way energy can be added. Heat is another, but we'll mostly ignore heat and thermal energy. Generally, $$W_\mathrm{external} = \Delta E$$ where $E$ is the total energy of the system. External work causes energy to be added to the system, but once inside that energy could be potential, kinetic, thermal, chemical ...
Potential energy is defined to be the negative of the work done by forces internal to the system:$$W_\mathrm{internal} = -\Delta PE$$
Now our system. Let's take it to be the wire, the battery terminal, the conductors inside the battery, but not the chemicals and processes that generate the "chemical force". The chemical processes are a source of energy, so we'll take it to be outside of our system. The work done by the chemical processes are external, and do external work on the charge carriers $$W_\mathrm{external}=q\mathcal{E}$$
But the internal voltage due to the separated charges within the battery, also do work, but this work is internal to our system, and thus changes the potential energy of the system $$W_\mathrm{interal} = -\Delta PE = -qV$$
but recall $$W_\mathrm{external} = \Delta E = \Delta PE = -W_\mathrm{internal}$$ (ignoring stores of energy other than potential energy within the system). Finally $$q\mathcal{E}=qV$$ $$\mathcal{E} = V$$
Yes, Kirchhoff's voltage law (KVL):
Sum of voltage drops across all elements connected via perfect conducting wire in series in to a closed circuit is zero.
is valid for lumped element RLC circuits, so also for inductors (for currents that do not change too fast, so voltage can be measured in practice). In practical circuits designed not to radiate, voltage can be measured across any element and KVL can be validated experimentally. It is valid for common frequencies, up to hundreds of MHz and even higher to GHz range if parasitic elements are added to the model.
The whole theory of RLC circuits with harmonic voltage sources is derived from KVL being valid all the time, while currents and voltages change.
Some people say Kirchhoff's law is not valid for a circuit with an inductor, since $\oint \mathbf E \cdot d \mathbf s \neq 0$ if ideal inductor is in the circuit. However, that is actually not a problem for KVL, because KVL is formulated using voltage drops, not integrals of total electric field. Voltage drop across inductor may be non-zero, even if total electric field in the wire is zero, because the drop is defined not by integral of total electric field, but by integral of electrostatic component of that field.
Best Answer
I mean the basic principle is, you're right, this setup is described by the math of classical electrical circuits as one which should drive an infinite current, and if you only have one charge to move then an infinite current necessitates it moving with infinite speed. If you do not give it a way to lose energy then it will continue to gain energy indefinitely.
Kirchhoff's voltage law can still be applied to such a condition but it is somewhat less informative. The voltage law comes from the fact that there is a scalar potential field $\varphi$ defined over space, it is just one field, it has only one value and no other. It is something like wondering why I cannot gain infinite energy by sledding down a hill: well, whatever speed you are getting on the way down, I can prove that you’re in for an equal-and-opposite trek up. So if this field is well-defined then Kirchhoff’s voltage law holds good.
This field $\varphi$ in turn comes from the Maxwell equations, which in my favorite units look like,$$ \begin{align} \nabla \cdot E &= c\rho & \nabla\times E &= -\dot B\\ \nabla \cdot B &= 0 & \nabla \times B &= J + \dot E \end{align} $$ where $\dot X = \partial X/\partial w = c^{-1} \partial X/\partial t.$
First, the bottom-left one is used to rewrite $B = \nabla \times A$ for some “vector potential” $A$ and then the top-right one says $$\nabla \times (E + \dot A) = 0$$ implying that there exists a scalar potential function $\varphi$ such that $$E + \dot A = -\nabla \varphi.$$ So this is a mathematical theorem, this scalar function $\varphi$ always exists. You can really say that there is a voltage drop of 6 volts from one side of the terminal to the other, and that is fine. Kirchhoff’s voltage law holds everywhere.
But notice what is not said: the connection to a particle’s kinetic energy depends on work which depends on this electric field $E$ and its behavior over closed loops. And $\nabla\varphi$ vanishes over closed loops, but there is no reason that $\dot A$ must, and so a particle can gain/lose energy in a closed loop, if the vector potential is increasing/decreasing in time.
So, normally we can just step back from a circuit and assume that whatever the vector potential $A$ is, the circuit comes to some equilibrium value where it is not changing and $\dot A = 0$. Then $E = -\nabla \phi$ and we know that the thing must have lost all of its energy by the time that it comes back around the loop. That is how we know that the electron has lost its energy.
But, in the case that you are describing, we know that the Liénard–Wiechert potentials $\phi, A$ for a moving charge contain an $A$ which runs parallel to the charge, in rough proportion to its speed. As the charge is going faster and faster, this $A$ is increasing and increasing around the loop, and this factor of $\dot A$ is non-negligible.
Kirchhoff’s rule is not technically wrong, what is wrong is the assumption that energy only changes from a change in potential, that these go hand-in-hand. The Maxwell equations do not say this, they say instead that there is this other thing, this vector-potential $A$, which must change in that scenario.