It all boils down to what is meant by "electrically neutral".
In fact, the wire is almost electrically neutral, in that the amount of negative charge is actually extremely close to the amount of positive charge, but not quite, which results in the observed electric field.
From the above electric field formula and the total potential, you can calculate the charge in the wire, and compare it to the total amount of electrons in such a wire. You will find that it is much much smaller.
This charge imbalance is indeed due to the battery, which pushes electrons around. The "load" at the other end of the wire, which closes the circuit, resists this electron motion, meaning that electrons "build up" upstream of the load up to such amount that corresponds to the given $V_0$. If you increase the load resistance, for a given $I_0$ more electrons will pile up more upstream corresponding to a higher $V_0$; if the battery cannot supply the necessary (electromotive) force the current will decrease instead.
You can also directly understand this from Maxwell's equations. An electric field can only be produced by :
A time-varying magnetic field, per Faraday's law (no such thing in your coaxial cable example, hence the electric field must be curl-free)
A charge imbalance, per Gauss's law (that's the case here)
Again, the key here is that the amount of electrons in any real-life object is HUGE, even a minute charge imbalance leads to a strong electric field. Conversely, the energy needed to strip all electrons from any significant amount of matter is ridiculously large, therefore it never happens.
About the solenoid
If the current in the solenoid is time-varying, then Faraday's law of induction applies and the electric field comes from the variation of the magnetic field induced by the current. If the current in the solenoid is constant, there is no induction, $\nabla \times E = 0$, and any electric field will come (as above) from the charge "surplus" needed to push the electrons in the solenoid wire with finite resistance. For a proper copper solenoid, resistance is very small, the necessary potential difference is tiny, and the corresponding electric field along the solenoid is negligible.
Best Answer
Inside the solenoid:
The field is $$B(t)=\mu_0ni(t)$$
So for $r<a$ the flux is $$\phi_B=\mu_0ni(t)*\pi r^2$$
$$\oint E\,ds=\frac{d\phi_B}{dt}$$ $$E2\pi r=\mu_0nk\pi r^2$$ $$E=\frac{\mu_0nkr}{2}$$
Outside the solenoid:
$$B=0$$ So the magnetic flux is only due to the field inside the solenoid: $$\phi_B=\mu_0ni(t)*\pi a^2$$
$$\oint E\,ds=\frac{d\phi_B}{dt}$$ $$E2\pi r=\mu_0nk\pi a^2$$ $$E=\frac{\mu_0nk a^2}{2r}$$