This problem can be solved by a conformal inversion, but the answer becomes more difficult when the plates are not infinitely far apart.
Quick sketch
Choose the point of inversion to be the contact point of the plane and the sphere, the constant electric field at the left infinite half-plane becomes a dipole field right at the origin, while the sphere becomes a second parallel plane.
The method of images then produces reflected dipoles with the same orientation, with a uniform spacing along the x axis. The field is given by a superposition of these dipole fields, which allows you to solve for the electric field on the plane corresponding to the inverted sphere, and inverting back gives the charge.
The sum over the n-th image dipole is complicated before integrating for the total charge, but the total charge contributed by the n-th image on the right (plus its partner on the left) surprisingly falls off as $1/(n+1)^2$, making a sum which can be expressed completely in terms of $\zeta(2)={\pi^2\over 6}$.
The answer
The charge induced on the sphere when the plates are infinitely far away is equal to
$$ Q= (4\pi \sigma R^2) {\pi^2\over 12}$$
Where $\sigma$ is the initial charge density on the plates. The sphere acquires a charge which is about .82 of the charge it would have if it got a uniform density of charge equal to the density on the plates.
Electrostatic conventions
Normalize the electrostatic Laplace equation so that a point source has a field
$$ Q \over 4\pi r$$
With this convention, you can find the field of a dipole of magnitude D by superposing two opposite charges infinitesimally close, so that the answer is
$$ D x\over 4\pi r^3 $$
A conducting surface in this convention has a surface charge equal to that electric field outside coming in perpendicularly, so I will use $\sigma$ and surface $E$ interchangably.
Conformal inversion
In 3d, a solution $\phi$ to Laplaces equation has conformal dimension 1/2, which means that it can be inverted around the origin about a circle with radius a to give another solution to Laplace's equation:
$$ \phi'(x,y,z) = {a\over r} \phi({a^2 x \over r^2}, {a^2 y \over r^2}, {a^2 z\over r^2}) $$
The multiplying factor in front comes form the linear scale factor of an inversion, which is $a^2/r^2$ at a point a distance r, but with a square root to account for the 1/2 scale dimension of $\phi$.
The best way to prove that inversions are a symmetry is to check that the inversion takes a fundamental solution (1/r potential) to another fundamental solution at a different location. This means that inversion will move source points to source points under inversion, and keep source-free the images of those regions where Laplace's equation is satisfied source-free.
When you have a conductor, you can add a constant to the potential to make it be at zero potential. In this case, it stays a conductor under the inversion formula given above (it is easy to check that surfaces with $\phi=0$ are transformed into new surfaces with $\phi=0$). The inversion of a charged spherical conducting shell is another charged spherical conducting shell, or a conducting plane, in the case that the sphere passes through the origin of the inversion. If you invert a plane in a point on the plane, you get the same plane back.
When you invert a conductor at zero potential, the surface charge distribution is changed. The electric field is still perpendicular to the transformed surface, because a zero-potential surface transforms to a zero potential surface, but the electric field is modified by the 3/2 power of the conformal scale-factor. The reason is that the electric field is the inverse distance between equipotential lines, and the distance is increased by the scale factor, while the potential itself is decreased by the square-root of the inversion factor.
Inverting a sphere in contact with a conductor
Let the radius of the sphere be R, and place the point of contact of the sphere and the plane at the origin. Invert the origin with an inversion radius 2R, and the plane conductor stays the same plane, and the sphere becomes another parallel plane at zero potential at a distance 2R along the x-axis.
The constant electric field E which was at left infinity before inverts into a point dipole of dipole moment $E(2R)^3$ sitting right on top of the origin. The problem is to determine the field of a charge between two conducting plates, touching one of the two plates, making a dipole with its image charge, right at this point.
This problem is simple by images. If you place a dipole with the same orientation at every point on the x axis where $x=2nA$, where n is an integer, it is easy to check that symmetry (and convergence) guarantee that the solution is constant potential on the conducting planes.
This infinite sum, when centered on the inversion image-plane of the sphere, gives the charge density of the surface.
Summing up the image dipoles at the location of the plane gives the surface charge
$$ \sigma(u) = 2E \sum_{n \;\;\;\mathrm{odd}} { u^2 - 2n^2 \over (u^2 + n^2)^{5/2}} $$
where $u={\rho\over 2R}= \tan({\theta\over 2})$, where $\rho$ is the distance from the origin of the plane, and $\theta$ is the angular location on the sphere, with $\theta=0$ the contact point of the sphere and the image plane, and the sum is over all positive odd numbers.
The surface charge on the sphere
If the charge density on the image plane of the sphere at distance $\rho$ from the origin is $\sigma(\rho)$, the charge density on the sphere at any $\theta,\phi$ coordinates (where $\theta=0$ is along the axis linking the origin to the inversion plane) is given by
$$ \sigma(\theta) = {\sigma(\rho)\over cos({\theta\over 2})^3 } $$
and the total charge on the sphere is given by
$$ Q = \int \sigma(\theta) d\cos\theta d\phi = 2\pi R^2 \int \sigma(\theta) d\cos(\theta) $$
This is a consequence of the inversion formula for electric fields.
The resulting infinite sum of integrals is
$$ Q = 4\pi R^2 \sum_{n=1,3,5,...} \sqrt{2} \int_{-1}^1 {(2n^2-1) + (2n^2+1) z \over [(n^2+1) + (n^2-1) z]^{5/2}} dz $$
Where z is $\cos(\theta)$, performing the integral is elementary (but tedious), and is easily done by a version of partial fractions, as follows:
$$ {A + Bz\over (C+Dz)^{5/2}} = {A- {BC\over D}\over (C+Dz)^{5/2}} + {{B\over D} \over (C+Dz)^{3/2}}$$
Where $A=2n^2-1$, $B=2n^2 +1$, $C= n^2+1$ and $D=n^2-1$. Once the integrand is decomposed this way, the elementary integral (including the $\sqrt{2}$ in front) over each part gives
$$ \sqrt{2} {3\over 2} {AD - BC\over D^2} ( {1\over (C+D)^{3/2}} - {1\over (C-D)^{3/2}}) + \sqrt{2} {2B\over D^2} ( {1\over (C+D)^{1/2}} - {1\over (C-D)^{1/2}}) $$
There is a nontrivial check on the algebra in the limit $D\rightarrow 0$, where the pole terms of the two parts must cancel, because there is no real divergence in this limit before the partial-fracton decomposition. Substituting for A,B,C,D is made easier by first noting that $C+D=2n^2$ and $C-D=2$, and the determinant like thing is $AD-BC= -6n^2$. The result simplifies absurdly (it must cancel the pole-parts at n=1, but the cancellation of the pole at n=0 is a surprise)
$$ {2\over (n+1)^2}$$
This essentially finishes the computation. The sum is
$$ \sum_{n=1,3,5,7} {2\over (n+1)^2} = {1\over 2} \sum_{n=1,2,3..} {1\over n^2} = {\pi^2\over 12}$$
And this multiplies the factor $4\pi R^2\sigma$, which is the charge on the sphere if it were uniformly charged with the same density as on the infinite plates to begin with.
Corrections for finite size of capacitor
When the second plate is not infinitely far away, the inversion process produces a small sphere with charge instead of a point dipole. The corrections here can be computed to leading order.
Best Answer
First, note that the electric field outside of any capacitor is not zero. It is zero only for the ideal case of a perfect infinite parallel plate capacitor.
Your inference about the movement of the positive charge is wrong. Yes the potential is higher there than it is at the other plate, but that is not enough to cause a force on the charge. What is needed is a gradient of the potential. $$F=-\frac{\mathrm{d}U}{\mathrm{d}x} = -q\frac{\mathrm{d} V}{\mathrm{d}x}$$ If the potential is strictly zero, then the gradient is also zero, and there is no force.
Update after comment
A charged particle does not respond to its own field, and hence it does not feel its contribution to potential. The positive particle will feel only the fields due to the capacitor plates. The field outside is zero, and the potential that the charge feels is constant.