I found in an italian book (Barone) that using special relativity precession is 1/6 of the 43''/century observed. The book has merit to treat an argument usually ignored (orbit precession in special relativity) but I find the treatment very concise so I tried to be more explicit and I connect result to a specific initial condition. I find delightful this problem and I find charming that this strange analysis, despite it doesn't give the correct value, hit squarely the order of magnitude.
Suppose to have in the origin of axes the Sun, and consider a planet, with negligible mass, with initial position and speed like in this figure:
STEP 1: conservation of angular momentum in relativity
As it happens in classical mechanics, we can define $ \mathbf M = \mathbf r \times \mathbf F $ and $ \mathbf L = \mathbf r \times \mathbf p $, finding that $ \dot {\mathbf L} = \mathbf M$ (why this important subject is ignored in relativity books? This equation works in relativity too, because in relativity too force is time derivative of momentum, that has the same direction of speed).
STEP 2: write in a different way the relativistic angular momentum
Using $\mathbf p = \gamma m \mathbf{v}$, speed in polar coordinates ($ \mathbf{v} = \dot{r} \hat{\mathbf{r}} + r \dot{\theta} \hat{\mathbf{\theta}}$: I use hat for versors but I can't bold greek letters) and writing vector position as $\mathbf{r} = r \hat{\mathbf{r}}$, we can write relativistic angular momentum in this way
$$
\mathbf L = \frac{m r^2 \dot{\theta}}{\sqrt{1-\frac{v^2}{c^2}}} \mathbf{\hat z}
$$
where $\mathbf{\hat z} = \hat{\mathbf{r}} \times \hat{\mathbf{\theta}}$ (sometimes called $\mathbf k$). Now observe that
$$
\dot{r} = \bar r \dot{\theta}
$$
where we used the notation $\bar r = \frac{dr}{d\theta}$.
So we have that $v^2 = \dot{r}^2 + r^2 \dot{\theta}^2$ can be written in this way
$$
v^2 = (\bar r^2 + r^2) \dot{\theta}^2
$$
This allows us to write $|\mathbf L|$ in this way
$$
L = \frac{m r^2 \dot{\theta}}{\sqrt{1-\frac{(\bar r^2 + r^2) \dot{\theta}^2}{c^2}}}
$$
STEP 3: write differential equation that rules the orbit
From the equation above we have
$$
\dot{\theta}^2 = \frac{1}{\frac{\bar r^2 + r^2}{c^2} + \frac{m^2 r^4}{L^2}}
$$
Substituting this into conservation of relativistic energy, that using $v^2 = (\bar r^2 + r^2) \dot{\theta}^2$ can be written is this way
$$
E = V + \frac{m c^2}{\sqrt{1-\frac{(\bar r^2 + r^2) \dot{\theta}^2}{c^2}}}
$$
we get, after some calculus,
$$
E = V + mc^2 \sqrt{1+ \frac{L^2 (\bar r^2 + r^2)}{m^2 r^4 c^2}}
$$
From wich we get this differential equation
$$
\left( \frac{dr}{d\theta} \right)^2 + r^2 = \frac{r^4}{L^2 c^2}
\left[ (E-V)^2 - m^2 c^4 \right]
$$
Now we "just" have to solve it.
STEP 4: simplify differential equation
Sobstituting $u=\frac{1}{r}$ (notice that $\frac{dr}{d\theta} = - \frac{1}{u^2} \frac{du}{d\theta}$) into the DE obtained, and multiplying by $u^4$, we get
$$
\left( \frac{du}{d\theta} \right)^2 + u^2 = \frac{1}{L^2 c^2}
\left[ (E-V)^2 - m^2 c^4 \right]
$$
Our potential energy is $V=\frac{-\alpha m}{r}$ (where $\alpha = G M_\odot$), so
$$
\left( \frac{du}{d\theta} \right)^2 + u^2 = \frac{1}{L^2 c^2}
\left[ (E+\alpha m u)^2 - m^2 c^4 \right]
$$
Do derivative in $\theta$:
$$
2 \frac{du}{d\theta} \frac{d^2 u}{d\theta^2} + 2 u \frac{du}{d\theta} = \frac{1}{L^2 c^2}
\left( 2 \alpha^2 m^2 u \frac{du}{d\theta} + 2 E \alpha m \frac{du}{d\theta} \right)
$$
Divide by $2 \frac{du}{d\theta}$ and rearrange in this way:
$$
\frac{d^2 u}{d\theta^2} + u = \frac{\alpha m}{L^2 c^2} (E + \alpha m u)
$$
You can check that this equation can be written in this way
$$
\frac{d^2 u}{d \theta^2} + q^2 u = \frac{q^2}{p}
$$
where
$$
q = \sqrt{1-\frac{\alpha^2 m^2}{L^2 c^2}}
$$
and
$$
p = \frac{L^2 c^2 - \alpha^2 m^2 }{\alpha m E}
$$
Then, with the substitution $w = u - \frac{1}{p}$ we get (notice that $\frac{d^2 u}{d \theta^2} = \frac{d^2 w}{d \theta^2}$ because $p$ is constant)
$$
\frac{d^2 w}{d \theta^2} + q^2 w = 0
$$
STEP 5: the shape of the orbit
Write in this way the solution of the DE we obtained
$$
w = A \cos \left[ q (\theta - B) \right]
$$
where $A$ and $B$ are arbitrary constants. Doing backwards sobstitutions we done ($u=w+\frac{1}{p}$ and $r=\frac{1}{u}$) we get
$$
r = \frac{p}{1+e \cos \left[ q (\theta - B) \right]}
$$
where we called $e$ the $Ap$ constant. Imposing the initial condition of the figure ($\mathbf{r}_0 = r_0 \hat{\mathbf{x}} $ and $ \mathbf{v}_0 = v_0 \hat{\mathbf{y}}$), implies that at $t=0$ ($\Rightarrow$ the starting $ \theta = 0$) we have $\frac{dr}{d\theta} = 0$ (we can simply put $B=0$) and $r=r_0$ (so $e=\frac{p}{r_0}-1$)
$$
r (\theta) = r_0 \frac{ \frac{p}{r_0} }{1 + \left( \frac{p}{r_0} -1 \right) \cos (q \theta)}
$$
where (to connect $p$ and $q$ to initial value, we exploited the fact that at $t=0$ angular momentum is $L=\gamma_0 m r_0 v_0$, while energy is $E = \gamma_0 mc^2 - \frac{\alpha m}{r_0}$: we only have to substitute this constants $E$ and $L$ into $q$ and $p$ previously written)
$$
q = \sqrt{1-\frac{\alpha^2 (c^2 - v_0^2)}{c^4 r_0^2 v_0^2}}
$$
and
$$
p = \frac{
\frac{r_0^2 v_0^2 c^4}{c^2 - v_0^2} - \alpha^2
}{
\alpha \left( \frac{c^3}{\sqrt{c^2 - v_0^2}} - \frac{\alpha}{r_0} \right)
}
$$
If $c \to \infty$ we get $q \to 1$ and $p \to \frac{r_0^2 v_0^2}{\alpha}$: we have the classical elliptical orbit, as it must to be.
In the ordinary situations, the relativistic effect is very small, but we can see graphically that this is a rosetta orbit by changing artificially $c$ value to $2 v_0$. In the case of Mercury (in SI unit we will assume $|\mathbf r_0| = 4{,}6 \cdot 10^{10} $ and $|\mathbf v_0| = 5{,}9 \cdot 10^{4}$) we obtain the orbit in figure (I stopped the orbit after 21$\pi$ radians, unit in axes is $10^6$ km)
STEP 6: estimate of the precession
Observing $r (\theta)$ we see that distance is minimum when $\cos (q \theta) = 1$, i.e. $\theta$ is
$$
\theta_k = \frac{2\pi}{q} k \qquad \qquad k \in \mathbb{Z}
$$
Between a minimum and the next minimum $\theta$ change of $\theta_{k+1} - \theta_k = \frac{2\pi}{q}$. Subtracting $2\pi$ we get the angular distance between two minima: $\Delta \theta = 2 \pi (q^{-1}-1)$. Substituting $q$ we find
$$
\Delta \theta = 2 \pi \left[
\left(1-\frac{\alpha^2 (c^2 - v_0^2)}{c^4 r_0^2 v_0^2} \right)^{-\frac{1}{2}} - 1
\right]
$$
But if $\epsilon \ll 1$ works $(1-\epsilon)^{-\frac{1}{2}} \approx 1+ \frac{\epsilon}{2}$, so if $\frac{\alpha^2}{c^2 r_0^2} \ll v_0^2 \ll c^2 $ we can use this simpler and delightful formula
$$
\Delta \theta = \frac{K}{(v_0 r_0)^2} \qquad \qquad \left( K=\frac{\pi G^2 M_\odot^2}{c^2} \approx 6{,}160 \cdot 10^{23} \, \textrm{m}^4 / \textrm{s}^{2} \right)
$$
For Mercury this formula is exploitable and gives about $0{,}017''$/revolution. Mercury revolution lasts 88 days so this give about $7''$/century. I explored the case in which gravitational mass of orbiting planet grows with $\gamma$ factor, it was a tour de force but I solved numerically finding that, at least fot almost circular and weakly relativistic orbit, $\Delta \theta$ simply doubles (1/3 of the observed angle). Intriguing but still we are far from the true $43''$/century.
Best Answer
You are essentially correct to assume that the precession is sometimes backwards as well. The frequently quoted value of 531"/cy is only the long term net average. In reality there is a considerable oscillatory component added to this, as illustrated in the following schematic figure
The oscillation period here is essentially the orbital period of the disturbing planet. The effect is exaggerated here just to demonstrate it. If you do the same calculation with the actual data for all planets, you get the following graph for Mercury's Newtonian perihelion precession
Although the increase of the perihelion angle is now closer to a linear increase, there is still a considerable wobble, and at times it is going backwards for a few years. The two main periods in the oscillation visible here are those of Jupiter and Venus, which have the largest impact on the precession (Jupiter due to its mass and Venus due to its proximity)
For more details see the web page Mercury's Perihelion Advance from which also the above figures were taken
EDIT: Let me add a qualitative explanation of what is happening here:
an outer planet in opposition to Mercury (i.e. on the same side of the Sun) will result in a small force pulling it away from the Sun. So if Mercury is moving away from the Sun again after its closest approach (perihelion) the gravitational force of the Sun will effectively be reduced slightly and it takes longer to bring the radial velocity back to zero again. This means Mercury will go slightly further in its orbit until it has reached its furthest point (aphelion) i.e. a positive precession. If however the disturbing planet is on the opposite side of the Sun, its gravitational field will effectively increase the gravitational field of the Sun, so the effect will be the other way around, leading to a negative (backwards) precession. In the latter case the planet is further away though than in opposition, so the effect is smaller, leaving an overall positive (forward) precession on average. The net result can therefore be seen as an oscillatory precession angle (related to the time dependent orbital motion of the disturbing planet) superposed on a linearly increasing precession angle related to the time averaged mass distribution of the planet (which is effectively a circle/ellipse and thus always produces an outward directed force i.e. a positive precession)