You are correct in your assertion that pairs of charged point particles can interact magnetically in ways that seemingly violate Newton's 3rd law, and therefore also seem to violate the conservation of both linear and angular momentum. This is a fundamental result and it is the decisive (thought) experiment which forces us to change our viewpoint on electrodynamics from something like
charged particles interact with each other
to a field-based one that says
charged particles interact with the electromagnetic field.
What this means, and the key point here, is that
- the electromagnetic field should be considered as a dynamical entity of its own, on par with material particles, and it can hold energy, momentum, and angular momentum of its own.
The linear and angular momentum of the complete dynamical system, which includes the particles and the field, is indeed conserved. This means that in a situation like your diagram, where there is a net force and torque on the mechanical side of the system (i.e. the particles), there are corresponding and opposite net forces and torques on the electromagnetic field.
So, how much linear and angular momentum are there? This is a solid piece of classical electrodynamics: these momenta are 'stored' throughout space, with densities
$$
\mathbf g =\epsilon_0 \mathbf E\times\mathbf B
$$
and
$$
\mathbf j =\epsilon_0\mathbf r\times\left( \mathbf E\times\mathbf B\right),
$$
respectively. Once you account for these, it follows from Maxwell's equations and the Lorentz force that, for an isolated system, the total momenta are conserved. The details of the calculation are a bit messy, and so are the actual conservation laws; I gave a nice derivation of the linear momentum one in this answer.
Conservation of angular momentum really is a new phenomenon, one that does not follow from the Newtonian mechanics you already know; therefore it deserves its own place as a law. Specifically, you have proven that
If a system experiences no torque, then its angular momentum is conserved.
However, this statement by itself is useless. Maybe all systems always experience torque; maybe a system can exert a torque on itself. What we really want to say, i.e. the actual law of conservation of angular momentum, is more like
An isolated system's angular momentum is conserved.
To see how these are not equivalent, suppose we have a system of two isolated particles, one above the other. Newton's third law does not forbid the particles from pushing left and right on each other. But then the system will begin spontaneously rotating! It changes its own angular momentum by exerting a torque on itself.
To force conservation of angular momentum, we need to use the strong form of Newton's third law,
Forces between particles come in action/reaction pairs, and these forces are directed along the line of separation between the two particles.
This is a fundamentally new assumption, so angular momentum really is its own thing. On a deeper level, conservation of linear and angular momentum follow from translation and rotational symmetry of space, and it's possible to have spaces which only are translationally symmetric, or only are rotationally symmetric. The two are independent.
Best Answer
If a (mechanical) system is invariant under arbitrary spatial translations in a certain direction (say, the $x$-direction), then the corresponding component of the momentum ($p_x$ in our case) is conserved (i.e. time-independent).
Analogously, if a system is invariant under arbitrary rotations with repect to a certain axis (say, the $z$-axis), then the corresponding angular-momentum component ($L_z$ in this case) is a constant of motion.
Likewise, if the system is time-translation invariant, energy is conserved.
As an example, consider the motion of a particle described by the Lagrangian function $$L(x,y,z,\dot{x},\dot{y},\dot{z})=m(\dot{x}^2+\dot{y}^2+\dot{z}^2)/2-V(z), \tag{1} \label{1}$$ where the potential depends only on the coordinate $z$. The system is obviously invariant under $x\to x+x_0$, $y\to y+y_0$ for arbitrary $x_0, y_0 \in \mathbb{R}$. As a consequence, the momenta $p_x=m\dot{x}$ and $p_y=m \dot{y}$ are conserved. This can be seen explicitly from the equations of motion, $$\begin{align} \frac{d}{dt} \frac{\partial L}{\partial \dot{x}} &= \frac{\partial L}{\partial x} \quad \Rightarrow \quad \frac{d}{dt} (m \dot{x}) = 0, \tag{2} \label{2}\\[5pt] \frac{d}{dt}\frac{\partial L}{\partial \dot{y}} &= \frac{\partial L}{\partial y} \quad \Rightarrow \quad \frac{d}{dt}(m\dot{y})=0. \tag{3} \label{3} \end{align}$$ Because of the $z$-dependence of the potential, $p_z=m\dot{z}$ is not conserved: $$\begin{align}\frac{d}{dt} \frac{\partial L}{\partial \dot{z}}&=\frac{\partial L}{\partial z} \quad \Rightarrow \quad \frac{d}{dt}(m \dot{z}) =-V^\prime(z) \ne 0. \tag{4} \label{4} \end{align}$$ As the Lagrangian \eqref{1} is also invariant under rotations around the $z$-axis by an arbitrary angle $\alpha$, $$ x \to x\cos \alpha + y\sin \alpha , \quad y \to -x\sin \alpha +y \cos \alpha , \quad z \to z, \tag{5} \label{5} $$ the angular momentum $L_z= m (x \dot{y}-y\dot{x})$ is conserved, which can be checked by taking the time-derivative of $L_z$ and using \eqref{2} and \eqref{3}. On the other hand, neither $L_x=m(y \dot{z}-z \dot{y})$ nor $L_y=m(z \dot{x}-x \dot{z})$ are conserved, as the system is not invariant under rotations around the $x$- or $y$-axis.
Finally, as \eqref{1} is invariant under $t \to t+t_0$ for arbitrary $t_0 \in \mathbb{R}$, the energy of the system, given by $$ E= \dot{x} p_x+\dot{y} p_y+\dot{z}p_z-L=m(\dot{x}^2+\dot{y}^2+\dot{z}^2)/2+V(z), \tag{6} \label{6}$$ is also a constant of motion.