The magnetic moment of the electron is a magnetic moment, so the right magnetic field around it is
$$ \mathbf{B}({\mathbf{r}})=\nabla\times{\mathbf{A}}=\frac{\mu_{0}}{4\pi}\left(\frac{3\mathbf{r}(\mathbf{\mu}\cdot\mathbf{r})}{r^{5}}-\frac{{\mathbf{\mu}}}{r^{3}}\right). $$
The world is quantum mechanical – and so is any viable description of the spin – so we have to respect the postulates of quantum mechanics. In particular, the magnetic field above corresponds to a "state" (e.g. spin up and spin down) and one may construct complex linear superpositions of such states. It is important to realize that the quantum mechanical superposition of states in the Hilbert space in no way implies that the corresponding magnetic fields are being added according to the classical electromagnetic field's superposition principle.
Indeed, they're linear superpositions of states that contain different profiles of the magnetic field.
The magnetic field around the electron is so weak that indeed, it can in no way be thought of as a classical magnetic field, in the sense that the classical fields are "large". But the classical formula for the magnetic field is still right! This formula defines the magnetic moment. The quantum effects are always important, however. Also, if you try to measure this very weak magnetic field, it will unavoidably influence the state of the measured system, including the electron's spin itself.
It is of course completely wrong to imagine that we could measure such a weak magnetic field by a big macroscopic apparatus, like a fridge magnet. The effect of one electron's magnetic field on such a big object would be nearly zero, of course. In fact, quantum mechanics guarantees quantization of many "phenomena", so instead of predicting a very tiny effect on the fridge magnet, it predicts a finite effect on the fridge magnet that occurs with a tiny probability.
You may "measure" the electron's magnetic field by creating a bound state with another magnet in the form of an elementary particle. For example, the electron and the proton in a hydrogen atom are exerting the same kind of force that you would expect from the usual "classical" formulae – but it's important to realize that all the quantities in the equations are operators with hats.
Let me show you an example of a simple consistency check implying that there is no contradiction. Calculate the expectation value of the magnetic field (an operator) $\vec B(\vec r)$ at some point for the state $c_{up}|up\rangle + c_{down} |down\rangle$. The column vector of the amplitudes $c$ is normalized. The check is that you get the same expectation value of $\vec B(\vec r)$ for each $\vec r$ if you first compute it for the "up" component and "down" component separately, and then you add the terms, or if you first realize that it's a spin state "up" with respect to a new axis $\vec n$, and compute $\vec B$ from that.
It's a nice exercise. The point is that the expectation value of $\vec B(\vec r)$ is a bilinear expression in the bra-vector and ket-vector $|\psi\rangle$, much like the direction $\vec n$. And indeed, $\vec B$ is linear in the direction $\vec n$, according to the formula above, so things will agree. You may insert anything else to the expectation value, in fact, so the check works for all linear expressions in $\vec B$. The higher powers of $\vec B$ also have expectation values but they will behave differently than in classical physics because there will be extra contributions from the "uncertainty principle", analogous to zero-point energies of the harmonic oscillator in quantum mechanics.
It's extremely important to realize that the field $\vec B(\vec r)$ is also an operator – so it has nonzero off-diagonal matrix elements with respect to the "up" and "down" spin states of the electron. In fact, if you just write $\vec \mu$ in the formula for the magnetic field I started with as a multiple of the Pauli matrices (electron's spin), you will exactly see how the "key" term in the magnetic field behaves with respect to the up- and down- spin states. The off-diagonal elements do not contribute to the expectation value in the up state or in the down state but they do impact the "mixed matrix elements between up and down, and those affect the expectation values in spin states polarized along non-vertical axes.
BTW I added a semi-popular blog version of my answer here
http://motls.blogspot.com/2014/07/does-electrons-magnetic-field-look-like.html?m=1
Classically, the magnetic potential energy is $U=-\boldsymbol{\mu}\cdot\textbf{B}$. Energy is conserved. Therefore if there is no mechanism for dissipation or for exchanging potential energy for kinetic energy, the angle between the two vectors has to stay the same. If you take a bar magnet and hang it from a string, then it can precess in the earth's field, or just keep its original orientation, or oscillate, depending on the initial state of rotation. If it's oscillating, then there is an interchange of kinetic and potential energy. In a hiking compass, it will oscillate, but there is strong damping because it's water filled, so the energy is dissipated rapidly into heat, and the needle ends up aligned with the field.
Quantum-mechanically, there really isn't any concept of precession for a microscopic body. For example, suppose an electron is in a magnetic field that's in the $z$ direction. Say we put the electron in a state of definite energy. Since there is no dissipative mechanism and no way to exchange PE with KE, the electron must have constant $\boldsymbol{\mu}\cdot\textbf{B}$, i.e., $\mu_z$ stays constant. But in this situation $\mu_x$ and $\mu_y$ are completely uncertain, so it doesn't make sense to imagine the moment as a vector that's precessing.
Best Answer
The magnetic moment $\vec{\mu}$ of a charge configuration is defined by the torque $\vec{\tau}$ it feels when being in an external magnetic field $\vec{B}$ $$\vec{\tau}=\vec{\mu}\times\vec{B}$$ or equivalently by its potential energy $U$ when being in this external field $\vec{B}$ (see Magnetic Moment - Effects of an external magnetic field) $$U=-\vec{\mu}\cdot\vec{B}.$$
This definition is used both for classical and for quantum-mechanical systems.
The difference begins when you want to get a relation between magnetic moment $\vec{\mu}$ and angular momentum. For orbital momentum $\vec{L}$ you have (both classically and quantum-mechanically) $$\vec{\mu} = g\frac{q}{2m}\vec{L}\quad\text{, with } g=1$$ which can be derived theoretically and confirmed experimentally (by measuring the torque or energy).
But for spin angular momentum $\vec{S}$ of the electron experiments show $$\vec{\mu} = g\frac{q}{2m}\vec{S}\quad\text{, with } g=2.0023$$ which is roughly double the size than for orbital momentum. This cannot be understood by classical mechanics. But from Pauli's equation (i.e. with non-relativistic quantum mechanics) or from Dirac's equation (i.e. with relativistic quantum mechanics) you can derive this formula with $g=2$. And with the full theory of quantum electrodynamics you can even derive it with the exact value $g=2.0023$ (see $g$-factor).