It comes from the cross product. Every situation in which you have to use the right-hand rule corresponds to some mathematical equation that involves a cross product.
In this case, the relevant equation is the Biot-Savart law,
$$\vec{B} = \frac{\mu_0}{4\pi}\int \frac{I\,\mathrm{d}\vec{l}\times\vec{r}}{r^3}$$
If you use the right-hand rule - the version you know to use for cross products - to compute the cross product of $\mathrm{d}\vec{l}$ and $\vec{r}$, like this (apologies for the crude drawing):
you will get the same result as from the curling-fingers form of the right-hand rule that you've shown in your question. That's by design, and in fact the curling-fingers form of the rule is just a shortcut for an infinite number of applications of the cross-product form.
In case you're curious, there is a deeper reason the right hand rule is needed for cross products. When you take the cross product of a vector and another vector, you get a slightly different mathematical object called a pseudovector or axial vector. The magnetic field at a point is the best-known example of a pseudovector. Despite looking just like a vector, pseudovectors actually represent a magnitude and an oriented plane, whereas an ordinary vector represents a magnitude and direction. Now, if you have a plane, and you want to represent it with an arrow, in a sense you can do that by picking the arrow to be perpendicular to the plane, and then your convention is that an arrow represents the plane that is perpendicular to it. But there are two arrows perpendicular to the plane; how do you choose which one to use? That's where the right-hand rule comes in. It plays a role in mapping a pseudovector (the thing that e.g. magnetic field really is) to a vector (the thing we use to represent e.g. magnetic field).
Note that a current carrying wire produces a circular magnetic field that's why it doesn't matter how you hold your hand ie how you rotate your hand around your arm as long as your thumb shows the direction of the current.
Edit after comments:
See the illustration I've added below. Now use your hand in the way that you've learned and convince yourself that what I've drawn below is correct.
Best Answer
When current flows in a circle, we can use the right hand rule to find that the magnetic field points in one direction inside the loop, and in the other direction outside the loop (to be precise, the magnetic field lines wrap around the wire, as shown in the second diagram below). Thus, in the case of current carrying loops, if we curl the fingers of our right hand in the direction of the current flow, our right thumb will point in the direction of the loop's magnetic dipole moment, which is also the direction of the magnetic field at the center of the loop. The two pictures below show a current carrying loop in a top-down view and a side cross-sectional view. Hope they help. Thus, this other right hand rule is derived from the first one, it's just a quick short cut we can use for current carrying loops (or spinning charges).