Consider the following three parts of the sphere: let $P_A$ be the lune created from the triangle $ABC$ plus the triangle adjacent across the $BC$ line segment, plus the opposite lune (on the opposite side of the sphere), and similarly for parts $P_B$ and $P_C$.
The area of $P_A$ is $4\alpha R^2$: the total area of the sphere is $4\pi R^2$, and the area of $P_A$ is certainly proportional to $\alpha$.
Notice now that $P_A\cup P_B\cup P_C$ is the entire sphere, and that $P$'s intersect at the triangle + the opposite triangle. We thus have:
area($P_A$) + area($P_B$) + area($P_C$) = area of the sphere + 2 area of the triangle +2 area of the opposite triangle.
As the two triangles have the same area, you get your formula.
I too would interpret the symbols $\alpha$, $\beta$, and $\gamma$ as angles
between the sides, and $a$ as the arc length of a side.
I know of two sets of formulas that might be called the spherical law of cosines.
You can find both of them on the page for
spherical trigonometry at Wolfram MathWorld.
One set of formulas they call the cosine rule for sides:
$$\cos a = \cos b \cos c + \sin b \sin c \cos\alpha$$
(with corresponding formulas with $\cos b$ and $\cos c$, respectively, on the left).
The other set of formulas they call the cosine rule for angles:
$$\cos\alpha = -\cos\beta \cos\gamma + \sin\beta \sin\gamma \cos a$$
(with corresponding formulas with $\cos\beta$ and $\cos\gamma$, respectively, on the left).
You can easily put any of these formulas in a format like the one you have used;
from the cosine rule for angles you would end up with
$$\cos a = \frac{\cos\alpha + \cos\beta \cos\gamma}{\sin\beta \sin\gamma}.$$
Note the "$+$" operation in this formula where you have "$-$".
It turns out this has no effect on your result, since the right-hand operand
turns out to be zero.
Where you make a misstep is here:
$$\frac{\sqrt{6}+\sqrt{2}}{2}=4\cos(\alpha)$$
$$2(\sqrt{6}+\sqrt{2})=\cos(\alpha)$$
You need to divide both sides of the first equation by $4$, so the second equation
should have been
$$\frac{\sqrt{6}+\sqrt{2}}{8} = \cos\alpha.$$
One way you might notice the error is that $2(\sqrt{6}+\sqrt{2}) > 1$,
meaning it cannot be the cosine of any real-valued angle.
The result is not a "nice" angle like $60$ degrees or $75$ degrees, so
either you give a calculator's approximation or you leave it as an expression
involving $\cos^{-1}$.
You could find the other two sides of the triangle using law of cosines,
or you could use the Law of Sines for a spherical triangle, which is
relatively easy to remember:
$$\frac{\sin\alpha}{\sin a} = \frac{\sin\beta}{\sin b} = \frac{\sin\gamma}{\sin c}.$$
That's actually three equations, two of which allow you to solve for your
unknown sides $b$ and $c$.
Best Answer
$A(0, 0, 1)$, $B(0, 1, 0)$ and $C(\dfrac{1}{\sqrt{2}}, 0, \dfrac{1}{\sqrt{2}})$ with $|A|=|B|=|C|=1$ these point lie on unit sphere. These points specify three plane $x=0$, $y=0$ and $x=z$ then the angle between them are $\dfrac{\pi}{2}$, $\dfrac{\pi}{2}$ and $\dfrac{\pi}{4}$, since their normal vectors are $\vec{i}$, $\vec{j}$ and $\vec{i}-\vec{k}$, respectively (by $\cos\theta=\dfrac{u.v}{|u||v|}=u.v$).
At last $\sigma=\dfrac{\pi}{4}+\dfrac{\pi}{2}+\dfrac{\pi}{2}-\pi=\dfrac{\pi}{4}$.