The main idea is to split each equation into a real and a complex part.
To easily see how to do this take a look at complex multiplication as a linear transformation.
$(a + bi) * (c + di) = (ac - bd) + (bc + ad)i$ will become
$\left(\begin{array}{cc}c&-d\\d&c\end{array}\right)
\left(\begin{array}{c}a\\b\end{array}\right)=
\left(\begin{array}{c}ac-bd\\bc+ad\end{array}\right)$
You can use this pattern to rewrite your example as
$\left(\begin{array}{cccc|c}
25&-100&-10&80&100\\
100&25&-80&-10&0\\
-10&80&30&-190&0\\
-80&-10&190&30&0
\end{array}\right)$
The variables $V_{1},V_{2},V_{3}$ are not real numbers, but complex numbers,
as can be seen by solving the given system of complex equations. It can be
solved algebraically using complex numbers, as commented above by Travis.
Since $\cos 30{{}^\circ}=\frac{\sqrt{3}}{2},\sin 30{{}^\circ}=\frac{1}{2}$, we get from equation 3
\begin{equation*}
V_{3}=V_{2}-10\sqrt{3}-j10.
\end{equation*}
Then equation 2 becomes
\begin{equation*}
\left( -4-\frac{1}{10+j6}\right) V_{1}+\left( \frac{1}{10+j6}+\frac{1}{-2j}+
\frac{1}{8+7j}\right) V_{2}+\left( \frac{1}{-2j}+\frac{1}{8+7j}\right)
\left( -10\sqrt{3}-10j\right) =0
\end{equation*}
or (rewriting the coefficients in the algebraic form)
\begin{equation*}
\left( -\frac{277}{68}+\frac{3}{68}j\right) V_{1}+\left( \frac{1109}{7684}+j
\frac{3027}{7684}\right) V_{2}-\frac{80}{113}\sqrt{3}+\frac{495}{113}
+j\left( -\frac{80}{113}-\frac{495}{113}\sqrt{3}\right) =0.
\end{equation*}
Since $\cos 45{{}^\circ}=\sin 45{{}^\circ}=\frac{\sqrt{2}}{2}$, the equation 1 becomes
\begin{equation*}
\left( \frac{1}{5}+\frac{1}{10+j6}\right) V_{1}-\frac{1}{10+j6}V_{2}-5\sqrt{2
}-j5\sqrt{2}=0
\end{equation*}
or, simplifying the coefficients
\begin{equation*}
\left( \frac{93}{340}-j\frac{3}{68}\right) V_{1}-\left( \frac{5}{68}-\frac{3}{68}%
j\right) V_{2}-5\sqrt{2}-j5\sqrt{2}=0.
\end{equation*}
Solving the equivalent system of equations 1 and 2
\begin{equation*}
\left\{
\begin{array}{c}
\left( \frac{93}{340}-j\frac{3}{68}\right) V_{1}-\left( \frac{5}{68}-j\frac{3}{68}
\right) V_{2}-5\sqrt{2}-j5\sqrt{2}=0 \\
\left( -\frac{277}{68}+\frac{3}{68}j\right) V_{1}+\left( \frac{1109}{7684}+j
\frac{3027}{7684}\right) V_{2}-\frac{80}{113}\sqrt{3}+\frac{495}{113}
+j\left( -\frac{80}{113}-\frac{495}{113}\sqrt{3}\right) =0
\end{array}
\right.
\end{equation*}
we obtain the solutions
\begin{eqnarray*}
V_{1} &=&\frac{7270}{42\,661}\sqrt{3}+\frac{327\,550}{42\,661}\sqrt{2}+\frac{
42\,960}{42\,661}+i\left( \frac{7270}{42\,661}-\frac{90\,050}{42\,661}\sqrt{2
}-\frac{42\,960}{42\,661}\sqrt{3}\right) \\
&\approx &12.16-j4.558\,9
\end{eqnarray*}
and
\begin{eqnarray*}
V_{2} &=&\frac{73\,362}{42\,661}\sqrt{3}+\frac{237\,490}{42\,661}\sqrt{2}+
\frac{120\,156}{42\,661}+j\left( \frac{73\,362}{42\,661}-\frac{3289\,970}{
42\,661}\sqrt{2}-\frac{120\,156}{42\,661}\sqrt{3}\right) \\
&\approx &13.668-j112.22
\end{eqnarray*}
From equation 3 we obtain $V_{3}$
\begin{eqnarray*}
V_{3} &=&\frac{73\,362}{42\,661}\sqrt{3}+\frac{237\,490}{42\,661}\sqrt{2}+
\frac{120\,156}{42\,661}-10\sqrt{3}+j\left( \frac{73\,362}{42\,661}-\frac{
3289\,970}{42\,661}\sqrt{2}-\frac{120\,156}{42\,661}\sqrt{3}-10\right) \\
&\approx &-3.\,652\,7-j122.22
\end{eqnarray*}
Best Answer
The most important concept for you to grasp is that a complex number carries two pieces of information: the real part and the imaginary part. A real number only carries one piece of information: its location on the number line.
So when you add two complex numbers, you need to keep track of the real parts and the imaginary parts separately. In your case, $ a + b = 4$ and $-2a -3b = -6$. So $a =6$ and $b=-2$.
In most cases, it's actually better to think of complex numbers as having magnitude and direction (measured as a counterclockwise angle), but that discussion is not for here.