The name of the concept you are looking for is the Schur index. The Schur index is $1$ iff the representation can be realized over the field of values. The Schur index divides the degree of the character.
In your case, the the Schur index is either $1$ or $2$. You can use a variety of tests to eliminate $2$, but for instance:
Fein, Burton; Yamada, Toshihiko, The Schur index and the order and exponent of a finite group, J. Algebra 28, 496-498 (1974). ZBL0243.20008.
shows that if the Schur index was $2$, then $4$ divides the exponent of $G$.
In other words, all of your representations are realizable over the field of values.
Isaacs's Character Theory of Finite Groups has most of this in it, and I found the rest of what I needed in Berkovich's Character Theory collections. Let me know if you want more specific textbook references.
Edit: I went ahead and looked up the Isaacs pages, and looks like textbook is enough here: Lemma 10.8 on page 165 handles induced irreducible characters from complemented subgroups, and shows that the Schur index divides the order of the original character. Taking the subgroup to be the rotation subgroup and the original character to be faithful (or whichever one you need for your particular irreducible when $n$ isn't prime), you get that the Schur index divides $1$. The basics of the Schur index are collected in Corollary 10.2 on page 161.
At any rate, Schur indices are nice to know about, and if Isaacs's book doesn't have what you want, then Berkovich (or Huppert) has just a silly number of results helping to calculate it.
Edit: Explicit matrices can be found too. If $n=4k+2$ is not divisible $4$, and $G$ is a dihedral group of order $n$ with presentation $\langle a,b \mid aa=b^n=1, ba=ab^{n-1} \rangle$, then one can use companion polynomials to give an explicit representation (basically creating an induced representation from a complemented subgroup). Send $a$ to $\begin{pmatrix}0 & 1\\1 & 0\end{pmatrix}$, also known as multiplication by $x$. Send $b$ to $\begin{pmatrix}0 & -1\\1 & \zeta + \frac{1}{\zeta}\end{pmatrix}$, also known as the companion matrix to the minimum polynomial of $\zeta$ over the field $\mathbb{Q}(\zeta+\frac{1}{\zeta})$, where $\zeta$ is a primitive $(2k+1)$st root of unity.
Compare this to the more direct choice of $a = \begin{pmatrix}0 & 1\\1 & 0\end{pmatrix}$ and $b = \begin{pmatrix} \zeta & 0\\0 & \frac{1}{\zeta}\end{pmatrix}$. If you conjugate this by $\begin{pmatrix}1 & \zeta \\\zeta & 1\end{pmatrix}$ then you get my suggested choice of a representation.
In general, finding pretty, (nearly-)integral representations over a minimal splitting field is hard (and there may not be a unique minimal splitting field), but in some cases you can do it nicely.
Let me know if you continue to find this stuff interesting. I could ramble on quite a bit longer, but I think MO prefers focused answers.
Neither statement requires compactness as a hypothesis. The key result to both, and the only place where any work is needed, is the following:
If $G$ is a connected Lie group and $H$ is a Lie group, then the differentiation map
$$\text{Hom}(G, H) \to \text{Hom}(\mathfrak{g}, \mathfrak{h})$$
is injective (straightforward). If $G$ is simply connected, it is bijective (takes work).
Both statements you want are exercises assuming this result.
For the connection between representations of a simply connected Lie group and its Lie algebra, take $H = \text{GL}_n(\mathbb{C})$ and use the fact that complexification is left adjoint to the forgetful functor from complex to real Lie algebras.
For the connection between Lie groups having the same Lie algebra, if $G$ is a simply connected Lie group and $H$ is a connected Lie group with the same Lie algebra, then there is an isomorphism $\mathfrak{g} \sim \mathfrak{h}$, and by the above it lifts to a map $G \to H$ which is a local diffeomorphism and hence (with a bit of work) a covering map. It's an exercise to show that a covering map between connected topological groups is a quotient by a discrete subgroup of the center. Since $G \to H$ is surjective, a representation of $H$ is determined by the corresponding representation of $G$, and the only condition to check is that it factors.
The key result should be in any book that discusses the relationship between Lie groups and Lie algebras; to give two random examples, it is proven in Section 8.3 of Fulton and Harris, and it is Theorem 3.27 in Warner's Foundations of Differentiable Manifolds and Lie Groups.
Best Answer
Some of the comments to the question have already indicated that finite abelianisation has something to do with it. If $G$ is a finitely generated profinite group, the following are equivalent:
This result is contained in Proposition 2 in this paper by Bass, Lubotzky, Magid, and Mozes.