Here's a cross section of the situation, with two oriented planes. If you're rotating the red/green plane in the direction of its green side, does that mean clockwise or counterclockwise?
So I don't think you can use the orientation of the planes to fix an interpretation of the angle. However, you write that you have a preferred orientation of the line of intersection, and that sounds more promising.
Namely, you can decide to turn the first plane counterclockwise when looking in the preferred direction along the intersection line.
Somewhat tedious formula-churning implementation of this: Fix some point on the intersection line and consider the plane that is normal to it at that point. Equip that plane with an orthonormal coordinate system that creates a right-handed system together with the preferred direction of the intersection. The orthonormal coordinates create a parametric representation of the normal plane; compose that with the equations for the two original planes; that gives 2D equations within the normal plane for its intersection with the original plane. Now find the angle in the normal plane using standard 2D techniques (e.g as the difference of atan2
values).
Computationally slicker: You can find $|\cos\theta|$ by the dot product or normals in 3D without needing to project into a specific cross section; the only remaining information you need is whether to use $\theta_0$, $\pi-\theta_0$, $\theta_0-\pi$ or $-\theta_0$, where $\theta_0$ is the arccosine of the dot product. But you can actually distinguish between those four cases simply by the combination of (a) the sign of the dot product and (b) whether the cross product of the normals is parallel or antiparallel to the chosen direction of the intersection line. I'd probably make some embarassing mistake if I tried to work our exactly what the connection should be, but there are only 4! possible ways to do it anyway, so just figure it out with trial and error...
Let's find the dihedral angle between the planes by taking the dot product of normal unit vectors Three points are needed to determine a plane, so in the chain of bonds C-G-O-A we'll take the elements three at a time to get the two planes: C-G-O and G-O-A.
Let us now calculate unit vectors $\mathbf{n}_{CGO}$ and $\mathbf{n}_{GOA}$ normal to these planes. First note that C, G, and O all have $x$-component $0$, so C-G-O lies in the $yz$-plane and we can take $\mathbf{n}_{CGO} = (1,0,0)$. For the other plane, we have:
$$\begin{align}
OG \times OA &= \big((0, 0, 0) - (0, 0.705, 1.2211)\big) \times \big((0, 0, 0) - (-1.2211, -0.705, 0)\big)\\
&=(0, -0.705, -1.2211) \times (1.2211, 0.705, 0)\\
&=(0.8609,-1.4911,0.8609)
\end{align}$$
Before making this a unit vector, let's divide by $0.8609$ to clean things up: this last vector is parallel to $(1,-1.73206,1)$, which I take to be $(1,-\sqrt{3},1)$. Interesting. Shrinking this to a unit vector:
$$\begin{align}
\mathbf{n}_{GOA} &= \frac{OG \times OA}{|OG \times OA|}\\
&=\frac{(1,-\sqrt{3},1)}{||(1,-\sqrt{3},1)||}\\
&=\frac{1}{\sqrt{5}}(1,-\sqrt{3},1)\\
\end{align}$$
Finally, the dihedral angle $\phi$ between our two planes satisfies $\cos{\phi} = \mathbf{n}_{CGO} \cdot \mathbf{n}_{GOA} = \frac{1}{\sqrt{5}}$. In other words, $\phi = \arccos(\frac{1}{\sqrt{5}}) = 1.1071$, or about $63.43^\circ$.
The dihedral angle for D-A-O-G can be calculated the same way.
Best Answer
Note that we can consider two angles between the planes and the normal vectors, the first one $\alpha \le \frac \pi 2$ and the second one $\beta \ge \frac \pi 2$.
In your picture you are considering the acute angle $\alpha$ for the planes and the obtuse angle $\beta$ for the normal vectors and we have $\alpha=\pi - \beta$.