You can’t really measure the “vertical” angle between the vectors via orthogonal projection because that distorts the angle. The underlying reason that projecting onto the $x$-$y$ plane (in the second coordinate system in the question) works for the horizontal angles but in general there’s no orthogonal projection that will get the vertical angles right is that all of the yaw rotations use the same rotation axis, but pitch rotations don’t.
You can, however, project the vectors/points onto the unit sphere, i.e., convert to spherical coordinates: $$\theta=\arccos{z\over\sqrt{x^2+y^2+z^2}} \\ \varphi = \arctan\frac yx.$$ (We don’t care about $\rho$, the distance from the origin, here.) You need to select the correct quadrant for the arc tangent, but math packages have a built-in variant of this function that takes care of this for you. Note, too, that $\theta$ increases downwards from the $z$-axis. Since there’s no camera roll, the horizontal visual angular separation is just the difference between the azimuths $\varphi$ and the vertical visual angular separation the difference in inclination $\theta$.
Using your example of $a=(4,3,1)$ and $b=(2,7,9)$, we have $\varphi_a=\arctan\frac34\approx36.870°$ and $\varphi_b=\arctan\frac72\approx74.055°$, for a difference of approximately $37.185°$, well within the field of vision. This matches the angle obtained from the projections onto the $x$-$y$ plane. It’s probably less computationally expensive to use the latter method since you can compare directly against $\cos 80°$ instead of computing an arc cosine, but this really only works because the visual field is horizontally symmetric. If you had different angles to the left and right, using the arc tangent is better since cosine doesn’t distinguish between different directions.
For the vertical angles, we get $\theta_a=\arctan 5\approx78.690°$ and $\theta_b=\arctan{\sqrt{53}\over9}\approx38.969°$ for a difference of $39.721°$, also within the field of view.
It’s also possible to compare the vertical angles by rotating $b$ so that it lies in the same vertical plane as $a$, but that seems like more work than just converting to spherical coordinates.
If you have to check a lot of points for visibility, computing all of those inverse trigonometric functions could get expensive. The field of view is bounded by four planes through the origin, so an alternative approach is to compute the equations $ax+by+cz=0$ ($\mathbf n\cdot\mathbf x=0$ in vector form) of these planes that make up the sides of this pyramid. The sign of dot product of the normal to the plane $\mathbf n$ with an arbitrary point tells you on which side of the plane the point lies: if positive, then the point lies in the direction of $\mathbf n$ from the plane; if negative, it’s on the opposite side; if zero, the point, of course, lies on the plane. You can arrange for all of the normals to be inward-pointing, so that a point is visible iff all four of these test values are non-negative.
Your cross-product argument is faulty, because the inverse sine cannot distinguish between angles in the interval $[0,90°]$ and angles in the interval $[90°,180°]$.
The correct angle is that obtained from the scalar-product argument, $\arccos(-1/3) \approx 109°$, and you should be able to verify (numerically, at least) that this angle satisfies
$$
\sin\mathopen{}\left(\arccos(-1/3)\right)\mathclose{}
=
\frac{\sqrt{8}}{3}
= \frac{||A\times B||}{||A|| \, ||B||}.
$$
The arc-sine, on the other hand, is always restricted to producing angles in the interval $[-90°,90°]$, which means that it reflects that $109.5°$ about the $90°$ mark to produce the $70.5°$ that you observe.
Because of this limitation, your vector-product method is unreliable and it shouldn't be used to calculate angles between vectors.
Best Answer
In general, the angle $\theta$ between two complex vectors $u$ and $v$ (itself possibly complex) is given by $$ \theta = \cos^{-1}\frac{u\cdot{v^H}}{\left\lVert{u}\right\rVert\left\lVert{v}\right\rVert} $$ For the example above, $v^H$ = $(2+3i,4-i)$ and $u\cdot{v^H}$ = $6 - i$. The norm of a complex vector $u = \{u_1, u_2, \dots, u_n\}$ is given by $$ \left\lVert{u}\right\rVert = \sqrt{\sum_{i=1}^n\ u_iu_i^*} = \sqrt{\sum_{i=1}^n\ \left|u_i\right|^2} $$ This, of course, reduces to the standard (Euclidean) norm for real vectors. We see for $u$ and $v$ above the norms are $\left\lVert{u}\right\rVert = \sqrt{7}$ and $\left\lVert{v}\right\rVert = \sqrt{30}$ respectively, yielding a value for the angle between the vectors of $$ \theta = \cos^{-1}\frac{6-i}{\sqrt{210}} \approx 1.14521 + 0.0756928i $$