Determine the orientation of convex/concave hexahedra

algorithmscomputational geometryconvex-geometryorientationpolyhedra

I am writing a code that checks the orientation of a list of vertices (along with face connectivity) describing both convex and concave hexahedra. The face connectivity table stores the list of vertex indices that constitute each face, as described on p.27 of this document. I flag a hexahedron as being positively oriented if all faces are oriented (i.e. clockwise or counter-clockwise) such that the face normal points outward. The direction of the face normal can be determined by applying the right-hand rule in the plane of the face: four fingers curl in the direction of the face vertex orientation, and the thumb points in the direction of the face normal.

I also store the vertex list in the same order as in the linked document. However, the issue I have is that there are certain geometric operations (e.g. reflection about a plane) in my code that flip the orientation of a given hexahedron. I am currently able to check the orientation robustly for convex hexes by computing the centroid and ensuring that the face normals are outward-pointing with respect to it. However, this is not a robust procedure in the case of concave hexes because – (i) the centroid may lie outside the hex, or (ii) the interior side of a face may not be 'visible' to the centroid.

Can anyone suggest a robust procedure for checking the orientation of arbitrary hexahedra? Any advice would be greatly appreciated.

Best Answer

Given a simple flat face $ABCD$ with normal $n$, the signed area $x$ of $ABCD$ wrt $n$ is given by

$$x := \frac 14(\det(\overline{BA}, \overline{BC}, n) + \det(\overline{CB}, \overline{CD}, n) + \det(\overline{DC}, \overline{DA}, n)+ \det(\overline{AD}, \overline{AB}, n)).$$

This value is positive iff the normal satisfies the right-hand rule.