I would approach this by first using the Union tool.
The output from that will have attributes that enable you to distinguish three sets of polygons i.e. those where the two input feature classes overlap, and those where one or other exists but do not overlap with the other.
From this foundation you can start to apply your other rules - perhaps using a combination of Select, Dissolve and Append/Merge.
Once you have your procedure it should be amenable to turning into a model or Python script.
From @user30184's comment, I checked the code and first found this:
public static Geometry overlay(Geometry geom0, Geometry geom1,
int opCode, PrecisionModel pm)
{
OverlayNG ov = new OverlayNG(geom0, geom1, pm, opCode);
Geometry geomOv = ov.getResult();
return geomOv;
}
I then checked the getResult() method and found this:
else {
// handle case where both inputs are formed of edges (Lines and Polygons)
result = computeEdgeOverlay();
}
Checking the computeEdgeOverlay function led me to this:
return extractResult(opCode, graph);
I then checked the extractResult function and found this:
private Geometry extractResult(int opCode, OverlayGraph graph) {
boolean isAllowMixedIntResult = ! isStrictMode;
//--- Build polygons
List<OverlayEdge> resultAreaEdges = graph.getResultAreaEdges();
PolygonBuilder polyBuilder = new PolygonBuilder(resultAreaEdges, geomFact);
List<Polygon> resultPolyList = polyBuilder.getPolygons();
boolean hasResultAreaComponents = resultPolyList.size() > 0;
Finally, I checked the getResultAreaEdges method and found the following comment after it:
/**
* Traverse the star of DirectedEdges, linking the included edges together.
* To link two dirEdges, the <code>next</code> pointer for an incoming dirEdge
* is set to the next outgoing edge.
* <p>
* DirEdges are only linked if:
* <ul>
* <li>they belong to an area (i.e. they have sides)
* <li>they are marked as being in the result
* </ul>
* <p>
* Edges are linked in CCW order (the order they are stored).
* This means that rings have their face on the Right
* (in other words,
* the topological location of the face is given by the RHS label of the DirectedEdge)
* <p>
* PRECONDITION: No pair of dirEdges are both marked as being in the result
*/
public void linkResultDirectedEdges()
This comment includes the words CCW order, RHS label, and in the result, which highly suggests that the algorithm that is being used can be found in the paper
"An Edge Labeling Approach to Concave Polygon Clipping" by Klamer Schutte (1995)
since these terms explicitly appear in this paper. This is also the second paper mentioned in the JTS FAQ question Are there any references which describe the algorithms used in JTS?. However, there is no way to say for sure that this is true, and I believe that special cases are handled using the algorithms found in the paper
"A General and Efficient Implementation of Geometric Operators and Predicates" by Edward P. F. Chan and Jimmy N. H. Ng (1997)
which is also mentioned in the JTS FAQ.
Best Answer
A great circle is formed by the intersection of a sphere and a plane that passes through the sphere's center (one the Earth this is a line of longitude, see image below from Paul Bourke - lots of handy stuff on that site). You are looking for the intersection of:
If your polygons are actually great circles, then you are describing a lune.
If you are describing polygons that "envelop" the sphere (e.g., cover an entire line of longitude) or cover a large part of it, then you can re-project the sphere's surface to a plane (see this question) and handle the intersection computation with standard computational geometry tools. Depending on your data formats and license concerns there are a lot of options: