[Math] What’s that shape? Inferring a 3D shape from random shadows

discrete geometryinverse-problemsmg.metric-geometrypolyhedrareference-request

Let $P$ be a bounded, simply connected region of $\mathbb{R}^3$.
$P$ could be a polyhedron, or a smooth shape, or an arbitrary shape;
I'll assume below that $P$ is a (non-degenerate, perhaps non-convex) polyhedron of $n$ vertices.

Suppose you have available a number $k$ of shadows of $P$, projected orthogonally to
the $xy$-plane, of $P$ after a random 3D reorientation, uniformly random over
the sphere of orientations. You have only the outline of the opaque shadow;
no internal details are visible.
For example, here are $k=5$ random shadows of a 3D polyhedron:


RandomShadows5


You do not know the orientations of the object that produced the shadows,
only that they are chosen randomly.
Perhaps you know the number of vertices of the polyhedron; $n=20$ above.
My question is not the ideal sharp MO question, but still I think there is
a substantive issue here:

Q. Under what circumstances can one (approximately) reconstruct $P$
from the $k$ shadows?

It seems that as $k \to \infty$, there should be a convergence-to-$P$ result.
As this is a natural question,1
I expect it has been explored in the literature,
in which case, pointers would be appreciated.


Click inside the region below to see the polyhedron that produced the five shadows above:

Polyhedron


One can view my question as a randomized version of the orthogonal
Gödel, Escher, Bach GEB projections:

(Of course, the same question can be asked for objects in $\mathbb{R}^d$
shadow-projected
to lower-dimensional hyperplanes.)


1

Watching the shadow of a hanging object twisting in the wind is a commonly encountered instance.


Addendum.
Several have pointed out that certain shape features are hidden from
influencing any shadow, and so cannot be reconstructed from shadows, even with
known orientations.
It turns out that the shape that can be reconstructed is known as the visual hull
(which I found by searching for "shadow hull"):

Laurentini, Aldo. "The visual hull concept for silhouette-based image understanding." IEEE Trans. Pattern Analysis Machine Intelligence, 16.2 (1994): 150-162.
(Journal link.)

This Laurentini paper points out that the visual hull of a polyhedron is not
always itself a polyhedron: Its surface can in general have curved surface patches.
There is work on reconstructing the visual hull when the orientations producing the shadows are known:

Matusik, Wojciech, Chris Buehler, and Leonard McMillan. Polyhedral visual hulls for real-time rendering. Springer Vienna, 2001. (PDF download.)

And Dustin Mixon identified literature addressing unknown orientations:

Singer, Amit, and Yoel Shkolnisky. "Three-dimensional structure determination from common lines in cryo-EM by eigenvectors and semidefinite programming." SIAM Journal Imaging Sciences. 4.2 (2011): 543-572. (Journal link.)

Best Answer

This is very similar to the cryo-electron microscopy problem: You want to image a certain macromolecule, and the scale of the macromolecule requires the use of an electron microscope. Unfortunately, such an imaging process is harmful to the specimen. In the 80s, they realized you can protect the specimen by freezing it first, but a single exposure will undo the protection. As such, they freeze a bunch of identical macromolecules (which end up at unknown orientations), and then image each one, producing a two-dimensional projection image (similar to your shadow). The cryo-EM problem is to recover the three-dimensional macromolecule from these images (see this paper and references therein)