Currently I'm working on a basic 3D model viewer in MATLAB, which can import .OBJ files (modeled in e.g. Blender). The purpose of this viewer is to test different subdivision schemes (like Catmull-Clark, or Doo-Sabin). A subdivision scheme is basically a method to convert a mesh to a smoother (refined) mesh.
In order to do this properly, I would like to implement some functionality to select Vertices and Edges in my viewer.
Vertex selection isn't very hard — the way this works is that when I click somewhere in the viewer, MATLAB provides me with two points, the begin- and end points of the line perpendicular to the screen (by clicking on a 2D screen that depicts a 3D scene, you actually click on a line, not a single point). Using this method it is relatively simple to find the closest Vertex.
However, I'm not quite sure how to do the same for an Edge. Imagine that the imported geometry is a simple cube, so 8 Vertices and 12 Edges. If I know click on an arbitrary point in the viewer (i.e. on some line perpendicular to the screen), how can I then determine which of the 12 Edges is the closest one?
[First Edit] Just to be clear, I'm not really looking for a piece of (MATLAB) code, but rather for some theoretical explanation (or just a reference to an article/book/website). Thanks!
[Second Edit] By the way, is this the correct place for such a question? Or might the Mathematics or StackOverflow boards be better suited?
Best Answer
A similar question has actually been posed on stackoverflow before and is answered there (with matlab): https://stackoverflow.com/a/702174/545346
Edit: I was actually thinking about writing something about spatial indexing, but I then thought it was not necessary since you spoke about 12 edges.
A logical approach would be a filter using the distance of both vertices to your new line: if you have found an edge with both vertices further away from your line than a previous edge you are sure that the second edge will not be the closest one, so you don't need to do more advanced calculations.
A more advanced procedure would actually use spatial indexes to make sure not too many combinations are used. A good start could be http://en.wikipedia.org/wiki/R-tree
Anyway, this is not something you generally want to implement yourself: you could check out postgis, which has support for 3d objects.