Hybrid stitching
In a system or method where three-dimensional data is acquired as a sequence of frames of data along a camera path, disparate sequences are related to one another through a number of geometric stitches obtained by direct registration of three-dimensional data between frames in the disparate sequences. These geometric stitches are then used in addition to or in place of other camera path relationships to form a single virtual stitch graph for the combined model, upon which an integrated global path optimization can be performed.
1. A method comprising:
obtaining a first three-dimensional model including a first plurality of sets of three-dimensional points spatially related to one another by a first camera path;
obtaining a second three-dimensional model including a second plurality of sets of three-dimensional points spatially related to one another by a second camera path;
registering one of the first plurality of sets of three-dimensional points to one of the second plurality of sets of three-dimensional points to place the first three-dimensional model and the second three-dimensional model in a shared coordinate system;
creating a virtual stitch graph that spatially relates the first plurality of sets and the second plurality of sets, thereby providing a combined model that includes the first plurality of sets and the second plurality of sets all spatially related by the virtual stitch graph; and
refining the combined model with a global motion optimization of the virtual stitch graph.
2. The method of claim 1 further comprising generating a first surface mesh of the first three-dimensional model and a second surface mesh of the second three-dimensional model.
3. The method of claim 2 wherein registering includes registering the first surface mesh to the second surface mesh, thereby providing a combined surface.
4. The method of claim 3 wherein registering includes registering with an iterative closest point algorithm.
5. The method of claim 3 further comprising selecting a first selected one of the first plurality of sets and a second selected one of the second plurality of sets that overlap on the combined surface.
6. The method of claim 5 wherein at least one of the first selected one and the second selected one is a set of three-dimensional data closest to a centroid of the combined surface.
7. The method of claim 5 further comprising calculating a rigid transformation and a mean error between the first selected one and the second selected one.
8. The method of claim 7 further comprising selecting a first neighborhood of the first plurality of sets from about the one of the first plurality of sets and a second neighborhood of the second plurality of sets from about the one of the second plurality of sets.
9. The method of claim 8 further comprising:
extracting a sub-region of the combined surface;
spatially relating two or more of the first plurality of sets to two or more of the second plurality of sets within the sub-region to provide a virtual stitch graph; and
minimizing an error among spatial relationships in the virtual stitch graph, thereby providing a combined data set for the sub-region.
10. The method of claim 9 further comprising creating at least one virtual stitch between two unstitched ones of the first plurality of sets.
11. The method of claim 1 further comprising refining the combined model with a deformable registration.
12. A computer program product comprising computer executable code embodied on a non-transitory computer readable medium that, when executing on one or more computing devices, performs the steps of:
obtaining a first three-dimensional model including a first plurality of sets of three-dimensional points spatially related to one another by a first camera path;
obtaining a second three-dimensional model including a second plurality of sets of three-dimensional points spatially related to one another by a second camera path;
registering one of the first plurality of sets of three-dimensional points to one of the second plurality of sets of three-dimensional points to place the first three-dimensional model and the second three-dimensional model in a shared coordinate system;
creating a virtual stitch graph that spatially relates the first plurality of sets and the second plurality of sets, thereby providing a combined model that includes the first plurality of sets and the second plurality of sets all spatially related by the virtual stitch graph; and
refining the combined model with a global motion optimization of the virtual stitch graph.
13. The computer program product of claim 12 further comprising code that performs the step of generating a first surface mesh of the first three-dimensional model and a second surface mesh of the second three-dimensional model.
14. The computer program product of claim 13 wherein registering includes registering the first surface mesh to the second surface mesh, thereby providing a combined surface.
15. The computer program product of claim 14 wherein registering includes registering with an iterative closest point algorithm.
16. The computer program product of claim 14 further comprising code that performs the step of selecting a first selected one of the first plurality of sets and a second selected one of the second plurality of sets that overlap on the combined surface.
17. The computer program product of claim 16 wherein at least one of the first selected one and the second selected one is a set of three-dimensional data closest to a centroid of the combined surface.
18. The computer program product of claim 16 further comprising code that performs the step of calculating a rigid transformation and a mean error between the first selected one and the second selected one.
19. The computer program product of claim 18 further comprising code that performs the step of selecting a first neighborhood of the first plurality of sets from about the one of the first plurality of sets and a second neighborhood of the second plurality of sets from about the one of the second plurality of sets.
20. The computer program product of claim 19 further comprising code that performs the steps of:
extracting a sub-region of the combined surface;
spatially relating two or more of the first plurality of sets to two or more of the second plurality of sets within the sub-region to provide a virtual stitch graph; and
minimizing an error among spatial relationships in the virtual stitch graph, thereby providing a combined data set for the sub-region.
21. The computer program product of claim 20 further comprising code that performs the step of creating at least one virtual stitch between two unstitched ones of the first plurality of sets.
22. The computer program product of claim 12 further comprising code that performs the step of refining the combined model with a deformable registration.
23. A system comprising:
a scanner configured to obtain a first three-dimensional model including a first plurality of sets of three-dimensional points spatially related to one another by a first camera path and a second three-dimensional model including a second plurality of sets of three-dimensional points spatially related with one another by a second camera path;
a processor and memory configured to register one of the first plurality of sets of three-dimensional points to one of the second plurality of sets of three-dimensional points to place the first three-dimensional model and the second three-dimensional model in a shared coordinate system, to create a virtual stitch graph that spatially relates the first plurality of sets and the second plurality of sets, thereby providing a combined model that includes the first plurality of sets and the second plurality of sets all spatially related by the virtual stitch graph, and to refine the combined model with a global motion optimization of the virtual stitch graph; and
a display configured to render the combined model.
24. A system comprising:
scanning means for obtaining a first three-dimensional model including a first plurality of sets of three-dimensional points spatially related to one another by a first camera path and a second three-dimensional model including a second plurality of sets of three-dimensional points spatially related with one another by a second camera path; and
processing means for registering one of the first plurality of sets of three-dimensional points to one of the second plurality of sets of three-dimensional points to place the first three-dimensional model and the second three-dimensional model in a shared coordinate system, creating a virtual stitch graph that spatially relates the first plurality of sets and the second plurality of sets, thereby providing a combined model that includes the first plurality of sets and the second plurality of sets all spatially related by the virtual stitch graph, and refining the combined model with a global motion optimization of the virtual stitch graph.