Identifying and filling holes across multiple aligned three-dimensional scenes
The capture and alignment of multiple 3D scenes is disclosed. Three dimensional capture device data from different locations is received thereby allowing for different perspectives of 3D scenes. An algorithm uses the data to determine potential alignments between different 3D scenes via coordinate transformations. Potential alignments are evaluated for quality and subsequently aligned subject to the existence of sufficiently high relative or absolute quality. A global alignment of all or most of the input 3D scenes into a single coordinate frame may be achieved. The presentation of areas around a particular hole or holes takes place thereby allowing the user to capture the requisite 3D scene containing areas within the hole or holes as well as part of the surrounding area using, for example, the 3D capture device. The new 3D captured scene is aligned with existing 3D scenes and/or 3D composite scenes.
1. A method, comprising:
using a processor to execute the following computer executable instructions stored in a memory to perform the following acts:
receiving three-dimensional data captured via one or more three-dimensional capture devices, the three-dimensional data comprising a plurality of three-dimensional scenes;
aligning the three-dimensional data to create a three-dimensional composite scene, wherein the aligning comprises:
generating possible alignment schemes between two or more three-dimensional scenes selected from the plurality of three-dimensional scenes,
evaluating the possible alignment schemes based on comparison of shape and visual appearance metrics associated with the two or more three-dimensional scenes as aligned with respect to each of the possible alignment schemes,
selecting an alignment scheme from the possible alignment schemes based on the evaluating, and
creating the three-dimensional composite scene based on the alignment scheme; and
filling a hole identified in the three-dimensional composite scene.
2. The method of claim 1 , wherein the hole comprises three-dimensional data absent from the three-dimensional composite scene.
3. The method of claim 1 , wherein the hole comprises three-dimensional data of a particular quality.
4. The method of claim 1 , further comprising displaying the hole on a graphics display device prior to filling the hole.
5. The method of claim 4 , further comprising prompting a user to capture new three-dimensional data to fill the hole.
6. The method of claim 5 , further comprising:
receiving the new three dimensional data; and
merging the new three-dimensional data with the three-dimensional data to fill the hole.
7. The method of claim 5 , further comprising providing feedback as to when the hole has been filled.
8. The method of claim 1 , wherein the filling the hole comprises filling the hole using texture and geometry information from an area surrounding the hole.
9. The method of claim 1 , wherein the filling the hole comprises using a robotic apparatus to reposition the one or more three-dimensional capture devices, and receiving new three-dimensional data to fill the hole from the one or more three-dimensional capture devices following the reposition thereof.
10. The method of claim 9 , wherein a position and orientation of the one or more three-dimensional capture devices is determined based on a geometric property of the hole or an area surrounding the hole.
11. The method of claim 9 , further comprising merging the new three-dimensional data with the three-dimensional data to fill the hole.
12. The method of claim 1 , wherein the creating the three-dimensional composite scene comprises creating the three-dimensional composite scene using the two or more three dimensional scenes.
13. The method of claim 12 , wherein the creating the three-dimensional composite scene further comprises creating the three-dimensional composite scene based on the alignment scheme using one or more additional three-dimensional scenes from the plurality of three-dimensional scenes.
14. A non-transitory computer readable storage medium comprising computer-readable instructions that, in response to execution, cause a computing system to perform operations, comprising:
receiving three-dimensional data comprising a plurality of three-dimensional scenes;
aligning the three-dimensional data to create a three-dimensional composite scene, wherein the aligning comprises:
generating possible alignment schemes between two or more three-dimensional scenes selected from the plurality of three-dimensional scenes,
evaluating the possible alignment schemes based on comparison of shape and visual appearance metrics associated with the two or more three-dimensional scenes as aligned with respect to each of the possible alignment schemes,
selecting an alignment scheme from the possible alignment schemes based on the evaluating, and
creating the three-dimensional composite scene based on the alignment scheme; and
filling a hole identified in the three-dimensional composite scene.
15. The non-transitory computer readable storage medium of claim 14 , wherein the hole comprises three-dimensional data absent from the three-dimensional composite scene.
16. The non-transitory computer readable storage medium of claim 14 , wherein the hole comprises three-dimensional data of a particular quality.
17. The non-transitory computer readable storage medium of claim 14 , wherein the operations further comprise:
displaying the hole to a user prior to filling the hole; and
prompting the user to capture new three-dimensional data to fill the hole.
18. The non-transitory computer readable storage medium of claim 17 , wherein the operations further comprise:
receiving the new three-dimensional data; and
merging the new three dimensional data with the three-dimensional data to fill the hole.
19. The non-transitory computer readable storage medium of claim 14 , wherein the filling the hole comprises filling the hole using texture and geometry information from an area surrounding the hole.
20. The non-transitory computer readable storage medium of claim 14 , wherein the filling the hole comprises directing a robotic apparatus to position capture hardware to obtain new three-dimensional data to fill the hole.
21. The non-transitory computer readable storage medium of claim 20 , wherein the operations further comprise determining at least one geometric property of the hole or an area surrounding the hole, and wherein the directing the robotic apparatus comprises directing the robotic apparatus to place the capture hardware at a position and an orientation that is determined based on the at least one geometric property.
22. A system, comprising:
a memory to store executable instructions; and
a processor, coupled to the memory, that facilitates execution of the executable instructions to perform operations, comprising:
receiving three-dimensional data comprising shape and visual appearance metrics for a plurality of three-dimensional scenes;
generating alignment schemes between two or more three-dimensional scenes selected from the plurality of three-dimensional scenes;
comparing the alignment schemes based on shape and visual appearance metrics associated with the two or more three-dimensional scenes as aligned with respect to each of the alignment schemes;
selecting an alignment scheme from the alignment schemes based on the comparing;
creating a three-dimensional composite scene based on the alignment scheme; and
filling a hole identified in the three-dimensional composite scene.