Capturing and aligning three-dimensional scenes
Systems and methods for building a three-dimensional composite scene are disclosed. Certain embodiments of the systems and methods may include the use of a three-dimensional capture device that captures a plurality of three-dimensional images of an environment. Some embodiments may further include elements concerning aligning and/or mapping the captured images. Various embodiments may further include elements concerning reconstructing the environment from which the images were captured. The methods disclosed herein may be performed by a program embodied on a non-transitory computer-readable storage medium when executed the program is executed a processor.
1. A system, comprising:
a memory that stores computer executable components;
a processor that executes at least the following computer executable components stored in the memory:
a reception component configured to receive three-dimensional data objects corresponding to different perspectives of an object or environment as the three-dimensional data objects are respectively captured in association with a capture process that comprises respectively capturing the three-dimensional data objects over a period of time from a three-dimensional capture device;
an alignment component configured to perform an alignment process that comprises iteratively aligning the three-dimensional data objects relative to one another and a common three-dimensional coordinate space as the three-dimensional data objects are respectively received, resulting in a dynamically updated alignment between the three-dimensional data objects;
a modeling component configured to generate and dynamically update a three-dimensional representation of the object or environment over the period of time based on the dynamically updated alignment; and
an interface component configured to generate a graphical user interface that facilitates viewing the three-dimensional representation as it is generated and dynamically updated and controlling the capture process, the graphical user interface comprising a primary display area configured to display the three-dimensional representation as it is generated and dynamically updated.
2. The system of claim 1 , wherein the primary display area is configured to display the three-dimensional representation of the object or environment in relation to an area of the object or environment for which three-dimensional data objects have not been received or successfully aligned and distinguish the three-dimensional representation of the object or environment from the area of the object or environment for which the three-dimensional data objects have not been received or successfully aligned.
3. The system of claim 1 , wherein the primary display area is configured to display the three-dimensional representation from a perspective determined based on an aligned position of a three-dimensional data object of the three-dimensional data objects that was most recently aligned by the alignment component.
4. The system of claim 1 , wherein the graphical user interface further comprises, a live display area configured to display a live video feed of image data as it is captured by the three dimensional capture device.
5. The system of claim 1 , wherein the graphical user interface further comprises a secondary display area identifies an area of the object or the environment that facilitates directing movement of a position and orientation of the three-dimensional capture device during the capture process.
6. The system of claim 1 , wherein the graphical user interface further comprises a secondary display area that provides an indication of an area of the object or environment that the three-dimensional capture device should be aimed at to capture a three-dimensional data object of the three-dimensional data objects that facilitates alignment of other three-dimensional data objects of the three-dimensional data objects.
7. The system of claim 6 , wherein the area is selected based on reception of no or insufficient three-dimensional image data objects corresponding to the area or a vicinity of the area.
8. The system of claim 1 , wherein the graphical user interface further comprises a guidance display area configured to display a known area of the object or environment to which the three-dimensional capture device can be pointed to determine a current position and orientation of the three-dimensional capture device.
9. The system of claim 1 , wherein an orientation of the three-dimensional representation as displayed in one or more of the embodiments is configured to be modified using a dragging gesture applied to a touch screen via which the graphical user interface is displayed.
10. The system of claim 1 , wherein the primary display area further comprises a visual indication of a position of the three-dimensional capture device at a time when a most recent three-dimensional data object of the three-dimensional data objects was captured.
11. The system of claim 1 , wherein the interface component is further configured to generate a prompt in response to a failure to align a subset of three-dimensional data objects of the three-dimensional data objects, wherein the prompt requests user input identifying an area of or around the three-dimensional representation to attempt realignment.
12. The system of claim 1 , wherein the interface component is further configured to determine quality of data within respective three-dimensional data objects of the three-dimensional data objects corresponding to respective portions of the three-dimensional representation, and wherein the interface component is configured to distinguish the respective portions of the three-dimensional representation as displayed in the primary display area.
13. The system of claim 1 , wherein the interface component is further configured to distinguish respective portions of the three-dimensional representation as displayed in the primary display area based on timing of receipt of respective three-dimensional data objects of the three-dimensional data objects associated with the respective portions.
14. The system of claim 1 , wherein the interface component is further configured to identify holes in the three-dimensional representation.
15. The system of claim 1 , further comprising a map display area configured to display a map view of a physical area including the environment or object and provide a visual indication of a position of the three-dimensional capture device relative to physical the area, wherein the map view provides an overhead orthographic view of the physical area.
16. The system of claim 1 , wherein the reception component is configured to receive auxiliary data in association with the capture process, wherein the alignment component is configured spatially align the auxiliary data with the three-dimensional representation, and wherein the primary display area is configured to display the auxiliary data based on how the auxiliary data is spatially aligned with the three-dimensional representation and from a perspective based on a current position of the three-dimensional capture device.
17. A non-transitory computer-readable storage medium comprising computer-readable instructions that, in response to execution, cause a computing system to perform operations, comprising:
receiving three-dimensional data objects corresponding to different perspectives of an environment as the three-dimensional data objects are respectively captured from one or more three-dimensional capture devices over a period of time;
iteratively aligning the three-dimensional data objects with respect to one another and a common three-dimensional coordinate space as they are respectively received, resulting in a dynamically updated alignment between the three-dimensional data objects;
generating and dynamically updating a three-dimensional model of the environment over the period of time based on the dynamically updated alignment; and
generating an interface that facilitates viewing the three-dimensional model as it is generated and dynamically updated, the interface comprising a primary display area configured to display the three-dimensional model as it is generated and dynamically updated.
18. The non-transitory computer-readable storage medium of claim 17 , the operations further comprising:
determining an area of the environment for which three-dimensional data objects have not been received or successfully aligned; and
distinguishing the three-dimensional model from the area as displayed in the primary display area.
19. The non-transitory computer-readable storage medium of claim 17 , wherein the operations further comprise:
determining a perspective for viewing the three-dimensional model based on an aligned position of a three-dimensional data object of the three-dimensional data objects that was most recently aligned; and
displaying the three-dimensional model from the perspective in the primary display area.
20. The non-transitory computer-readable storage medium of claim 17 , wherein the interface further comprises, a live display area configured to display a live video feed of image data as it is captured by the one or more three-dimensional capture devices.
21. The non-transitory computer-readable storage medium of claim 17 , wherein the operations further comprise:
determining an area of the environment that the one or more three-dimensional capture devices should be aimed at to capture a three-dimensional data object of the three-dimensional data objects that facilitates alignment of other three-dimensional data objects of the three-dimensional data objects; and
identifying the area in a secondary display area of the interface.
22. The non-transitory computer-readable storage medium of claim 21 , wherein the determining comprises determining the area based on reception of no or insufficient three-dimensional data objects corresponding to a vicinity of the area.
23. The non-transitory computer-readable storage medium of claim 17 , wherein the interface further comprises a guidance display area configured to display a known area of the environment to which the one or more three-dimensional capture devices can be pointed to determine a current position and orientation of the one or more three-dimensional capture devices.
24. The non-transitory computer-readable storage medium of claim 17 , wherein the operations further comprise:
generating a prompt in response to a failure to align a subset of the three-dimensional data objects, wherein the prompt requests user input identifying an area of or around the three-dimensional model to attempt re-alignment.
25. The non-transitory computer-readable storage medium of claim 17 , wherein the operations further comprise:
determining quality of the data of respective three-dimensional data objects of the three-dimensional data objects corresponding to respective portions of the three-dimensional model; and
distinguishing the respective portions of the three-dimensional model as displayed in the primary display area.
26. The non-transitory computer-readable storage medium of claim 17 , wherein the operations further comprise;
distinguishing respective portions of the three-dimensional model as displayed in the primary display area based on timing of receipt of respective three-dimensional data objects of the three-dimensional data objects associated with the respective portions.
27. The non-transitory computer-readable storage medium of claim 17 , wherein the interface further comprises a map display area configured to display a map view of a physical area including the environment and provide a visual indication of a current position of the one or more three-dimensional capture devices relative to the physical area.
28. A method, comprising:
using a processor to execute the following computer executable instructions stored in a memory to perform the following acts:
aligning first three-dimensional image data captured of an environment in response to reception of the first three-dimensional image data from a three-dimensional capture device, resulting in first aligned three-dimensional image data;
generating a three-dimensional representation of the environment based on the first aligned three-dimensional image data;
displaying the three-dimensional representation via a graphical user interface;
aligning and combining second three-dimensional image data captured of the environment with the first aligned three-dimensional image data in response to reception of the second three-dimensional image data from the three-dimensional capture device or another three-dimensional capture device, resulting in second aligned three-dimensional image data;
updating the three-dimensional representation of the environment based on the second aligned three-dimensional image data, resulting in an updated three-dimensional representation of the environment; and
displaying the updated three-dimensional representation via the graphical user interface, wherein the graphical user interface comprises a control panel comprising one or more controls that facilitate managing the capture of the first three-dimensional image data, the second three-dimensional image data, or additional three-dimensional image data of the environment.
29. The method of claim 28 , further comprising:
determining an area of the environment for which three-dimensional image data has not been received or successfully aligned; and
identifying the area in the graphical user interface relative to the three-dimensional representation or the updated three-dimensional representation of the environment.
30. The system of claim 1 , wherein three-dimensional data objects respectively comprise points associated with volumes that respectively correspond to portions of the object or environment, and wherein the alignment process comprises iteratively aligning the three-dimensional data objects relative to one another and the common three-dimensional coordinate space based on three-dimensional position information for the points.
31. The non-transitory computer-readable storage medium of claim 17 , wherein three-dimensional data objects respectively comprise points associated with volumes that respectively correspond to portions of the environment, and wherein the iteratively aligning comprises iteratively aligning the three-dimensional data objects relative to one another and the common three-dimensional coordinate space based on three-dimensional position information for the points.