Capturing and aligning three-dimensional scenes
Systems and methods for building a three-dimensional composite scene are disclosed. Certain embodiments of the systems and methods may include the use of a three-dimensional capture device that captures a plurality of three-dimensional images of an environment. Some embodiments may further include elements concerning aligning and/or mapping the captured images. Various embodiments may further include elements concerning reconstructing the environment from which the images were captured. The methods disclosed herein may be performed by a program embodied on a non-transitory computer-readable storage medium when executed the program is executed a processor.
1. A system for building a three-dimensional composite scene, the system comprising:
a three-dimensional capture device for capturing a plurality of three-dimensional images of an object or an environment at different capture positions relative to a three-dimensional coordinate space over a capture process, each of the plurality of three-dimensional images including metadata;
a processor for executing instructions stored in memory, the processor being remote to the three-dimensional capture device, wherein execution of the instructions by the processor includes:
receiving the plurality of three-dimensional images of the object or the environment; and
aligning the received plurality of three-dimensional images to generate a three-dimensional representation of the object or the environment, the aligning being based on the metadata associated with each of the received plurality of three-dimensional images; and
an interface device for displaying at least a part of the three-dimensional representation of the object or the environment, the interface device including a live display area configured to display a live video feed of the three-dimensional representation as it is captured by the three-dimensional capture device, the interface device further including a secondary display area that provides an indication of an area of the object or environment that the three-dimensional capture device should be aimed at to capture a three-dimensional data object of the three-dimensional data objects that facilitates alignment of other three-dimensional data objects of the three-dimensional data objects.
2. The system of claim 1 , wherein the interface device is remote to the three-dimensional capture device.
3. The system of claim 1 , wherein the aligning the received plurality of three-dimensional images is done iteratively.
4. The system of claim 1 , wherein the metadata includes distance information for features included in the plurality of three-dimensional images.
5. The system of claim 4 , wherein the aligning the received plurality of three-dimensional images is based on the distance information for the features respectively included therein and aligning the features to one another in the three-dimensional coordinate space based on the distance information to generate the three-dimensional representations of the object or the environment concurrently with the capture process.
6. The system of claim 1 , wherein the three-dimensional capture device includes a transmitter to wirelessly transmit the plurality of three-dimensional images to the processor.
7. The system of claim 1 , wherein the system further includes a rotatable stage on which the three-dimensional capture device is mounted thereon.
8. A system for building a three-dimensional composite scene, the system comprising:
a three-dimensional capture device for capturing a plurality of three-dimensional images of an object or an environment at different capture positions relative to a three-dimensional coordinate space over a capture process, each of the plurality of three-dimensional images including metadata; and
a processor for executing instructions stored in memory, the processor being remote to the three-dimensional capture device, wherein execution of the instructions by the processor includes:
receiving the plurality of three-dimensional images of the object or the environment;
aligning the received plurality of three-dimensional images to generate a three-dimensional representation of the object or the environment, the aligning being based on the metadata associated with each of the received plurality of three-dimensional images; and
an interface device for displaying at least a part of the three-dimensional representation of the object or the environment, the interface device being configured to generate a prompt in response to a failure to align a subset of three-dimensional data, objects of the three-dimensional data objects, wherein the prompt requests user input identifying an area of or around the three-dimensional representation to attempt realignment.
9. The system of claim 1 , wherein the interface device is configured to identify holes in the three-dimensional representation.
10. A system comprising:
a processor; and
a memory including executable instructions to configure the processor to receive a plurality of three-dimensional images of an object or an environment from a three-dimensional capture device captured by the three-dimensional capture device at different positions relative a three-dimensional coordinate space during a capture process, each of the plurality of three-dimensional images including metadata, and send the plurality of three-dimensional images to a remote processing system as they are respectively received, wherein the remote processing system is configured to align the plurality of three-dimensional images to generate a three-dimensional representation of the object or the environment, the align the plurality of three-dimensional images being based on the metadata associated with each of the plurality of three-dimensional images, sending the three-dimensional representation of the object or the environment to a remote display for rendering, the remote display including a live display area to display a live video feed of the three-dimensional representation of the object or the environment as it is received from the three-dimensional capture device, the remote display further including a secondary display area that provides an indication of an area of the object or environment that the three-dimensional capture device should be aimed at to capture a three-dimensional data object of the three-dimensional data objects that facilitates alignment of other three-dimensional data objects of the three-dimensional data objects.
11. The system of claim 10 , wherein the align the plurality of three-dimensional images is done iteratively.
12. The system of claim 10 , wherein the metadata includes distance information for features included in the plurality of three-dimensional images.
13. The system of claim 12 , wherein the align the received plurality of three-dimensional images is based on the distance information for the features respectively included therein and align the features to one another in the three-dimensional coordinate space based on the distance information to generate the three-dimensional representations of the object or the environment concurrently with the capture process.
14. A method comprising:
capturing, by a three-dimensional capture device, a plurality of three-dimensional images of an object or an environment at different capture positions relative to a three-dimensional coordinate space over a capture process, each of the plurality of three-dimensional images including metadata;
receiving, by a processor remote to the three-dimensional capture device, the plurality of three-dimensional images of the object or the environment;
aligning the received plurality of three-dimensional images to generate a three-dimensional representation of the object or the environment, the aligning being based on the metadata associated with each of the received plurality of three-dimensional images;
displaying, by an interface device, at least a part of the three-dimensional representation of the object or the environment, the interface device including a live display area configured to display a live video feed of the three-dimensional representation as it is captured by the three-dimensional capture device; and
providing, by a secondary display area of the interface device, an indication of an area of the object or environment that the three-dimensional capture device should be aimed at to capture a three-dimensional data object of the three-dimensional data objects that facilitates alignment of other three-dimensional data objects of the three-dimensional data objects.
15. The method of claim 14 , wherein the interface device is remote to the three-dimensional capture device.
16. The method of claim 14 , wherein the aligning the received plurality of three-dimensional images is done iteratively.
17. The method of claim 14 , wherein the metadata includes distance information for features included in the plurality of three-dimensional images.
18. The method of claim 17 , wherein the aligning the received plurality of three-dimensional images is based on the distance information for the features respectively included therein and align the features to one another in the three-dimensional coordinate space based on the distance information to generate the three-dimensional representations of the object or the environment concurrently with the capture process.
19. A system comprising:
a processor; and
a memory including executable instructions to configure the processor to:
receive a plurality of three-dimensional images of an object or an environment from a three-dimensional capture device captured by the three-dimensional capture device at different positions relative a three-dimensional coordinate space during a capture process, each of the plurality of three-dimensional images including metadata, and send the plurality of three-dimensional images to a remote processing system as they are respectively received, wherein the remote processing system is configured to align the plurality of three-dimensional images to generate a three-dimensional representation of the object or the environment, the align the plurality of three-dimensional images being based on the metadata associated with each of the plurality of three-dimensional images, sending the three-dimensional representation of the object or the environment to a remote display for rendering, the remote display generating a prompt in response to a failure to align a subset of three-dimensional data, objects of the three-dimensional data objects, the prompt requesting user input identifying an area of or around the three-dimensional representation to attempt realignment.
20. A method comprising:
capturing, by a three-dimensional capture device, a plurality of three-dimensional images of an object or an environment at different capture positions relative to a three-dimensional coordinate space over a capture process, each of the plurality of three-dimensional images including metadata;
receiving, by a processor remote to the three-dimensional capture device, the plurality of three-dimensional images of the object or the environment;
aligning the received plurality of three-dimensional images to generate a three-dimensional representation of the object or the environment, the aligning being based on the metadata associated with each of the received plurality of three-dimensional images;
displaying, by an interface device, at least a part of the three-dimensional representation of the object or the environment; and
generating, by the interface device, a prompt in response to a failure to align a subset of three-dimensional data, objects of the three-dimensional data objects, the prompt requesting user input identifying an area of or around the three-dimensional representation to attempt realignment.