Curated photogrammetry
Approaches to generating photogrammetry data are provided. In some implementations, a capture device captures a sequence of images of a physical environment while device poses of the capture device are tracked as the capture device moves around the physical environment. For each of image of images in the sequence of images, a corresponding device pose is stored as metadata of the image. Using photogrammetry, a three-dimensional (3D) model of the physical environment is generated from the images, where the photogrammetry uses for each image of the images, the corresponding device pose from the metadata of the image to generate the 3D model. In further respects, a user feedback loop can be used to direct a user to move the capture device to particular device poses for the capture of the images. Further camera properties used to capture an image may be stored in its metadata and used by the photogrammetry.
1. A computer-implemented system comprising:
one or more processors; and
one or more computer-readable media having executable instructions embodied thereon, which, when executed by the one or more processors, cause the one or more processors to perform a method comprising:
capturing, using a capture device, a sequence of images of a physical environment while tracking device poses of the capture device as the capture device moves around the physical environment;
for each image of a plurality of images in the sequence of images, storing a corresponding device pose of the device poses as metadata of the image;
determining a desired device pose for the capture device based on an analysis of one or more images in the sequence of images;
directing movement of the capture device to the desired device pose for capturing an additional image using a user feedback loop generated from the device poses, wherein storing the corresponding device pose for the additional image is based on determining that the corresponding device pose corresponds to the desired device pose; and
generating, by photogrammetry, a three-dimensional (3D) model of the physical environment from the plurality of images, the photogrammetry using for each image in the plurality of images, the corresponding device pose from the metadata of the image to generate the 3D model.
2. The computer-implemented system of claim 1 , wherein the generating of the 3D model comprises:
determining common points between images based on the corresponding device pose of images of the plurality of images; and
estimating 3D coordinates of an object in the physical environment based on the determined common points.
3. The computer-implemented system of claim 1 , wherein the method further comprises for the plurality of images, storing corresponding camera properties of the capture device as metadata of the image, wherein the photogrammetry further uses for each image in the plurality of images, the corresponding camera properties from the metadata of the image to generate the 3D model.
4. The computer-implemented system of claim 1 , wherein the method comprises during the capturing of the sequence of images:
assigning a score to an image in the sequence of images based on an analysis of the image;
refraining from including the image in the plurality of images based on the score;
adjusting camera properties of the capture device based on the score;
assigning an additional score to an additional image in the sequence of images based on an analysis of the image, the additional image captured using the adjusted camera properties; and
based on the additional score including the additional image in the plurality of images.
5. The computer-implemented system of claim 1 , wherein the method further comprises:
wherein directing movement of the capture device to the desired device pose using the user feedback loop comprises intensifying device feedback in the user feedback loop as the capture device approaches the desired device pose and fading the device feedback as the capture device moves further from the desired device pose.
6. The computer-implemented system of claim 1 , wherein the capturing of the sequence of images is from a live video feed.
7. The computer-implemented system of claim 1 , wherein each of the device poses comprises a respective orientation of the capture device in space.
8. A computer-implemented method comprising:
analyzing a feed of capture data of a physical environment from a capture device as the capture device moves around a physical environment;
determining a desired device pose for the capture device based on the analyzing of the capture data;
directing movement of the capture device to the desired device pose using a user feedback loop generated from the feed of capture data, wherein directing the movement of the capture device to the desired device pose comprises intensifying device feedback in the user feedback loop as the capture device approaches the desired device pose and fading the device feedback as the capture device moves further from the desired device pose;
based on determining a device pose of the capture device corresponds to the desired device pose, storing an association between image data captured by the capture device and the device pose; and
generating, by photogrammetry, a three-dimensional (3D) model of the physical environment from the image data, the photogrammetry using the device pose with the image data to generate the 3D model based on the stored association.
9. The computer-implemented method of claim 8 , wherein the generating of the 3D model comprises:
determining common points between images based on the device pose and the image data; and
estimating 3D coordinates of an object in the physical environment based on the determined common points.
10. The computer-implemented method of claim 8 , wherein the feed of capture data comprises a live video feed.
11. The computer-implemented method of claim 8 , wherein the device pose comprises an orientation of the capture device in space.
12. The computer-implemented method of claim 8 , wherein the association is further between camera properties corresponding to the image data and the image data, and the photogrammetry further users the camera properties with the image data to generate the 3D model based on the stored association.
13. The computer-implemented method of claim 8 , further comprising repeatedly capturing images corresponding to the desired device pose and adjusting camera properties of the capture device until an analysis of the images indicates at least one image of the images is sufficient for the photogrammetry, wherein the image data corresponds to the image.
14. The computer-implemented method of claim 8 , wherein the desired device pose comprises a respective orientation of the capture device in space device feedback.
15. The computer-implemented method of claim 8 , wherein the desired device pose is based on determining coverage of the physical environment by photogrammetry data.
16. One or more computer storage media having executable instructions embodied thereon, which, when executed by one or more processors, cause the one or more processors to perform a method comprising:
capturing, using a capture device, a sequence of images of a physical environment while tracking device poses of the capture device as the capture device moves around the physical environment;
for each image of a plurality of images in the sequence of images, storing a corresponding device pose of the device poses as metadata of the image;
directing movement of the capture device to a desired device pose using a user feedback loop for capturing an additional image, based on an analysis of one or more images in the sequence of images;
based on determining that a device pose of the capture device corresponds to the desired device pose, storing an association between image data for the additional image captured by the capture device and the device pose; and
generating, by photogrammetry, a three-dimensional (3D) model of the physical environment from the plurality of images, the photogrammetry using for each image in the plurality of images, the device pose from the metadata of the image to generate the 3D model.
17. The one or more computer storage media of claim 16 , wherein the generating of the 3D model comprises:
determining common points between images based on the corresponding device pose of images of the plurality of images; and
estimating 3D coordinates of an object in the physical environment based on the determined common points.
18. The one or more computer storage media of claim 16 , wherein the capturing of the sequence of images is from a live video feed.
19. The one or more computer storage media of claim 16 , wherein each of the device poses comprises a respective orientation of the capture device in space.
20. The one or more computer storage media of claim 16 , further comprising for the plurality of images, storing corresponding camera properties of the capture device as metadata of the image, wherein the photogrammetry further uses for each image in the plurality of images, the corresponding camera properties from the metadata of the image to generate the 3D model.