3D model updates using crowdsourced video
View Patent ↗An exemplary method includes prompting a user to capture video data at a location. The location is associated with navigation directions for the user. Information representing visual orientation and positioning information associated with the captured video data is received by one or more computing devices, and a stored data model representing a 3D geometry depicting objects associated with the location is accessed. Between corresponding images from the captured video data and projections of the 3D geometry, one or more candidate change regions are detected. Each candidate change region indicates an area of visual difference between the captured video data and projections. When it is detected that a count of the one or more candidate change regions is below a threshold, the stored model data is updated with at least part of the captured video data based on the visual orientation and positioning information associated with the captured video data.
1. A computer-implemented method, comprising:
obtaining, using one or more computing devices, video data comprised of a plurality of images depicting a location, the video data including positioning data representing a visual orientation and position of each image with respect to the location;
identifying, by the one or more computing devices, data representing a 3D geometry of objects associated with the location;
comparing, by the one or more computing devices, visual features of objects depicted in one or more corresponding areas between images from the video data and image projections of the 3D geometry; and
updating, by the one or more computing devices, the data representing the 3D geometry based at least in part on the comparison by integrating images from the video data into the 3D geometry.
2. The method of claim 1 , wherein obtaining the video data further comprises receiving an indication that a user arrived at the location.
3. The method of claim 1 , wherein identifying the data representing the 3D geometry further comprises determining whether positioning data associated with the video data overlaps with positioning data for the 3D geometry.
4. The method of claim 1 , further comprising determining a visual correlation between one or more objects depicted in images from the video data and one or more objects depicted in the image projections.
5. The method of claim 1 , wherein the comparing further comprises detecting visual differences between features of objects depicted in a given area of a particular image from the video data and features objects depicted in a corresponding area of the image projections.
6. The method of claim 1 , further comprising determining an alignment of an integrated image from the video data based on a corresponding image projection from the 3D geometry.
7. The method of claim 6 , further comprising adjusting the alignment of the integrated images in the 3D geometry using the visual orientation information associated with the video data.
8. A non-transitory computer readable medium, storing instructions that, when executed by one or more processors, cause the one or more processors to perform a method, the method comprising:
obtaining, using one or more computing devices, video data comprised of a plurality of images depicting a location, the video data including positioning data representing a visual orientation and position of each image with respect to the location;
identifying, by the one or more computing devices, data representing a 3D geometry of objects associated with the location;
comparing, by the one or more computing devices, visual features of objects depicted in one or more corresponding areas between images from the video data and image projections of the 3D geometry; and
updating, by the one or more computing devices, the data representing the 3D geometry based at least in part on the comparison by integrating images from the video data into the 3D geometry.
9. The computer readable medium of claim 8 , wherein obtaining the video data further comprises receiving an indication that a user arrived at the location.
10. The computer readable medium 8 , wherein identifying the data representing the 3D geometry further comprises determining whether positioning data associated with the video data overlaps with positioning data for the 3D geometry.
11. The computer readable medium 8 , wherein the comparing further comprises detecting visual differences between features of objects depicted in a given area of a particular image from the video data and features objects depicted in a corresponding area of the image projections.
12. The computer readable medium 11 , further comprising determining an alignment of an integrated image from the video data based on a corresponding image projection from the 3D geometry.
13. The computer readable medium 12 , further comprising adjusting the alignment of the integrated images in the 3D geometry using the using the visual orientation information associated the corresponding image projection from the 3D geometry.
14. A system, comprising:
a memory; and
one or more processors coupled to the memory, the one or more processors being configured to:
obtain video data comprised of a plurality of images depicting a location, the video data including positioning data representing a visual orientation and position of each image with respect to the location;
identify data representing a 3D geometry of objects associated with the location;
compare visual features of objects depicted in one or more corresponding areas between images from the video data and image projections of the 3D geometry; and
update the data representing the 3D geometry based at least in part on the comparison by integrating images from the video data into the 3D geometry.
15. The system of claim 14 , wherein to obtain the video data, the one or more processors are further configured to receive an indication that a user arrived at the location.
16. The system of claim 14 , wherein to identify the data representing the 3D geometry, the one or more processors are further configured to determine whether positioning data associated with the video data overlaps with positioning data for the 3D geometry.
17. The system of claim 14 , wherein the one or more processors are further configured to determine a visual correlation between one or more objects depicted in images from the video data and one or more objects depicted in the image projections.
18. The system of claim 14 , wherein to compare visual features of objects, the one or more processors are further configured to detect visual differences between features of objects depicted in a given area of a particular image from the video data and features objects depicted in a corresponding area of the image projections.
19. The system of claim 14 , wherein the one or more processors are further configured to determine an alignment of an integrated image from the video data based on a corresponding image projection from the 3D geometry.
20. The system of claim 19 , wherein the one or more processors are further configured to adjust the alignment of the integrated images in the 3D geometry using the visual orientation information associated with the video data.