IP Library Granted Patent US 10,235,800
Granted Patent B2
US 10,235,800 · App. 15/835,683 · Granted Mar 19, 2019

Smoothing 3D models of objects to mitigate artifacts

Inventor: Scott Benjamin Satkin (Palo Alto, CA)
Assignee: Google LLC
G06T17/05G06T17/20G06T19/20H04N13/239G06T2219/2021H04N2013/0081
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,235,800
App. No.
15/835,683
Granted
Mar 19, 2019
Kind
B2
Abstract

A system and method is provided for smoothing portions of a 3D model of an object based on the likelihood of a portion being an accurate representation of the surface of the object, and based on whether the surface of the object according to the model is relatively jagged or relatively smooth.

Claims (47)

1. A non-transitory computing-device readable storage medium on which computing-device readable instructions of a program are stored, the instructions, when executed by one or more computing devices, causing the one or more computing devices to perform a method, the method comprising:

receiving a plurality of polygons representing a three-dimensional model of a surface of a geographic object, the three-dimensional model being based on a comparison of a first image and a second image capturing the surface of the object, where each image is captured from a different camera location;

determining a first pixel location of the first image;

determining a second pixel location of the second image based on visual similarity to the first pixel location;

determining a first ray extending from the first pixel location and a second ray extending from the second pixel location, wherein each ray is associated with a camera angle of a pixel location;

determining a first intersection location associated with the first pixel location at which the first ray intersects a surface represented by the three-dimensional model;

determining a second intersection location associated with the second pixel location at which the second ray intersects a surface represented by the three-dimensional model;

determining a confidence value based on the distance between the first and second intersection locations; and

smoothing a portion of the three-dimensional model by decreasing distances between vertices of a plurality of the polygons of the portion based on the confidence value.

2. The medium of claim 1 , wherein the polygons of the portion are textured based on the visual characteristics of the first image.

3. The medium of claim 1 , wherein the polygons of the portion are textured based on a projection of the visual characteristics of the first image onto the smoothed portion.

4. The medium of claim 3 , wherein a plurality of pixels of the first image are associated with data representing a camera angle relative to the pixel, and the projection of the visual characteristics is based on the camera angles.

5. The medium of claim 4 , wherein the camera angle data is based on information provided by a geographic component configured to determine a geographic location and orientation of a client device comprising the camera.

6. The medium of claim 5 , wherein the information provided by the geographic component comprises a cardinal direction provided by a compass and an orientation data provided by a gyroscope.

7. The medium of claim 1 , wherein determining the confidence value comprises assigning a relatively higher confidence value when the first and second intersection locations are in a single polygon.

8. The medium of claim 1 , wherein determining the confidence value comprises assigning a relatively lower confidence value when the first and second intersection locations are in two different polygons.

9. A method comprising:

receiving a plurality of polygons representing a three-dimensional model of a surface of a geographic object, the three-dimensional model being based on a comparison of a first image and a second image capturing the surface of the object, where each image is captured from a different camera location;

determining a first pixel location of the first image;

determining a second pixel location of the second image based on visual similarity to the first pixel location;

determining a first ray extending from the first pixel location and a second ray extending from the second pixel location, wherein each ray is associated with a camera angle of a pixel location;

determining a first intersection location associated with the first pixel location at which the first ray intersects a surface represented by the three-dimensional model;

determining a second intersection location associated with the second pixel location at which the second ray intersects a surface represented by the three-dimensional model;

determining a confidence value based on the distance between the first and second intersection locations; and

smoothing a portion of the three-dimensional model by decreasing distances between vertices of a plurality of the polygons of the portion based on the confidence value.

10. The method of claim 9 , wherein the polygons of the portion are textured based on the visual characteristics of the first image.

11. The method of claim 9 , wherein the polygons of the portion are textured based on a projection of the visual characteristics of the first image onto the smoothed portion.

12. The method of claim 11 , wherein a plurality of pixels of the first image are associated with data representing a camera angle relative to the pixel, and the projection of the visual characteristics is based on the camera angles.

13. The method of claim 12 , wherein the camera angle data is based on information provided by a geographic component configured to determine a geographic location and orientation of a client device comprising the camera.

14. The method of claim 13 , wherein the information provided by the geographic component comprises a cardinal direction provided by a compass and an orientation data provided by a gyroscope.

15. The method of claim 9 , wherein determining the confidence value comprises assigning a relatively higher confidence value when the first and second intersection locations are in a single polygon.

16. The method of claim 9 , wherein determining the confidence value comprises assigning a relatively lower confidence value when the first and second intersection locations are in two different polygons.

17. A system comprising:

one or more computing devices, and

memory storing instructions executable by the processor and a model associated with locations on the surface of a geographic feature,

wherein the instructions comprise:

receiving a plurality of polygons representing a three-dimensional model of a surface of a geographic object, the three-dimensional model being based on a comparison of a first image and a second image capturing the surface of the object, where each image is captured from a different camera location;

determining a first pixel location of the first image;

determining a second pixel location of the second image based on visual similarity to the first pixel location;

determining a first ray extending from the first pixel location and a second ray extending from the second pixel location, wherein each ray is associated with a camera angle of a pixel location;

determining a first intersection location associated with the first pixel location at which the first ray intersects a surface represented by the three-dimensional model;

determining a second intersection location associated with the second pixel location at which the second ray intersects a surface represented by the three-dimensional model;

determining a confidence value based on the distance between the first and second intersection locations; and

smoothing a portion of the three-dimensional model by decreasing distances between vertices of a plurality of the polygons of the portion based on the confidence value.

18. The system of claim 17 , wherein the polygons of the portion are textured based on the visual characteristics of the first image.

19. The system of claim 17 , wherein the polygons of the portion are textured based on a projection of the visual characteristics of the first image onto the smoothed portion.

20. The system of claim 17 , wherein a plurality of pixels of the first image are associated with data representing a camera angle relative to the pixel, and the projection of the visual characteristics is based on the camera angles.

Assignments (2)
CHANGE OF NAME Recorded Dec 12, 2017
From: GOOGLE INC.
To: GOOGLE LLC
Reel/Frame 044841/0863 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Dec 11, 2017
From: SATKIN, SCOTT BENJAMIN
To: GOOGLE INC.
Reel/Frame 044354/0336 →
Continuity (2)
Continuation 14877414 · Oct 7, 2015
Related Publication 20180101981A1 · Apr 12, 2018