IP Library Granted Patent US 9,875,575
Granted Patent B2
US 9,875,575 · App. 14/877,414 · Granted Jan 23, 2018

Smoothing 3D models of objects to mitigate artifacts

Inventor: Scott Benjamin Satkin (Palo Alto, CA)
Assignee: Google LLC
G06T17/05G06T5/002G06T17/20G06T19/20H04N13/0239G06T2207/10021G06T2207/10028G06T2207/20182G06T2219/2021H04N2013/0081
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 9,875,575
App. No.
14/877,414
Filed
Oct 7, 2015
Granted
Jan 23, 2018
Kind
B2
Art Unit
2614
USPC
345/420
Abstract

A system and method is provided for smoothing portions of a 3D model of an object based on the likelihood of a portion being an accurate representation of the surface of the object, and based on whether the surface of the object according to the model is relatively jagged or relatively smooth.

Claims (36)

1. A system comprising:

one or more computing devices, and

memory storing instructions executable by the processor and a model comprising a plurality of model locations associated with geographic locations on a surface of a geographic feature,

wherein the instructions comprise:

determining a confidence value for a first model location of the model, where the confidence value is associated with a likelihood of the first model location being an accurate representation of a geographic location on the surface of the geographic feature;

determining, with the one or more computing devices for a portion of the model comprising a plurality of model locations including the first model location, an eccentricity value based on a jaggedness of the surface according model locations;

determining a reference shape defining a plurality of reference locations based on an estimate of the geographic locations of the surface of the geographic feature, a first reference location of the plurality of reference locations being a first distance from the first model location; and

modifying the portion of the model by automatically moving the first model location and decreasing the first distance, wherein the amount of the decrease is larger when the confidence value is associated with a lower said likelihood and the eccentricity value is associated with a relatively jagged surface than when the confidence value is associated with a higher said likelihood and the eccentricity value is associated with a relatively smooth surface; and

providing the modified portion of the model.

2. The system of claim 1 wherein the confidence value is based on the first distance.

3. The system of claim 2 wherein modifying a location of the portion comprises moving the location closer to the reference shape.

4. The system of claim 2 wherein the reference shape is planar and determined based on a plurality of locations of the model.

5. The system of claim 2 wherein the model is determined based on imagery capturing the geographic feature, and wherein the instructions further comprise:

recognizing, based on visual characteristics of the imagery, a type of an object to which the geographic feature pertains, and

determining the reference shape based on the type of the object.

6. The system of claim 5 wherein the reference shape is planar and based on visual characteristics of a building.

7. The system of claim 1 wherein the confidence value of the first model location is based on whether the model was generated from imagery capturing the geographic feature.

8. The system of claim 1 wherein the model is generated based on a comparison of first imagery of the geographic feature captured from a first camera location and second imagery of the geographic feature captured from a second camera location different from the first camera location.

9. The system of claim 8 wherein the instructions further comprise:

determining a first pixel location of the first imagery;

determining a second pixel location of the second imagery based on visual similarity to the first pixel location;

determining a first ray extending from the first pixel location and a second ray extending from the second pixel location, wherein each ray is associated with a camera angle of a pixel location;

determining an intersection location associated with the location at which the ray intersects a surface represented by the model; and

wherein the confidence value is based on the distance between the intersection locations.

10. The system of claim 1 further comprising displaying the modified portion to a user.

11. A method of determining a model of a geographic feature comprising:

receiving, with one or more computing devices, a model having a plurality of model locations associated with geographic location on a surface of the geographic feature;

determining, with the one or more computing devices and for a first model location of the model, a confidence value, where the confidence value is associated with a likelihood of the first model location being an accurate representation of a geographic location on the surface of the geographic feature;

determining, with the one or more computing devices, a reference shape defining a plurality of reference locations based on an estimate of the geographic locations of the surface of the geographic feature, a first reference location of the plurality of reference locations being a first distance from the first model location;

determining, with the one or more computing devices and for a plurality of model locations in a portion of the model, said plurality of model locations comprising the first model location, an eccentricity value based on a jaggedness of the surface according to the distance between each model location relative to the reference shape and a difference between said distance and a distance between neighboring model locations and the reference shape;

automatically modifying, with the one or more computing devices, the first model location by decreasing the first distance, wherein the amount of the decrease is larger when the confidence value is associated with a lower said likelihood and the eccentricity value is associated with a relatively jagged surface than when the confidence value is associated with a higher said likelihood and the eccentricity value is associated with a relatively smooth surface; and

providing, with one or more computing devices, the modified model.

12. The method of claim 11 wherein the plurality of model locations of the portion is determined from an image or laser-range measurement of the geographic feature and determining the reference shape comprises determining a best-fitting shape associated with the portion.

13. The method of claim 12 wherein the reference shape is a best-fitting plane of the model locations prior to modification.

14. The method of claim 11 wherein the eccentricity value is based on a frequency and magnitude of changes in said distance between neighboring locations.

15. The method of claim 11 further comprising applying a smoothing function to the portion.

Assignments (2)
CHANGE OF NAME Recorded Oct 5, 2017
From: GOOGLE INC.
To: GOOGLE LLC
Reel/Frame 044129/0001 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Oct 8, 2015
From: SATKIN, SCOTT BENJAMIN
To: GOOGLE INC.
Reel/Frame 036757/0833 →
Continuity (1)
Related Publication 20170103568A1 · Apr 13, 2017