IP Library Granted Patent US 9,053,555
Granted Patent B1
US 9,053,555 · App. 13/478,652 · Granted Jun 9, 2015

Creating a mesh from a sparse structure-from-motion point cloud using co-visibility of points

View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 9,053,555
App. No.
13/478,652
Granted
Jun 9, 2015
Kind
B1
Abstract

A method for creating a three-dimensional mesh model of a structure includes accessing a set of three-dimensional points associated with a set of images of the structure. For each three-dimensional point in the set of three-dimensional points, the method determines a reference image, identifies a subset of images from the set of images of the structure taken within a distance from the reference image, determines a subset of three-dimensional points seen by the subset of images, filters the subset of three-dimensional points to retain only a set of co-visible points that lie in a visibility cone of the reference image, and selects a normal using the set of co-visible points. The three-dimensional mesh model of the structure is computed using the selected normal, and the model may be provided to a second computing device.

Claims (56)

1. A method for creating a three-dimensional mesh model of a structure, comprising:

accessing a set of three-dimensional points associated with a set of images of the structure;

for each three-dimensional point in the set of three-dimensional points:

determining, using one or more processors of a computing device, a reference image;

identifying, using the one or more processors, a subset of images from the set of images of the structure, the subset taken within a distance from the reference image;

determining, using the one or more processors, a subset of three-dimensional points seen by the subset of images;

filtering, using the one or more processors, the subset of three-dimensional points to retain only a set of co-visible points that lie in a visibility cone of the reference image;

creating, using the one or more processors, two possible solutions for a surface normal using the set of co-visible points, and

selecting, using the one or more processors, a normal from the two possible solutions for a surface normal;

computing, using the one or more processors, the three-dimensional mesh model of the structure using the selected normals; and

providing the three-dimensional mesh model to a second computing device.

2. The method of claim 1 , wherein the selecting the normal includes selecting the normal that points toward a camera center of the reference image.

3. The method of claim 1 , wherein each three-dimensional point is obtained by a structure-from-motion process using multiple images of the structure.

4. The method of claim 1 , wherein the set of images are stored in a central data store accessible by the computing device.

5. The method of claim 4 , wherein at least a portion of the set of images are provided by users of multiple computing devices to the central data store.

6. The method of claim 1 , wherein the set of images include photographs and laser scans of the structure.

7. The method of claim 1 , wherein determining the reference image includes randomly selecting the reference image from the set of images of the structure.

8. The method of claim 1 , wherein determining the reference image includes receiving an identification of the reference image from a user.

9. The method of claim 1 , wherein the utilizing includes fitting a plane to the set of co-visible points using random sample consensus.

10. The method of claim 1 , wherein the distance is a predetermined radius around the structure.

11. The method of claim 1 , wherein the three-dimensional mesh model is computing using Poisson reconstruction.

12. The method of claim 1 , further comprising:

determining the distance using a homography distance test.

13. The method of claim 1 , further comprising:

displaying the three-dimensional mesh model in a graphical user interface of the second computing device.

14. The method of claim 13 , further comprising:

enabling navigation of the three-dimensional mesh model via the graphical user interface of the second computing device.

15. The method of claim 1 , further comprising: refining, using the one or more processors, the three-dimensional mesh model after a new image is added to the set of images of the structure.

16. The method of claim 1 , further comprising:

providing a graphical user interface including the three-dimensional mesh model; and

enabling a user to edit the three-dimensional mesh model via the graphical user interface.

17. The method of claim 16 , wherein the graphical user interface includes a graphical map of an area including the structure.

18. A non-transitory computer readable medium including instructions for creating a three-dimensional mesh model of a structure via a computing device, the instruction comprising:

accessing a set of three-dimensional points associated with a set of images of the structure;

for each three-dimensional point in the set of three-dimensional points:

determining, using one or more processors of the computing device, a reference image;

identifying, using the one or more processors, a subset of images from the set of images of the structure, the subset taken within a distance from the reference image;

determining, using the one or more processors, a subset of three-dimensional points seen by the subset of images;

filtering, using the one or more processors, the subset of three-dimensional points to retain only a set of co-visible points that lie in a visibility cone of the reference image;

creating, using the one or more processors, two possible solutions for a surface normal using the set of co-visible points, and

selecting, using the one or more processors, a normal from the two possible solutions for a surface normal;

computing, using the one or more processors, the three-dimensional mesh model of the structure using the selected normals; and

providing the three-dimensional mesh model to a second computing device.

19. A system for creating a three-dimensional mesh model of a structure, comprising:

a memory; and

one or more processors operably coupled to the memory and configured to execute code to:

access a set of three-dimensional points associated with a set of images of the structure;

for each three-dimensional point in the set of three-dimensional points:

determine a reference image;

identify a subset of images from the set of images of the structure, the subset taken within a distance from the reference image;

determine a subset of three-dimensional points seen by the subset of images;

filter the subset of three-dimensional points to retain only a set of co-visible points that lie in a visibility cone of the reference image;

create two possible solutions for a surface normal using the set of co-visible points, and

select a normal using from the two possible solutions for a surface normal;

compute the three-dimensional mesh model of the structure using the selected normals; and

provide the three-dimensional mesh model to a computing device.

Assignments (2)
CHANGE OF NAME Recorded Oct 2, 2017
From: GOOGLE INC.
To: GOOGLE LLC
Reel/Frame 044334/0466 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded May 23, 2012
From: OGALE, ABHIJIT
To: GOOGLE INC.
Reel/Frame 028258/0269 →