IP Library Granted Patent US 9,437,033
Granted Patent B2
US 9,437,033 · App. 14/339,127 · Granted Sep 6, 2016

Generating 3D building models with ground level and orthogonal images

Inventors: Shaohui Sun (Mountain View, CA); Ioannis Pavlidis (San Mateo, CA); Adam J. Altman (San Francisco, CA)
Assignee: Hover Inc.
G06T15/04G06T17/05G06T2210/56
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 9,437,033
App. No.
14/339,127
Granted
Sep 6, 2016
Kind
B2
Abstract

A system and method is provided for generating textured 3D building models from ground-level and orthogonal imagery. Ground-level images for the sides of building objects are collected to form a 3D dense point cloud for identification of key architectural features, corresponding key façade geometry planes, and generation of a 3D building façade geometry. Orthogonal images are correlated to the 3D building model providing a properly geo-positioned, scaled and textured 3D building model.

Claims (39)

1. A method of generating a textured 3D building model, the method comprising:

collecting, using a smartphone, a plurality of captured ground-level images for a plurality of sides of a building object, the collecting including capturing and storing in computer memory of the smartphone the plurality of captured ground-level images from multiple angles and distances as a user of the smartphone circumnavigates the building object;

uploading and storing the plurality of captured ground-level images from the smartphone to computer memory of a remote server;

processing the plurality of ground-level images using a computer system, the processing including:

creating a point cloud from the collected, uploaded and stored plurality of captured ground-level images;

outlining a plurality of vertices for defining edges of a plurality of single planar surfaces;

selecting a plurality of data points from the created point cloud that represent the plurality of single planar surfaces within the created point cloud;

creating simplified façade geometry for each of the plurality of sides; and

texturing the simplified façade geometry with one or more of the collected plurality of captured ground-level images.

2. The method of claim 1 further comprising generating the textured 3D building model using textured ones of all of the plurality of sides.

3. The method of claim 2 further comprising:

retrieving an orthogonal image for the building object;

correlating the vertices of the orthogonal image with at least two of the plurality of vertices for each of the single planar surfaces; and

reconstructing the textured 3D building model using at least the correlated vertices.

4. The method of claim 3 , wherein the reconstructing includes at least scaling of the textured 3D building model.

5. The method of claim 3 , wherein the reconstructing includes at least rotation of the textured 3D building model.

6. The method of claim 1 , wherein the plurality of captured ground-level images are captured from a plurality of perspectives.

7. The method of claim 6 , wherein the plurality of perspectives include at least the boundaries of the plurality of sides of the building object.

8. The method of claim 6 , wherein the plurality of perspectives include at least the boundaries of the plurality of sides of the building object.

9. A method of claim 1 further comprising identifying architectural geometric features of the point cloud.

10. A method of claim 1 , wherein the selection of the plurality of data points from the point cloud that represent a plurality of single planar surfaces is performed manually or semi-automatically.

11. A method of claim 1 , wherein the plurality of data points from the point cloud comprises two or more.

12. A method of claim 1 , wherein the outlining of the plurality of vertices is performed manually or semi-automatically.

13. A system for generating a textured 3D building model, the system comprising:

a smartphone capture device for capturing and storing in computer memory a plurality of ground-level images of a building object while a user of the smartphone capture device circumnavigates the building object;

an orthogonal image database for loading and retrieving an orthogonal image of the building object;

an image processor for constructing a textured 3D building model of the building object from the captured plurality of ground-level images of the building object and reconstructing the textured 3D building model according to the retrieved orthogonal image; and

a viewer device for displaying the reconstructed textured 3D building model.

14. The system of claim 13 further comprising a communications network for transmitting the captured plurality of ground-level images of the building object.

15. The system of claim 13 wherein the orthogonal image database retrieves an orthogonal image of the building object based on location information determined by the captured plurality of ground-level images of the building object.

16. The system of claim 15 , wherein the location information is global positioning system (GPS) or a user defined location.

17. The system of claim 15 , wherein the user defined location is an address or a location on a map selected by a user.

18. A method for generating a textured 3D building model, the method comprising:

capturing, using a smartphone, a plurality of ground-level images of a plurality of façades of a building object, the capturing including, creating and storing in the smartphone, a plurality of the ground-level images captured from multiple angles and distances as a user circumnavigates the building object;

uploading and storing the plurality of ground-level images from the smartphone to computer memory of a remote server;

creating, using computer processing, a point cloud from the plurality of ground-level images retrieved from the remote server;

creating a plurality of textured façades from the created point cloud, the plurality of textured façades including all sides of the building object;

correlating the textured façade with a corresponding vertical edge of a retrieved orthogonal image; and

creating the textured 3D building model of the building object from the correlated textured façade and orthogonal image.

Assignments (10)
TERMINATION OF INTELLECTUAL PROPERTY SECURITY AGREEMENT Recorded Oct 6, 2022
From: SILICON VALLEY BANK
To: HOVER INC.
Reel/Frame 061622/0741 →
TERMINATION OF INTELLECTUAL PROPERTY SECURITY AGREEMENT Recorded Oct 6, 2022
From: SILICON VALLEY BANK, AS AGENT
To: HOVER INC.
Reel/Frame 061622/0761 →
TERMINATION AND RELEASE OF INTELLECTUAL PROPERTY SECURITY AGREEMENT Recorded May 28, 2021
From: SILICON VALLEY BANK
To: HOVER INC.
Reel/Frame 056423/0179 →
TERMINATION AND RELEASE OF INTELLECTUAL PROPERTY SECURITY AGREEMENT Recorded May 28, 2021
From: SILICON VALLEY BANK, AS AGENT
To: HOVER INC.
Reel/Frame 056423/0189 →
INTELLECTUAL PROPERTY SECURITY AGREEMENT Recorded May 28, 2021
From: HOVER INC.
To: SILICON VALLEY BANK, AS ADMINISTRATIVE AGENT
Reel/Frame 056423/0222 →
INTELLECTUAL PROPERTY SECURITY AGREEMENT Recorded May 28, 2021
From: HOVER INC.
To: SILICON VALLEY BANK
Reel/Frame 056423/0199 →
INTELLECTUAL PROPERTY SECURITY AGREEMENT Recorded Mar 25, 2020
From: HOVER INC.
To: SILICON VALLEY BANK
Reel/Frame 052229/0986 →
INTELLECTUAL PROPERTY SECURITY AGREEMENT Recorded Mar 25, 2020
From: HOVER INC.
To: SILICON VALLEY BANK, AS ADMINISTRATIVE AGENT
Reel/Frame 052229/0972 →
CORRECTIVE ASSIGNMENT TO CORRECT THE RECEIVING PARTY NAME PREVIOUSLY RECORDED AT REEL: 033535 FRAME: 0486. ASSIGNOR(S) HEREBY CONFIRMS THE ASSIGNMENT. Recorded Sep 15, 2016
From: SHAOHUI, SUN; IOANNIS, PAVLIDIS; ADAM, ALTMAN J.
To: HOVER INC.
Reel/Frame 040039/0228 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Aug 14, 2014
From: SUN, SHAOHUI; PAVLIDIS, IOANNIS; ALTMAN, ADAM J.
To: HOVER, INC.
Reel/Frame 033535/0486 →
Continuity (2)
Provisional Application 61857302 · Jul 23, 2013
Related Publication 20150029182A1 · Jan 29, 2015