IP Library Granted Patent US 10,643,380
Granted Patent B2
US 10,643,380 · App. 16/544,327 · Granted May 5, 2020

Generating multi-dimensional building models with ground level images

Inventors: Shaohui Sun (Mountain View, CA); Ioannis Pavlidis (Redwood City, CA); Adam J. Altman (San Francisco, CA)
Assignee: HOVER, Inc.
G06T17/05G06F3/14G06K9/00214G06K9/00637G06T15/04G06T15/20G06T17/00G06T19/003G06T19/20G06T2200/08G06T2200/24G06T2207/10028G06T2210/04G06T2210/56G06T2219/2008
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,643,380
App. No.
16/544,327
Granted
May 5, 2020
Kind
B2
Abstract

A system and method is provided for generating textured building models from wirelessly captured ground-level imagery. Ground-level images of facades of building objects are collected for identification of geometry planes and generation of building façade geometry. The 3D building model is properly geo-positioned, scaled and textured.

Claims (54)

1. A method of modeling planar surfaces, the method comprises:

receiving a plurality of ground-level images from a smartphone, the plurality of ground-level images corresponding to all sides of an entire building;

creating a point cloud with data points from the received plurality of ground-level images;

selecting a plurality of vertices defining edges of at least one single planar surface within the point cloud;

outlining the at least one single planar surface based on the selected plurality of vertices;

creating a simplified façade geometry for the at least one single planar surface based on the outlined single planar surface;

correlating data points within the point cloud to the outlined single planar surface;

averaging a planar fit for the outlined single planar surface based on the correlated data points; and

adjusting a position of the simplified façade geometry based on the average planar fit.

2. The method of claim 1 further comprises:

receiving an orthogonal image for the entire building;

identifying a plurality of orthogonal vertices of the orthogonal image;

correlating at least two of the plurality of orthogonal vertices with the adjusted simplified façade geometry; and

constructing a building model representing all surfaces of the entire building using at least the correlated plurality of orthogonal vertices.

3. The method of claim 1 further comprises applying a texture to the at least one planar surface.

4. The method of claim 3 , wherein the applying a texture comprises mapping at least one of the ground-level images to the at least one planar surface.

5. The method of claim 3 , wherein the applying a texture comprises using one or more of: camera information or ground-level image information.

6. A method of modeling planar surfaces, the method comprises:

receiving a plurality of ground-level images from a smartphone, the plurality of ground-level images corresponding to a plurality of sides of a building;

creating a three-dimensional (3D) point cloud with data points from the received plurality of ground-level images;

selecting a plurality of vertices defining edges of a plurality of planar surfaces within the 3D point cloud;

outlining the plurality of planar surfaces based on the selected plurality of vertices;

creating a simplified facade geometry for the plurality of sides of the building based on the outlined plurality of planar surfaces;

averaging a planar fit for the data points within the outlined plurality of planar surfaces; and

adjusting a position of at least one of the plurality of planar surfaces based on the average planar fit.

7. The method of claim 6 further comprises applying a texture to the plurality of planar surfaces.

8. The method of claim 7 , wherein the applying a texture comprises mapping at least one of the ground-level images to a corresponding planar surface of the plurality of planar surfaces.

9. The method of claim 7 , wherein the applying a texture comprises using one or more of: camera information or ground-level image information.

10. The method of claim 6 further comprises:

receiving an orthogonal image for an entire perimeter of the building;

identifying a plurality of orthogonal vertices of the orthogonal image;

correlating at least two of the plurality of orthogonal vertices with each of the plurality of planar surfaces; and

constructing a building model representing all surfaces of the building using at least the correlated plurality of orthogonal vertices.

11. A system for modeling planar surfaces, the system comprises:

a computer system for receiving, from a wireless capture device, a plurality of ground-level images of a building object captured by the wireless capture device circumnavigating a building object;

an image database for storing the received plurality of ground-level images of a building object;

an image processor, operative with the image database, to:

create a point cloud with data points from the received ground-level images;

select a plurality of vertices defining edges of at least one single planar surface within the point cloud;

outline the at least one single planar surface based on the selected plurality of vertices;

create a simplified facade geometry for the at least one single planar surface based on the outlined single planar surface;

correlate data points within the point cloud to the outlined single planar surface;

average a planar fit for the outlined single planar surface based on the correlated data points; and

adjust a position of the simplified façade geometry based on the average planar fit.

12. The system of claim 11 further comprises:

receiving an orthogonal image for the entire building;

identifying a plurality of orthogonal vertices of the orthogonal image;

correlating at least two of the plurality of orthogonal vertices with the adjusted simplified façade geometry; and

constructing a building model representing all surfaces of the entire building using at least the correlated plurality of orthogonal vertices.

13. The system of claim 11 further comprises applying a texture to the at least one planar surface.

14. The system of claim 13 , wherein the applying a texture comprises mapping at least one of the ground-level images to the at least one planar surface.

15. The system of claim 13 , wherein the applying a texture comprises using one or more of: camera information or ground level image information.

16. The system of claim 11 further comprises a three dimensional (3D) proximity buffer applied to selected data points in the point cloud to determine the at least one planar surface and identify data points in the point cloud that are in close proximity to a region around each of the selected data points.

17. The system of claim 16 , wherein a closest actual data point to the region is used as the data point to determine the at least one planar surface.

Assignments (9)
TERMINATION OF INTELLECTUAL PROPERTY SECURITY AGREEMENT Recorded Oct 6, 2022
From: SILICON VALLEY BANK
To: HOVER INC.
Reel/Frame 061622/0741 →
TERMINATION OF INTELLECTUAL PROPERTY SECURITY AGREEMENT Recorded Oct 6, 2022
From: SILICON VALLEY BANK, AS AGENT
To: HOVER INC.
Reel/Frame 061622/0761 →
TERMINATION AND RELEASE OF INTELLECTUAL PROPERTY SECURITY AGREEMENT Recorded May 28, 2021
From: SILICON VALLEY BANK, AS AGENT
To: HOVER INC.
Reel/Frame 056423/0189 →
TERMINATION AND RELEASE OF INTELLECTUAL PROPERTY SECURITY AGREEMENT Recorded May 28, 2021
From: SILICON VALLEY BANK
To: HOVER INC.
Reel/Frame 056423/0179 →
INTELLECTUAL PROPERTY SECURITY AGREEMENT Recorded May 28, 2021
From: HOVER INC.
To: SILICON VALLEY BANK
Reel/Frame 056423/0199 →
INTELLECTUAL PROPERTY SECURITY AGREEMENT Recorded May 28, 2021
From: HOVER INC.
To: SILICON VALLEY BANK, AS ADMINISTRATIVE AGENT
Reel/Frame 056423/0222 →
INTELLECTUAL PROPERTY SECURITY AGREEMENT Recorded Mar 25, 2020
From: HOVER INC.
To: SILICON VALLEY BANK, AS ADMINISTRATIVE AGENT
Reel/Frame 052229/0972 →
INTELLECTUAL PROPERTY SECURITY AGREEMENT Recorded Mar 25, 2020
From: HOVER INC.
To: SILICON VALLEY BANK
Reel/Frame 052229/0986 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Aug 20, 2019
From: SUN, SHAOHUI; PAVLIDIS, IOANNIS; ALTMAN, ADAM J.
To: HOVER INC.
Reel/Frame 050095/0481 →
Continuity (4)
Continuation 15255807 · Sep 2, 2016
Continuation 14339127 · Jul 23, 2014
Provisional Application 61857302 · Jul 23, 2013
Related Publication 20190378328A1 · Dec 12, 2019
Cited By (1)
US 12,657,826