IP Library › Granted Patent US 12,229,887
Granted Patent B2
US 12,229,887 · App. 18/117,312 · Granted Feb 18, 2025

3D building model materials auto-populator

Inventors: Steven Caamano (Spencerport, NY); Javier Cambon Sanjurjo (San Francisco, CA); Derek Halliday (Oakland, CA)
Assignee: Hover Inc.
G06T17/05G06F3/14G06Q30/0283G06Q30/0633G06T15/04G06T15/20G06T17/00G06T19/003G06T19/20G06V20/176G06V20/653G06T2200/08G06T2200/24G06T2207/10028G06T2210/04G06T2210/56G06T2219/2008
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,229,887
App. No.
18/117,312
Granted
Feb 18, 2025
Kind
B2
Abstract

Disclosed are systems and method for determining information related to building materials based on determined measurements from a multi-dimensional building model comprising features and elements embodying such materials and measurements. The multi-dimensional model may be based on a plurality of received images, such as ground-based images of a building. The multi-dimensional model may be scaled, or a scale extracted based on data of the model. The multi-dimensional model may comprise architectural elements, and the scale used to determine measurements of such architectural elements. With the scaled measurements of the architectural elements in the model, product information related to multi-dimensional model and its elements may be derived and combined in alternative means.

Claims (38)

1. A method of determining comprising:

receiving a plurality of captured building images;

generating, from the plurality of captured building images, an unscaled multi-dimensional building model, wherein the unscaled multi-dimensional building model includes one or more architectural elements present in the captured building images;

extracting a scaling factor of the multi-dimensional building model, wherein extracting a scaling factor comprises correlating pixels of one or more architectural elements present in at least one of the captured building images with pixels of the one or more architectural elements of the unscaled multi-dimensional building model;

scaling the unscaled multi-dimensional building model using the scaling factor;

calculating measurements of at least one of the one or more architectural elements located within the scaled multi-dimensional building model, the measurements of the one or more architectural elements based on the scaling factor; and

determining manufacturer product information of the at least one of the one or more architectural elements corresponding to the calculated measurements.

2. The method of claim 1 , wherein extracting a scaling factor is based on one or more architectural elements present in at least one of the captured building images that were used to generate the unscaled multi-dimensional building model.

3. The method of claim 1 , wherein the manufacturer product information comprises cost and shipping information from one or more providers of manufacturer products based on the calculated measurements of the one or more architectural elements and the manufacturer product information.

4. The method of claim 1 , further comprising generating an aggregated dimension for each architectural element, within the scaled multi-dimensional model, belonging to a classification of architectural element types.

5. The method of claim 4 , wherein the aggregated dimension is based on an incremental selection of surfaces of the scaled multi-dimensional model.

6. The method of claim 5 , wherein the measurements of the one or more architectural elements is further based on the incremental selection of surfaces of the scaled multi-dimensional model.

7. The method of claim 1 , further comprising quantifying the number of architectural elements, within the scaled multi-dimensional model, belonging to a classification of architectural element type.

8. The method of claim 7 , further comprising generating an aggregated dimension for each architectural element, within the scaled multi-dimensional building model, belonging to the classification of architectural element type.

9. The method of claim 1 , further comprising identifying an associated component to architectural elements in the captured building images.

10. The method of claim 9 , wherein the associated component comprises frieze board.

11. The method of claim 9 , wherein the associated component comprises an outside corner.

12. The method of claim 9 , further comprising correlating the associated component to a measurement of at least one of the one or more architectural elements located within the scaled multi-dimensional model.

13. The method of claim 12 , further comprising determining manufacturer product information of the associated component based on the measurement of the at least one of the one or more architectural elements located within the scaled multi-dimensional model.

14. One or more non-transitory computer readable medium storing instructions that, when executed, cause a processor to execute operations comprising:

receiving a plurality of captured building images;

generating, from the plurality of captured building images, an unscaled multi-dimensional building model, wherein the unscaled multi-dimensional building model includes one or more architectural elements present in the captured building images;

extracting a scaling factor of the unscaled multi-dimensional building model, wherein extracting a scaling factor comprises correlating pixels of one or more architectural elements present in at least one of the captured building images with pixels of the one or more architectural elements of the unscaled multi-dimensional building model;

scaling the unscaled multi-dimensional building model using the scaling factor;

calculating measurements of at least one of the one or more architectural elements located within the scaled multi-dimensional building model, the measurements of the one or more architectural elements based on the scaling factor; and

determining manufacturer product information of the at least one of the one or more architectural elements corresponding to the calculated measurements.

15. The one or more non-transitory computer readable medium of claim 14 , wherein extracting a scaling factor is based on one or more architectural elements present in at least one of the captured building images that were used to generate the unscaled multi-dimensional building model.

16. The one or more non-transitory computer readable medium of claim 14 , wherein the manufacturer product information comprises cost and shipping information from one or more providers of manufacturer products based on the calculated measurements of the one or more architectural elements and the manufacturer product information.

17. The one or more non-transitory computer readable medium of claim 14 , further storing instructions that, when executed, cause the processor to execute operations comprising generating an aggregated dimension for each architectural element, within the scaled multi-dimensional model, belonging to a classification of architectural element types.

18. The one or more non-transitory computer readable medium of claim 17 , wherein the aggregated dimension is based on an incremental selection of surfaces of the scaled multi-dimensional model.

19. The one or more non-transitory computer readable medium of claim 18 , wherein the measurements of the one or more architectural elements is further based on the incremental selection of surfaces of the scaled multi-dimensional model.

20. The one or more non-transitory computer readable medium of claim 14 , further storing instructions that, when executed, cause the processor to execute operations comprising quantifying the number of architectural elements, within the scaled multi-dimensional model, belonging to a classification of architectural element type.

21. The one or more non-transitory computer readable medium of claim 20 , further storing instructions that, when executed, cause the processor to execute operations comprising generating an aggregated dimension for each architectural element, within the scaled multi-dimensional building model, belonging to the classification of architectural element type.

22. The one or more non-transitory computer readable medium of claim 14 , further storing instructions that, when executed, cause the processor to execute operations comprising identifying an associated component to architectural elements in the captured building images.

23. The one or more non-transitory computer readable medium of claim 22 , wherein the associated component comprises frieze board.

24. The one or more non-transitory computer readable medium of claim 22 , wherein the associated component comprises an outside corner.

25. The one or more non-transitory computer readable medium of claim 22 , further storing instructions that, when executed, cause the processor to execute operations comprising correlating the associated component to a measurement of at least one of the one or more architectural elements located within the scaled multi-dimensional model.

26. The one or more non-transitory computer readable medium of claim 25 , further storing instructions that, when executed, cause the processor to execute operations comprising determining manufacturer product information of the associated component based on the measurement of the at least one of the one or more architectural elements located within the scaled multi-dimensional model.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 9, 2023
From: SANJURJO, JAVIER CAMBON; CAAMANO, STEVEN; HALLIDAY, DEREK
To: HOVER INC.
Reel/Frame 062931/0141 →
Continuity (6)
Continuation 16257491 · Jan 25, 2019
Continuation In Part 15411226 · Jan 20, 2017
Continuation In Part 15255807 · Sep 2, 2016
Continuation 14339127 · Jul 23, 2014
Provisional Application 61857302 · Jul 23, 2013
Related Publication 20230206557A1 · Jun 29, 2023
References Cited (76)
US 4798028A · Pinion · 1989 [cited by applicant]
US 5973697A · Berry · 1999 [cited by applicant]
US 7218318B2 · Shimazu · 2007 [cited by applicant]
US 7814436B2 · Schrag · 2010 [cited by applicant]
US 8040343B2 · Kikuchi · 2011 [cited by applicant]
US 8098899B2 · Ohashi · 2012 [cited by applicant]
US 8139111B2 · Oldroyd · 2012 [cited by applicant]
US 8339394B1 · Lininger · 2012 [cited by applicant]
US 8350850B2 · Steedly · 2013 [cited by applicant]
US 8390617B1 · Reinhardt · 2013 [cited by applicant]
US 8842036B2 · Pritt · 2014 [cited by applicant]
US 9478031B2 · Bhatawadekar · 2016 [cited by applicant]
US 9727834B2 · Reyes · 2017 [cited by applicant]
US 20030014224A1 · Guo · 2003 [cited by applicant]
US 20030052896A1 · Higgins · 2003 [cited by applicant]
US 20040196282A1 · Oh · 2004 [cited by applicant]
US 20050099630A1 · Reynolds · 2005 [cited by applicant]
US 20060037279A1 · Onchuck · 2006 [cited by applicant]
US 20060061566A1 · Verma · 2006 [cited by applicant]
US 20070168153A1 · Minor · 2007 [cited by applicant]
US 20080221843A1 · Shenkar · 2008 [cited by applicant]
US 20090043504A1 · Bandyopadhyay · 2009 [cited by applicant]
US 20090153549A1 · Lynch · 2009 [cited by applicant]
US 20090322742A1 · Muktinulalapati · 2009 [cited by applicant]
US 20100045869A1 · Baseley · 2010 [cited by applicant]
US 20100074532A1 · Gordon · 2010 [cited by applicant]
US 20100110074A1 · Pershing · 2010 [cited by applicant]
US 20100114537A1 · Pershing · 2010 [cited by applicant]
US 20100214291A1 · Muller · 2010 [cited by applicant]
US 20110029897A1 · Russell · 2011 [cited by applicant]
US 20110037639A1 · Duran Toro · 2011 [cited by applicant]
US 20110064312A1 · Janky · 2011 [cited by applicant]
US 20110181589A1 · Quan · 2011 [cited by applicant]
US 20120101783A1 · Stephens · 2012 [cited by applicant]
US 20120237083A1 · Lange · 2012 [cited by applicant]
US 20120257792A1 · Simon · 2012 [cited by applicant]
US 20130179206A1 · Harper · 2013 [cited by applicant]
US 20130191292A1 · Meadow · 2013 [cited by applicant]
US 20130195362A1 · Janky · 2013 [cited by applicant]
US 20130202157A1 · Pershing · 2013 [cited by applicant]
US 20140023280A1 · Lin · 2014 [cited by applicant]
US 20140023996A1 · Finn · 2014 [cited by examiner]
US 20140212028A1 · Ciarcia · 2014 [cited by applicant]
US 20140214473A1 · Gentile · 2014 [cited by examiner]
US 20140278697A1 · Thornberry · 2014 [cited by examiner]
US 20160350969A1 · Castillo · 2016 [cited by applicant]
US 20180075550A1 · Thornberry · 2018 [cited by examiner]
WO 2007147830A1 · 2007 [cited by applicant]
WO 2011079241A1 · 2011 [cited by applicant]
WO 2011091552A1 · 2011 [cited by applicant]
N. Brusco et al., “A Range Camera Collecting Multi-Spectral Texture for Architecture Applications,” Third International Symposium on 3D Data Processing, Visualization, and Transmission (3DPVT'06), Chapel Hill, NC, USA, … [cited by examiner]
D. Bekins and D. G. Aliaga, “Building-by-number: rearranging the real world to visualize novel architectural spaces,” VIS 05. IEEE Visualization, 2005., 2005, pp. 143-150, doi: 10.1109/Visual.2005.1532789 (Year: 2005). [cited by applicant]
S. Teller, “Automated controlled imagery capture in urban environments,” Proceedings of the Fifth International Conference on Information Fusion. Fusion 2002. (IEEE Cat. No. 02EX5997), 2002, pp. 1098-1105 vol. 2, doi: 1… [cited by applicant]
Abdul, Hasanulhakeem1; A tool to measure dimensions of building in various scales for Google Earth Plug-ins and 3D maps; Aug. 6, 2010; pp. 1-2; downloaded from internet: [https://groups.google.com/forum/#!/google-earth-… [cited by applicant]
Xiao, et al., “Image-based Facade Modeling,” ACM Transaction on Graphics (TOG), 2008, 10 pages. [cited by applicant]
Wang, et al.; Large-Scale Urban Modeling by Combining Ground Level Panoramic and Aerial Imagery; IEEE Third International Symposium on 3D Data Processing, Visualization, and Transmission; Jun. 14-16, 2006; pp. 806-813. [cited by applicant]
Bansal, et al., “Geo-Localization of Street Views with Aerial Image Databases,” Nov. 28-Dec. 1, 2011, pp. 1125-1128. [cited by applicant]
Becker, et al. “Semiautomatic 3-D model extraction from uncalibrated 2-D camera views,” MIT Media Laboratory, 15 pages. [cited by applicant]
Caramba App Development, “EasyMeasure-Measure with your Camera on the App Store on iTunes”, https://itunes.apple.com/us/app/easymeasure-measure-measure-your-camera/id349530105mt=8, 2 pages. [cited by applicant]
Chen, et al., “City-Scale Landmark Identification on Mobile Deviceds,” pp. 737-744. [cited by applicant]
Fairfax County Virginia, “Virtual Fairfax,” http://www.fairfaxcounty.gov/gis/virtualfairfax, Feb. 24, 2014; 2 pages. [cited by applicant]
Fruh and Zakhor, “Constructing 3D City Models by Merging Aerial and Ground Views,” IEEE Computer Graphics and Applications, Nov./Dec. 2003, pp. 52-61, 10 pages. [cited by applicant]
Huang and Wu, et al. “Towards 3D City Modeling through Combining Ground Level Panoramic and Orthogonal Aerial Imagery,” 2011 Workshop on Digital Media and Digital Content Management, pp. 66-72, 6 pages. [cited by applicant]
Vosselman et al. “3D Building Model Reconstruction from Point Clouds and Ground Plans,” Oct. 2001, Natural Resources Canada, vol. 34, No. 3/W4, pp. 37-44. [cited by applicant]
International Bureau; International Preliminary Report on Patentability; International Application No. PCT/US14/062253; May 6, 2016; 8 pages. [cited by applicant]
International Bureau; International Search Report and Written Opinion; International Application No. PCT/US14/062253; Feb. 3, 2015; 9 pages. [cited by applicant]
Jaynes, “View Alignment of Aerial and Terrestrial Imagery in Urban Environments,” Springer-Verlag Berlin Heidelberg 1999, pp. 3-19, 17 pages. [cited by applicant]
Kroepfl, et al., “Efficiently Locating Photographs in Many Panoramas,” Nov. 2-5, 2010, ACM GIS10. [cited by applicant]
Lee, et al. “Automatic Integration of Facade Textures into 3D Building Models with a Projective Geometry Based Line Clustering,” Eurographics 2002, vol. 2, No. 3, 10 pages. [cited by applicant]
Lee, et al. “Integrating Ground and Aerial Views for Urban Site Modeling,” 2002, 6 pages. [cited by applicant]
Murillo et al., “Visual Door Detection Integrating Appearance and Shape Cues”, Dec. 2007, Elsevier, vol. 56, No. 6, pp. 512-521. [cited by applicant]
Ou et al., “A new method of automatic large scale map updating using mobile mapping imagery”, Sep. 2013, Wiley Online Library, vol. 28, No. 143, pp. 240-260. [cited by applicant]
Pu et al., “Automatic Extraction of Building Features from Terrestrial Laser Scanning,” 2006, International Institute for Geo-Information Science and Earth Observation, 5 pages. [cited by applicant]
Scale & Area Measurement; (n.d.). Retrieved from http://www.geog.ucsb.edu/˜jeff/115a/lectures/scale_and_area_measurement.html, 8 pages. [cited by applicant]
Scope Technologies; Solutions; Mar. 4, 2014; pp. 1-2; downloaded from the internet: [http://myscopetech.com/solutions.php]. [cited by applicant]
SketchUp Knowledge Base, Tape Measure Tool: Scaling an entire model, http://help.sketchup.com/en/article/95006, 2013 Trimble Navigation Limited, 2 pages. [cited by applicant]