IP Library Granted Patent US 12,482,117
Granted Patent B2
US 12,482,117 · App. 18/349,324 · Granted Nov 25, 2025

Systems and methods for automated detection of changes in extent of structures using imagery

Inventors: Stephen Ng (Rochester, NY); David R. Nilosek (Rochester, NY); Phillip Salvaggio (Rochester, NY); Shadrian Strong (Bellevue, WA)
Assignee: Pictometry International Corp.
G06T7/344G06F18/21G06F18/22G06F18/24G06N3/045G06N3/088G06T7/13G06T7/70G06V10/44G06V10/7553G06V10/82G06V20/176G06V20/182G06V20/653G06T2207/10032G06T2207/20084
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,482,117
App. No.
18/349,324
Granted
Nov 25, 2025
Kind
B2
Abstract

Systems and methods for automated detection of changes in extent of structures using imagery are disclosed, including a non-transitory computer readable medium storing computer executable code that when executed by a processor cause the processor to: align an outline of a structure at a first instance of time to pixels within an image depicting the structure, the image captured at a second instance of time; assess a degree of alignment between the outline and the pixels within the image depicting the structure, using a machine learning model to generate an alignment confidence score; determine an existence of a change in extent of the structure based upon the alignment confidence score indicating that the outline and the pixels within the image are not aligned; identify a shape of the change in extent of the structure; and store the shape of the change in extent of the structure.

Claims (41)

1 . A non-transitory computer readable medium storing computer-executable instructions that when executed by one or more computer processors cause the one or more computer processors to:

align an outline of a structure at a first instance of time to pixels within an image depicting the structure by overlaying the outline of the structure onto the pixels within the image depicting the structure, the image captured at a second instance of time;

assess a degree of alignment between the outline and the pixels within the image depicting the structure, using a machine learning model to generate an alignment confidence score;

determine an existence of a change in extent of the structure based upon the alignment confidence score indicating that the outline and the pixels within the image depicting the structure are not aligned;

identify a shape of the change in extent of the structure; and

store the shape of the change in extent of the structure.

2 . The non-transitory computer readable medium of claim 1 , wherein the machine learning model is a convoluted neural network image classifier model.

3 . The non-transitory computer readable medium of claim 1 , wherein the machine learning model is a generative adversarial network image classifier model.

4 . The non-transitory computer readable medium of claim 1 , further storing computer-executable instructions that when executed by one or more computer processors cause the one or more computer processors to:

identify the shape of the change in the structure using any one or more of: a point cloud estimate, a convolutional neural network, a generative adversarial network, and a feature detection technique.

5 . The non-transitory computer readable medium of claim 1 , wherein aligning the outline further comprises:

creating the outline using an image of the structure captured at the first instance of time.

6 . The non-transitory computer readable medium of claim 1 , wherein the outline is a first outline and wherein the alignment confidence score is determined by analyzing shape intersection between the first outline and a second outline of the structure depicted within the pixels of the image.

7 . The non-transitory computer readable medium of claim 1 , wherein the outline is a vector boundary describing an extent of the structure at the first instance of time.

8 . The non-transitory computer readable medium of claim 1 , wherein the first instance of time is before the second instance of time.

9 . The non-transitory computer readable medium of claim 1 , wherein the first instance of time is after the second instance of time.

10 . The non-transitory computer readable medium of claim 1 , wherein aligning the outline further comprises:

detecting edges of the structure in the image;

determining one or more shift distance between the outline and one or more edges of the detected edges of the structure in the image; and

shifting the outline by the shift distance.

11 . The non-transitory computer readable medium of claim 10 , further storing computer-executable instructions that when executed by one or more computer processors cause the one or more computer processors to:

determine a structural modification based on the existence of the change and on a comparison between the outline and the pixels within the image depicting the structure, after the outline is shifted by the shift distance.

12 . A method, comprising:

aligning, with one or more computer processors, an outline of a structure at a first instance of time to pixels within an image depicting the structure by overlaying the outline of the structure onto the pixels within the image depicting the structure, the image captured at a second instance of time;

assessing, with the one or more computer processors, a degree of alignment between the outline and the pixels within the image depicting the structure, using a machine learning model to generate an alignment confidence score; and

determining, with the one or more computer processors, an existence of a change in the structure based upon the alignment confidence score indicating that the outline and the pixels within the image depicting the structure are not aligned;

identifying, with the one or more computer processors, a shape of the change in extent of the structure; and

storing the shape of the change in extent of the structure.

13 . The method of claim 12 , wherein the machine learning model is at least one of a convoluted neural network image classifier model and a generative adversarial network image classifier model.

14 . The method of claim 12 , wherein identifying, with the one or more computer processors, the shape of the change in extent of the structure utilizes any one or more of: a point cloud estimate, a convolutional neural network, a generative adversarial network, and a feature detection technique.

15 . The method of claim 12 , wherein aligning the outline further comprises:

creating, with the one or more computer processors, the outline using an image of the structure captured at the first instance of time.

16 . The method of claim 12 , wherein the outline is a first outline, and wherein the alignment confidence score is determined by analyzing shape intersection between the first outline and a second outline of the structure depicted within the pixels of the image.

17 . The method of claim 12 , wherein the outline is a vector boundary describing an extend of the structure at the first instance of time.

18 . The method of claim 12 , wherein the first instance of time is before the second instance of time or the first instance of time is after the second instance of time.

19 . The method of claim 12 , wherein aligning the outline further comprises:

detecting edges of the structure in the image;

determining one or more shift distance between the outline and one or more edges of the detected edges of the structure in the image; and

shifting the outline by the shift distance.

20 . The method of claim 19 , further comprising:

determining, with the one or more computer processors, a structural modification based on the existence of the change and on a comparison between the outline and the pixels within the image depicting the structure, after the outline is shifted by the shift distance.

Assignments (2)
FIRST LIEN SECURITY AGREEMENT Recorded Mar 28, 2025
From: EAGLE VIEW TECHNOLOGIES, INC.; PICTOMETRY INTERNATIONAL CORP.
To: MORGAN STANLEY SENIOR FUNDING, INC.
Reel/Frame 070671/0078 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jul 10, 2023
From: NG, STEPHEN; NILOSEK, DAVID R.; SALVAGGIO, PHILLIP; STRONG, SHADRIAN
To: PICTOMETRY INTERNATIONAL CORP.
Reel/Frame 064197/0370 →
Continuity (4)
Continuation 17587795 · Jan 28, 2022
Continuation 16891982 · Jun 3, 2020
Provisional Application 62858656 · Jun 7, 2019
Related Publication 20230351620A1 · Nov 2, 2023
References Cited (55)
US 7424133B2 · Schultz et al. · 2008 [cited by applicant]
US 7650047B2 · Jin et al. · 2010 [cited by applicant]
US 8259993B2 · Sasakawa et al. · 2012 [cited by applicant]
US 8285050B2 · Ozawa · 2012 [cited by applicant]
US 8385672B2 · Giuffrida et al. · 2013 [cited by applicant]
US 9389084B1 · Chen et al. · 2016 [cited by applicant]
US 9805261B1 · Loveland et al. · 2017 [cited by applicant]
US 11238282B2 · Ng · 2022 [cited by examiner]
US 11699241B2 · Ng · 2023 [cited by examiner]
US 20020051572A1 · Matsumoto et al. · 2002 [cited by applicant]
US 20090067725A1 · Sasakawa et al. · 2009 [cited by applicant]
US 20100246971A1 · Ohtsubo et al. · 2010 [cited by applicant]
US 20160005097A1 · Hsiao et al. · 2016 [cited by applicant]
US 20160027051A1 · Gross · 2016 [cited by applicant]
US 20160217537A1 · Childs · 2016 [cited by applicant]
US 20160259994A1 · Ravindran et al. · 2016 [cited by applicant]
US 20160371801A1 · Dawson · 2016 [cited by applicant]
US 20170091627A1 · Terrazas et al. · 2017 [cited by applicant]
US 20170236024A1 · Wang et al. · 2017 [cited by applicant]
US 20170308549A1 · Sims et al. · 2017 [cited by applicant]
US 20180268239A1 · Coulter et al. · 2018 [cited by applicant]
US 20190065907A1 · Strong et al. · 2019 [cited by applicant]
US 20190087952A1 · Milbert et al. · 2019 [cited by applicant]
GB 2554361A · 2018 [cited by applicant]
WO 2006120724 · 2016 [cited by applicant]
WO WO2018071983A1 · 2018 [cited by applicant]
ESRI, Getting Started with ArcGIS, 2005, available at: <http://downloads.esri.com/support/documentation/ao_/1003Getting Started_with_ArcGIS.pdf >, downloaded on Sep. 1, 2019 (Year: 2005). [cited by applicant]
Holt, Alec, “Spatial similarity and GIS: the grouping of spatial kinds,” 1999, available at: < https://pdfs.semanticscholar.org/1 ff7/49c905db58bb884a9d6609de9452edf7921 b.pdf >, downloaded on Sep. 1, 2019 (Year: 1999). [cited by applicant]
Artificial Neural Networks Technology, [online], archived on Nov. 23, 2015, available at:<https://web.archive.org/web/20151123211404/http://www2.psych.utoronto.ca/users/relngold/course> (Year: 2015). [cited by applicant]
Marr, Bernard, “A Short History of Machine Learning—Every Manager Should Read,” [online], Feb. 19, 2016, available at: < https://www .forbes.com/sites/bernardmarr/2016/02/19/a-short-history-of-machine-learning-every-man… [cited by applicant]
Krizhevsky et al., “ImageNet Classification with Deep Convolutional Neural Networks,” Advances in Neural Information Processing Systems 25, pp. 1097-1105, 2012. [cited by applicant]
Long et al., “Fully Convolutional Networks for Semantic Segmentation,” IEEE Conference on Computer Vision and Pattern Recognition, Jun. 2015. [cited by applicant]
Goodfellow, et al., “Generative Adversarial Networks,” Departement d'informatique et de recherche operationnelle Universite de Montreal, Jun. 2014. [cited by applicant]
Simonyan et al., “Very Deep Convolutional Networks for Large-Scale Image Recognition” International Conference on Machine Learning, 2015. [cited by applicant]
USPTO, Advisory Action regarding U.S. Appl. No. 15/634,879, dated Jan. 19, 2022. [cited by applicant]
Omniearth, Inc., Response to Jan. 19, 2022 Advisory Action and Sep. 14, 2021 Office Action regarding U.S. Appl. No. 15/634,879, filed Feb. 14, 2022. [cited by applicant]
USPTO, Notice of Allowance for U.S. Appl. No. 15/634,879, dated Mar. 17, 2022. [cited by applicant]
Qin et al., “3D Change Detection—Approaches and Applications”, ISPRS Journal of Photogrammetry and Remote Sensing, pp. 41-56, 2016. [cited by applicant]
Bouziani et al., “Automatic Change Detection of Buildings in Urban Environment from Very High Spatial Resolution Images Using Existing Geodatabase and Prior Knowledge”, ISPRS Journal of Photogrammetry and Remote Sensing… [cited by applicant]
Taneja et al., “Image Based Detection of Geometric Changes in Urban Environments”, 2011 International Conference on Computer Vision, 2011. [cited by applicant]
Ruf et al., “Towards Real-Time Change Detection in Videos Based on Existing 3D Models”, Image and Signal Processing for Remote Sensing XXII, vol. 10004, 2016. [cited by applicant]
Leichtle et al., “Unsupervised Change Detection in VHR Remote Sensing Imagery—An Object Based Clustering Approach in a Dynamic Urban Environment”, International Journal of Applied Earth Observation and Geoinformation 54… [cited by applicant]
European Patent Office acting as the International Searching Authority, International Search Report and Written Opinion regarding PCT App. No. PCT/US20/035945 mailed Sep. 18, 2020. [cited by applicant]
Galarreta et al., “UAV-based Urban Structural Damage Assessment Using Object-based Image Analysis and Semantic Reasoning,” Nat. Hazards Earth Syst. Sci., 15, 1087-1101; Copernicus Publications; 2015. [cited by applicant]
Galarreta, Jorge Fernandez; “Urban Structural Damage Assessment Using Object-Oriented Analysis and Semantic Reasoning,” Thesis submitted to the Faculty of Geo-Information Science and Earth Observation of the University … [cited by applicant]
Kang et al., “Building Instance Classification Using Street View Images,” ISPRS Journal of Photogrammetry and Remote Sensing, 145, 44-59, Elsevier, Mar. 2018. [cited by applicant]
United Kingdom Intellectual Property Office; First Examination Report regarding UK Patent Application No. GB2116809.1, Dated Mar. 30, 2023. [cited by applicant]
Canadian Intellectual Property Office, Office Action regarding Canadian Patent Application No. 3,139,486, Dated Mar. 25, 2024. [cited by applicant]
Pictometry International Corp., Response to Mar. 30, 2023 First Examination Report regarding UK Patent Application No. GB2116809.1, Dated Jul. 27, 2023. [cited by applicant]
IP Australia; Examination Report No. 1 regarding Australian Patent Application No. 2020287611, Dated May 22, 2024. [cited by applicant]
Ji, S., et al., “Building Instance Change Detection from Large-Scale Aerial Images using Convolutional Neural Networks and Simulated Samples”, Remote Sensing, Jun. 4, 2019, <URL: https://www.mdpi.com/2072-4292/11/11/134… [cited by applicant]
Liu, H., et al., “Line-Constrained Shape Feature for Building Change Detection in VHR Remote Sensing Imagery”, ISPRS International Journal of Geo-Information, Oct. 16, 2018, <URL: https://www.mdpi.com/2220-9964/7/10/410… [cited by applicant]
Gong, J., et al., “Patch Matching and Dense CRF-Based Co-Refinement for Building Change Detection from Bi-Temporal Aerial Images”, Sensors, Mar. 31, 2019, <URL: https://www.mdpi.com/1424-8220/19/7/1557>. [cited by applicant]
Pang, S., et al., “Co-Segmentation and Superpixel-Based Graph Cuts for Building Change Detection from Bi-Temporal Digital Surface Models and Aerial Images”, Remote Sensing, Mar. 26, 2019, <URL: https://www.mdpi.com/2072… [cited by applicant]
Pictometry International Corp.; Response to Mar. 25, 2024 Office Action regarding Canadian Patent Application No. 3,139,486, Dated Jul. 22, 2024. [cited by applicant]