IP Library Granted Patent US 10,643,379
Granted Patent B2
US 10,643,379 · App. 16/045,636 · Granted May 5, 2020

Systems and methods for facilitating imagery and point-cloud based facility modeling and remote change detection

Inventors: Scott Nowicki (Portland, OR); David Brandt (Portland, OR); Richard Armstrong (Portland, OR); Seth Hill (Portland, OR)
Assignee: Quantum Spatial, Inc.
G06T17/05G01J5/601G01S7/4808G01S17/89G06T17/20G06T19/00G06T2207/10024G06T2207/10028G06T2207/10036G06T2210/56
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,643,379
App. No.
16/045,636
Granted
May 5, 2020
Kind
B2
Abstract

Various embodiments are directed to facilitating imagery and point-cloud based facility modeling and remote change detection. A computing device may receive collected data for a facility. The collected data may include spatial image data obtained from light detection imaging and ranging systems (LiDAR), multispectral data, and thermal data. The computing device may then analyze, based on software models generated for previously collected data for the facility, the collected data to determine changes in the previously collected data. The computing device may then update the models upon determining changes in the previously collected data. Finally, the computing device may generate an alert based on the updated models when any changes in the previously collected data are above a predetermined threshold corresponding to a current security or operational condition associated with the facility.

Claims (56)

1. A system for facilitating imagery and point-cloud based facility modeling and remote change detection, the system comprising:

at least one processor; and

at least one memory storing computer-executable instructions that when executed by the at least one processor, cause the system to:

receive, by a computing device, collected data for a facility;

analyze, by the computing device, based on one or more models generated for previously collected data for the facility, the collected data to determine at least one change in the previously collected data, wherein the models are generated by:

receiving spatial image data in the previously collected data, the spatial image data describing elements associated with a physical structure of the facility, wherein the spatial image data comprises image-derived point cloud data generated by one or more autonomous vehicles in proximity to the facility, the autonomous vehicles comprising at least one of a ground and airborne light imaging direction and ranging (LiDAR) system;

generating, from the spatial image data, a first three-dimensional model identifying and coding each of the elements;

assigning a first base-level buffer to each of the elements, the first base level buffer representing a variability threshold with respect to dimensions and characteristics associated with each of the elements, wherein the elements associated with the physical structure of the facility comprise at least a fence surrounding a perimeter of the facility and a transformer within the facility, wherein the facility comprises a public utility substation;

receiving multispectral and thermal image data in the previously collected data, the multispectral and thermal image data describing a color, spectral reflectance, and temperature range for each of the elements from one or more multispectral and thermal cameras;

generating, from the multispectral and thermal image data, a second three-dimensional model identifying and coding each of the elements; and

assigning a second base-level buffer to each of the elements, the second-base level buffer representing a variability threshold with respect to the color, spectral reflectance, and temperature range for each of the elements, wherein the variability threshold with respect to the color, spectral reflectance, and temperature range for each of the elements comprises at least a presence of moisture associated with the transformer based on the spectral reflectance;

update, by the computing device, the models upon determining the at least one change in the previously collected data; and

generate, by the computing device, an alert based on the updated models when the at least one change in the previously collected data is above a predetermined threshold corresponding to a current security or operational condition associated with the facility, wherein the alert is generated, at least in part, by determining that the at least one change in the previously collected data exceeds the variability threshold represented by the second base-level buffer for the transformer, wherein the variability threshold is exceeded when the transformer is no longer operating between an operational range of current and voltage levels due to the presence of the moisture.

2. The system of claim 1 , wherein the elements associated with the physical structure of the facility comprise:

fixed and dynamic objects within the facility; and

fixed and dynamic objects in a proximity to the facility.

3. The system of claim 1 , wherein the collected data is analyzed by:

comparing the first and second three-dimensional models to new spatial image, multispectral, and thermal image data in the collected data; and

determining a change in at least one of the elements based on the comparison.

4. The system of claim 1 , wherein the spatial image data comprises image data generated by one or more fixed sensors associated with the facility.

5. A computer-implemented method for facilitating imagery and point-cloud based facility modeling and remote change detection, at least a portion of the method being performed by a computing device comprising at least one processor, the method comprising:

receiving, by the computing device, collected data for a facility;

analyzing, by the computing device, based on one or more models generated for previously collected data for the facility, the collected data to determine at least one change in the previously collected data, wherein the models are generated by:

receiving spatial image data in the previously collected data, the spatial image data describing elements associated with a physical structure of the facility, wherein the spatial image data comprises image-derived point cloud data generated by one or more autonomous vehicles in proximity to the facility, the autonomous vehicles comprising at least one of a ground and airborne light imaging direction and ranging (LiDAR) system;

generating, from the spatial image data, a first three-dimensional model identifying and coding each of the elements;

assigning a first base-level buffer to each of the elements, the first base-level buffer representing a variability threshold with respect to dimensions and characteristics associated with each of the elements, wherein the elements associated with the physical structure of the facility comprise at least a fence surrounding a perimeter of the facility and a transformer within the facility, wherein the facility comprises a public utility substation;

receiving multispectral and thermal image data in the previously collected data, the multispectral and thermal image data describing a color, spectral reflectance, and temperature range for each of the elements from one or more multispectral and thermal cameras;

generating, from the multispectral and thermal image data, a second three-dimensional model identifying and coding each of the elements; and

assigning a second base-level buffer to each of the elements, the second-base level buffer representing a variability threshold with respect to the color, spectral reflectance, and temperature range for each of the elements, wherein the variability threshold with respect to the color, spectral reflectance, and temperature range for each of the elements comprises at least a presence of moisture associated with the transformer based on the spectral reflectance;

updating, by the computing device, the models upon determining the at least one change in the previously collected data; and

generating, by the computing device, an alert based on the updated models when the at least one change in the previously collected data is above a predetermined threshold corresponding to a current security or operational condition associated with the facility, wherein the alert is generated, at least in part, by determining that the at least one change in the previously collected data exceeds the variability threshold represented by the second base-level buffer for the transformer, wherein the variability threshold is exceeded when the transformer is no longer operating between an operational range of current and voltage levels due to the presence of the moisture.

6. The computer-implemented method of claim 5 , wherein the elements associated with the physical structure of the facility comprise:

fixed and dynamic objects within the facility; and

fixed and dynamic objects in proximity to the facility.

7. The computer-implemented method of claim 5 , wherein analyzing the collected data comprises:

comparing the first and second three-dimensional models to new spatial image, multispectral, and thermal image data in the collected data; and

determining a change in at least one of the elements based on the comparison.

8. The computer-implemented method of claim 5 , wherein the spatial image data comprises image data generated by one or more fixed sensors associated with the facility.

9. A non-transitory computer-readable medium comprising one or more computer-executable instructions that, when executed by at least one processor of a computing device, cause the computing device to:

receive, by the computing device, collected data for a facility;

analyze, by the computing device, based on one or more models generated for previously collected data for the facility, the collected data to determine at least one change in the previously collected data, wherein the models are generated by:

receiving spatial image data in the previously collected data, the spatial image data describing elements associated with a physical structure of the facility, wherein the spatial image data comprises image-derived point cloud data generated by one or more autonomous vehicles in proximity to the facility, the autonomous vehicles comprising at least one of a ground and airborne light imaging direction and ranging (LiDAR) system;

generating, from the spatial image data, a first three-dimensional model identifying and coding each of the elements;

assigning a first base-level buffer to each of the elements, the first base-level buffer representing a variability threshold with respect to dimensions and characteristics associated with each of the elements, wherein the elements associated with the physical structure of the facility comprise at least a fence surrounding a perimeter of the facility and a transformer within the facility, wherein the facility comprises a public utility substation;

receiving multispectral and thermal image data in the previously collected data, the multispectral and thermal image data describing a color, spectral reflectance, and temperature range for each of the elements from one or more multispectral and thermal cameras;

generating, from the multispectral and thermal image data, a second three-dimensional model identifying and coding each of the elements; and

assigning a second base-level buffer to each of the elements, the second-base level buffer representing a variability threshold with respect to the color, spectral reflectance, and temperature range for each of the elements, wherein the variability threshold with respect to the color, spectral reflectance, and temperature range for each of the elements comprises at least a presence of moisture associated with the transformer based on the spectral reflectance;

update, by the computing device, the models upon determining the at least one change in the previously collected data; and

generate, by the computing device, an alert based on the updated models when the at least one change in the previously collected data is above a predetermined threshold corresponding to a current security or operational condition associated with the facility, wherein the alert is generated, at least in part, by determining that the at least one change in the previously collected data exceeds the variability threshold represented by the second base-level buffer for the transformer, wherein the variability threshold is exceeded when the transformer is no longer operating between an operational range of current and voltage levels due to the presence of the moisture.

10. The non-transitory computer-readable medium of claim 9 , wherein the elements associated with the physical structure of the facility comprise:

fixed and dynamic objects within the facility; and

fixed and dynamic objects in proximity to the facility.

11. The non-transitory computer-readable medium of claim 9 , wherein the collected data is analyzed by:

comparing the first and second three-dimensional models to new spatial image, multispectral, and thermal image data in the collected data; and

determining a change in at least one of the elements based on the comparison.

12. The non-transitory computer-readable medium of claim 9 , wherein the spatial image data comprises image data generated by one or more fixed sensors associated with the facility.

Assignments (7)
JOINDER TO PATENT SECURITY AGREEMENT Recorded Nov 5, 2025
From: NV5 GEOSPATIAL, INC.
To: JEFFERIES FINANCE LLC, AS COLLATERAL AGENT
Reel/Frame 073463/0044 →
RELEASE OF SECURITY INTEREST Recorded Aug 4, 2025
From: BANK OF AMERICA, N.A., AS ADMINISTRATIVE AGENT
To: NV5 GEOSPATIAL, INC. (FORMERLY NAMED QUANTUM SPATIAL, INC.)
Reel/Frame 071928/0192 →
CHANGE OF NAME Recorded Feb 15, 2023
From: QUANTUM SPATIAL, INC.
To: NV5 GEOSPATIAL, INC.
Reel/Frame 062762/0474 →
RELEASE OF SECURITY INTEREST Recorded Jan 17, 2020
From: TWIN BROOK CAPITAL PARTNERS, LLC
To: QUANTUM SPATIAL, INC.
Reel/Frame 051542/0759 →
SECURITY INTEREST Recorded Dec 23, 2019
From: QUANTUM SPATIAL, INC.
To: BANK OF AMERICA, N.A., AS ADMINISTRATIVE AGENT
Reel/Frame 051352/0311 →
CORRECTIVE ASSIGNMENT TO CORRECT THE INCORRECT APPLICATION NUMBER 29/688475 PREVIOUSLY RECORDED ON REEL 050277 FRAME 0594. ASSIGNOR(S) HEREBY CONFIRMS THE PATENT SECURITY AGREEMENT. Recorded Sep 23, 2019
From: QUANTUM SPATIAL, INC.
To: TWIN BROOK CAPITAL PARTNERS, LLC, AS AGENT
Reel/Frame 050461/0974 →
SECURITY INTEREST Recorded Sep 5, 2019
From: QUANTUM SPATIAL, INC.
To: TWIN BROOK CAPITAL PARTNERS, LLC, AS AGENT
Reel/Frame 050277/0594 →
Continuity (2)
Provisional Application 62539179 · Jul 31, 2017
Related Publication 20190035146A1 · Jan 31, 2019