IP Library › Granted Patent US 12,292,725
Granted Patent B2
US 12,292,725 · App. 17/735,563 · Granted May 6, 2025

System and method for autonomous inspection for asset maintenance and management

Inventors: Oleg Mikhailov (Piedmont, CA); Richard Clarke (Houston, TX); Dmitri Kossakovski (S. Pasadena, CA); Yaroslav Shchekaturov (Yerevan, AM)
Assignee: EXPLORATION ROBOTICS TECHNOLOGIES INC.
G05B19/406G05D1/2465G05D1/661G05B2219/32197G05D2109/10G05D2109/20
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,292,725
App. No.
17/735,563
Filed
May 3, 2022
Granted
May 6, 2025
Kind
B2
Art Unit
2117
USPC
700/108
Abstract

A method for performing an autonomous inspection. The method comprises traversing, by an autonomous sensor apparatus, a path through a site having three-dimensional objects located therein. The site includes three-dimensional objects located therein. The method comprises obtaining, by a plurality of sensors on-board the autonomous sensor apparatus, one or more data sets throughout the path. Each of the one or more data sets are associated with an attribute of one or more three-dimensional objects. The method comprises generating, by the first, second, or third processor, a working model from a collocated data set; and comparing, by the first, second, or third processor, the working model with one or more pre-existing models; to determine the presence and/or absence of anomalies. The presence and/or absence of anomalies are communicated as human-readable instructions.

Claims (26)

1. A method for performing an autonomous inspection comprising:

departing, by an autonomous sensor apparatus, a docking station, wherein a plurality of sensors are on-board the autonomous sensor apparatus, wherein the plurality of sensors include a camera sensor, a thermal sensor, and a chemical sensor that is a tunable diode laser sensor, and wherein at least one of the plurality of sensors are configured to sense a location of one or more points on at least one surface of one or more three-dimensional objects;

traversing, by the autonomous sensor apparatus, a path through a site having the one or more three-dimensional objects located therein;

locating and traveling to a proximity of, by the autonomous sensor apparatus, the one or more three-dimensional objects;

orienting toward, by the autonomous sensor apparatus and/or the plurality of sensors that are on-board the autonomous sensor apparatus, the one or more three-dimensional objects;

obtaining, by the plurality of sensors, one or more data sets along the path, each of the one or more data sets including one of a plurality of attributes of the one or more three-dimensional objects, wherein the plurality of attributes include the location of the one or more points, a visual signature at the location of the one or more points, a thermal signature at the location of the one or more points, and a chemical signature at the location of the one or more points, wherein the plurality of sensors obtain the one or more data sets from a plurality of different positions of the plurality of sensors along the path relative to the one or more points on the at least one surface of the one or more three-dimensional objects;

returning the autonomous sensor apparatus to the docking station;

pre-processing, by a processor on-board the docking station, the one or more data sets including: a) compensating, by the processor, for differences in illuminance with two-dimensional image data; b) removing, by the processor on-board the docking station, extraneous sensed data that is not associated with the at least one surface of the one or more three dimensional objects; and c) compressing the one or more data sets;

collocating, by the processor, the one or more data sets by assigning three-dimensional coordinates, wherein the assigned three-dimensional coordinates correspond to the location of the one or more points on the at least one surface of the one or more three-dimensional objects and each assigned three-dimensional coordinate corresponds individually to data points in the one or more data sets based on the location of the one or more points on the at least one surface of the one or more three-dimensional objects to produce a collocated data set, wherein the data points are arranged in a point cloud, and the data points are the visual signature, the thermal signature, and the chemical signature;

correcting the one or more data sets for a positional variance between the plurality of sensors, wherein the positional variance is corrected by a distance between the plurality of sensors; or the positional variance is corrected by a distance between each of the plurality of sensors and a point of reference;

generating, by the processor, a working model from the collocated data set by first determining the plurality of different positions of the plurality of sensors with respect to the one or more three-dimensional objects and then projecting the one or more data sets onto the working model;

determining, by the processor, an identity of the one or more three-dimensional objects using the working model;

comparing, by the processor, the working model with i) a baseline model of the one or more three-dimensional objects, the baseline model either provided by an original equipment manufacturer, created in accordance with a planned construction of the one or more three-dimensional objects, or both and ii) one or more pre-existing models to determine a presence and/or an absence of anomalies, wherein the comparing with the one or more pre-existing models includes comparing each of the data points of the working model to data points having corresponding three-dimensional coordinates of the one or more pre-existing models;

defining, by the processor, criticality of the anomalies;

receiving, by the processor, an overlay model, wherein the overlay model is an overlay of the working model, the baseline model, and the one or more pre-existing models, wherein the overlay model has a size that is 5 to 6 orders of magnitude less than the one or more data sets, and wherein the working model, the baseline model, the one or more pre-existing models, and the overlay model are three-dimensional digital models of the one or more three-dimensional objects, including the at least one surface thereof, including the plurality of attributes of the one or more three-dimensional objects; and

adjusting, autonomously by a respective controller of the one or more three-dimensional objects, one or more operating conditions of the one or more three-dimensional objects based on the determined anomalies, wherein the adjusting of the one or more operating conditions includes adjusting a temperature, adjusting a speed, adjusting a frequency, adjusting a fluid flow, or any combination thereof of the one or more three-dimensional objects.

2. The method according to claim 1 , wherein the pre-processing further comprises one or more of:

combining, by the processor, data sub-sets, wherein the data sub-sets are associated with redundant data obtained by each of the plurality of sensors which reduce the digital memory size occupied by the one or more data sets on a non-transitory storage medium and/or reduce noise of the one or more data sets;

correcting the one or more data sets for a positional variance between the plurality of sensors, wherein the positional variance is corrected by a distance between the plurality of sensors or the positional variance is corrected by a distance between each of the plurality of sensors and a point of reference;

calculating, by the processor, a mean of quantitative values associated with each of the one or more points on the at least one surface of the one or more three-dimensional objects obtained from the plurality of different positions along the path; and

correcting, by the processor, an angle of incidence of the plurality of sensors relative to the one or more points on the one or more three-dimensional objects, the angle of incidence being defined by the plurality of different positions of the plurality of sensors relative to an orthogonal axis of the one or more points on the at least one surface of the one or more three-dimensional objects.

3. The method according to claim 1 , the method further comprises:

obtaining an acoustic signature comprising acoustic data of the one or more three-dimensional objects, the acoustic data including a number of samples collected over a period of time;

deriving a first acoustic fingerprint from the acoustic signature at a first time in the period of time;

deriving a second acoustic fingerprint from the acoustic signature at a second time in the period of time; and

detecting a change in the one or more operating conditions by comparing the first acoustic fingerprint and the second acoustic fingerprint.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded May 27, 2022
From: MIKHAILOV, OLEG; CLARKE, RICHARD; KOSSAKOVSKI, DMITRI; SHCHEKATUROV, YAROSLAV
To: EXPLORATION ROBOTICS TECHNOLOGIES INC.
Reel/Frame 060036/0109 →
Continuity (3)
Provisional Application 63307879 · Feb 8, 2022
Provisional Application 63242818 · Sep 10, 2021
Related Publication 20230079054A1 · Mar 16, 2023
References Cited (69)
US 7357179B2 · Zheng et al. · 2008 [cited by applicant]
US 8242445B1 · Scanlon et al. · 2012 [cited by applicant]
US 8625854B2 · Valkenburg et al. · 2014 [cited by applicant]
US 8872818B2 · Freeman et al. · 2014 [cited by applicant]
US 8929176B2 · Debrunner et al. · 2015 [cited by applicant]
US 8942062B2 · Debrunner et al. · 2015 [cited by applicant]
US 9082015B2 · Christopulos et al. · 2015 [cited by applicant]
US 9129355B1 · Harvey et al. · 2015 [cited by applicant]
US 9262789B1 · Tofte · 2016 [cited by applicant]
US 9607370B2 · Freeman · 2017 [cited by examiner]
US 9619691B2 · Pang et al. · 2017 [cited by applicant]
US 9709394B2 · Schoner et al. · 2017 [cited by applicant]
US 10055831B2 · Loveland et al. · 2018 [cited by applicant]
US 10108867B1 · Vallespi-Gonzalez · 2018 [cited by examiner]
US 10127685B2 · Qian et al. · 2018 [cited by applicant]
US 10186049B1 · Boardman et al. · 2019 [cited by applicant]
US 10260875B2 · Schoner et al. · 2019 [cited by applicant]
US 10351240B1 · Sills et al. · 2019 [cited by applicant]
US 10586385B2 · Moghadam et al. · 2020 [cited by applicant]
US 10913154B2 · Tan et al. · 2021 [cited by applicant]
US 11435746B1 · Ebrahimi Afrouzi · 2022 [cited by examiner]
US 20010022614A1 · Lida et al. · 2001 [cited by applicant]
US 20090138140A1 · Pinney · 2009 [cited by examiner]
US 20140067333A1 · Rodney et al. · 2014 [cited by applicant]
US 20140270492A1 · Christopulos · 2014 [cited by examiner]
US 20160019108A1 · Banerjee · 2016 [cited by examiner]
US 20170028561A1 · Yamada · 2017 [cited by examiner]
US 20170322551A1 · Zang · 2017 [cited by examiner]
US 20180002010A1 · Bauer · 2018 [cited by examiner]
US 20180095458A1 · Sharber · 2018 [cited by examiner]
US 20180130196A1 · Loveland · 2018 [cited by examiner]
US 20180321692A1 · Castillo-Effen · 2018 [cited by examiner]
US 20180329433A1 · Zhao et al. · 2018 [cited by applicant]
US 20180329904A1 · Gupta et al. · 2018 [cited by applicant]
US 20190066283A1 · Gros · 2019 [cited by examiner]
US 20190096069A1 · Qian · 2019 [cited by examiner]
US 20190108396A1 · Dal Mutto et al. · 2019 [cited by applicant]
US 20190188521A1 · Barzelay et al. · 2019 [cited by applicant]
US 20190242696A1 · McAleenan et al. · 2019 [cited by applicant]
US 20190253623A1 · Lim · 2019 [cited by examiner]
US 20200019167A1 · Alshamrani · 2020 [cited by examiner]
US 20200045880A1 · Letsky · 2020 [cited by examiner]
US 20200066034A1 · Tham · 2020 [cited by examiner]
US 20200082168A1 · Fathi · 2020 [cited by examiner]
US 20200174129A1 · Abdelkader et al. · 2020 [cited by applicant]
US 20200219234A1 · Sotodate · 2020 [cited by examiner]
US 20200279320A1 · Stewart · 2020 [cited by examiner]
US 20200306969A1 · Bryner · 2020 [cited by examiner]
US 20200401138A1 · Rentz · 2020 [cited by examiner]
US 20210357664A1 · Kocer · 2021 [cited by examiner]
US 20220004672A1 · Santarone · 2022 [cited by examiner]
US 20220024577A1 · Stamatovski · 2022 [cited by examiner]
US 20220050461A1 · Karachalios · 2022 [cited by examiner]
US 20220107977A1 · Marthouse · 2022 [cited by examiner]
US 20220157136A1 · Metzler · 2022 [cited by examiner]
US 20220383128A1 · Gonzales · 2022 [cited by examiner]
US 20230221695A1 · Shao · 2023 [cited by examiner]
US 20240201240A1 · Blanc-Paques · 2024 [cited by examiner]
US 20240361124A1 · Michini · 2024 [cited by examiner]
CA 3039506A1 · 2018 [cited by examiner]
EP 3103043B1 · 2018 [cited by examiner]
EP 3996058A1 · 2022 [cited by examiner]
JP 2004005449A · 2004 [cited by examiner]
WO WO2020088739A1 · 2020 [cited by examiner]
WO WO2022098407A1 · 2022 [cited by examiner]
International Search Report and Written Opinion mailed on Nov. 3, 2022 for International Application PCT/US2021/044685. [cited by applicant]
European Search Report dated on Jan. 23, 2023 for EP Application No. 22194426.7. [cited by applicant]
Extended European Search Report for European Patent Application No. 21889774.2, mailed Sep. 19, 2024, 11 pages. [cited by applicant]
Examination Report for European Patent Application No. 22194426.7, mailed Nov. 25, 2024, 5 pages. [cited by applicant]