IP Library Granted Patent US 10,878,556
Granted Patent B2
US 10,878,556 · App. 15/875,607 · Granted Dec 29, 2020

Interactive semi-automated borescope video analysis and damage assessment system and method of use

Inventors: Paul D. Hestand (Amesbury, MA); Alan Matthew Finn (Hebron, CT); Dennis M. Moura (South Windsor, CT); Ziyou Xiong (Wethersfield, CT); Keith R. Russell (Middletown, CT)
Assignee: United Technologies Corporation
G06T7/001G01N21/8851G01N21/954G01N21/9515G06F16/5866G06T7/11G06T7/62G01N2021/8874G01N2021/8877G06T2207/30164
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,878,556
App. No.
15/875,607
Filed
Jan 19, 2018
Granted
Dec 29, 2020
Kind
B2
Art Unit
2669
USPC
382/152
Abstract

A method of assessing damage to a component includes displaying a sensor image of the component in a first viewing pane, displaying a reference image of the component, which is a graphical depiction of the component with accurate dimensions, in a second viewing pane, placing a plurality of first identification markers on the sensor image of the component in the first viewing pane to correspond to a matching location with a second identification marker on the component in the reference image, identifying a region of damage on the component in the sensor image, mapping the region of damage to the component in the reference image using the plurality of first and second identification markers, and calculating a size of the region of damage.

Claims (50)

1. A method of assessing damage to a component, the method comprising:

displaying a sensor image of the component in a first viewing pane;

displaying a reference image of the component in a second viewing pane, wherein the reference image is a graphical depiction of the component with accurate dimensions;

placing a plurality of first identification markers on the sensor image of the component in the first viewing pane, wherein each of the first identification markers corresponds to a matching location with a second identification marker on the component in the reference image;

identifying a region of damage on the component in the sensor image;

mapping the region of damage on the sensor image of the component to the reference image using the plurality of first and second identification markers, wherein mapping the region of damage is performed by a processor using image transformation;

evaluating, with the processor, whether a poor transformation is about to occur; and

responding to an evaluation that a poor transformation is about to occur by prompting the user to reposition one or more of the first or second identification markers to improve transformation;

calculating a size of the region of damage.

2. The method of claim 1 and further comprising:

identifying a type of damage in the region of damage in the sensor image.

3. The method of claim 1 and further comprising:

storing at least one data file to a database, wherein the data file is selected from the group consisting of the sensor image, the mapped region of damage, and the calculated size of the region of damage.

4. The method of claim 3 and further comprising:

repeating the steps of claim 1 for each of a plurality of sensor images of the component, wherein each sensor image provides a view of the component at a different date of operation; and

storing a plurality of data files to the database, wherein the plurality of data files is selected from the group consisting of the plurality of sensor images, the plurality of mapped regions of damage, and the plurality of calculated sizes of the region of damage.

5. The method of claim 4 and further comprising:

retrieving previously stored data files from the database, wherein the previously stored data files are selected from the group consisting of the plurality of sensor images, the plurality of mapped regions of damage, and the plurality of calculated sizes of the region of damage; and

displaying a history of the region of damage for the component.

6. The method of claim 3 and further comprising:

repeating the steps of claim 1 for each of a plurality of sensor images of components of the same type operating in different assemblies, wherein each sensor image provides a view of a component obtained from a different assembly; and

storing a plurality of data files to the database, wherein the plurality of data files is selected from the group consisting of the plurality of sensor images, the plurality of mapped regions of damage, and the plurality of calculated sizes of the region of damage.

7. The method of claim 6 and further comprising:

retrieving previously stored data files from the database, wherein the previously stored data files are selected from the group consisting of the plurality of sensor images, the plurality of mapped regions of damage, and the plurality of calculated sizes of the region of damage; and

displaying the previously stored data to compare the region of damage between the components operating in different assemblies.

8. A system for performing semi-automated damage detection and assessment, the system comprising:

a processor configured to receive and process data;

a database for storing data, wherein data comprises sensor images and reference images of a plurality of components;

a graphical user interface comprising:

a first viewing pane comprising:

a sensor image of a first component in a first orientation and having a region of damage; and

a plurality of first identification markers positionable by a user within the first viewing pane; and

a second viewing pane comprising:

a reference image of the first component, wherein the reference image is a graphical depiction of the component with accurate dimensions and is in a second orientation different than the first orientation;

a plurality of second identification markers positionable by a user within the second viewing pane;

wherein each of the second identification markers is positioned to correspond to a first identification marker and wherein pairs of corresponding first and second identification markers are positioned on the same location on the component; and

a region indicator corresponding to a location of the region of damage of the sensor image, wherein the region indicator delineates a size of the region of damage with absolute metrology on the reference image;

wherein the processor is capable of detecting when a poor transformation is about to occur and prompting the user to reposition one or more of the first or second identification markers to improve transformation.

9. The system of claim 8 , wherein the reference image is a two-dimensional projection of a three-dimensional CAD model of the component.

10. The system of claim 8 , wherein the region of damage is automatically identified and delineated by the processor.

11. The system of claim 8 , wherein a viewing angle of the first component in the sensor image is different than a viewing angle of the first component in the reference image.

12. The system of claim 8 , wherein the region of damage in the first viewing pane is mapped by transformation from the sensor image to the reference image using the first and second identification markers.

13. The system of claim 8 , wherein the region of damage is classified according to a type of damage present, wherein classification is performed automatically by the processor or performed manually by a user by selecting from a pre-populated menu of damage types.

14. The system of claim 13 , wherein at least one data file is saved to the database, wherein the at least one data file is selected from the group consisting of the sensor image, the mapped region of damage, and the calculated size of the region of damage, and the type of damage.

15. The system of claim 14 , wherein the database comprises a plurality of data files collected over a period of time during which the first component was susceptible to damage by damage-causing elements, wherein the plurality of data files is selected from the group consisting of a plurality of sensor images, a plurality of mapped regions of damage, a plurality of calculated sizes of the region of damage, and a plurality of the types of damage.

16. The system of claim 15 and further comprising:

a third viewing pane comprising aggregate information from the plurality of data files to provide a history of the region of damage.

17. The system of claim 14 , wherein the database comprises a plurality of data files collected for a plurality of first components operating in different assemblies, wherein the plurality of data files is selected from the group consisting of a plurality of sensor images, a plurality of mapped regions of damage, a plurality of calculated sizes of the region of damage, and a plurality of the types of damage.

18. The system of claim 17 and further comprising:

a third viewing pane comprising aggregate information from the plurality of data files to compare the region of damage between the plurality of first components.

Assignments (4)
CHANGE OF NAME Recorded Jul 27, 2023
From: RAYTHEON TECHNOLOGIES CORPORATION
To: RTX CORPORATION
Reel/Frame 064714/0001 →
CORRECTIVE ASSIGNMENT TO CORRECT THE AND REMOVE PATENT APPLICATION NUMBER 11886281 AND ADD PATENT APPLICATION NUMBER 14846874. TO CORRECT THE RECEIVING PARTY ADDRESS PREVIOUSLY RECORDED AT REEL: 054062 FRAME: 0001. ASSIGNOR(S) HEREBY CONFIRMS THE CHANGE OF ADDRESS. Recorded Mar 4, 2021
From: UNITED TECHNOLOGIES CORPORATION
To: RAYTHEON TECHNOLOGIES CORPORATION
Reel/Frame 055659/0001 →
CHANGE OF NAME Recorded Sep 4, 2020
From: UNITED TECHNOLOGIES CORPORATION
To: RAYTHEON TECHNOLOGIES CORPORATION
Reel/Frame 054062/0001 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jan 19, 2018
From: HESTAND, PAUL D.; FINN, ALAN MATTHEW; MOURA, DENNIS M.; XIONG, ZIYOU; RUSSELL, KEITH R.
To: UNITED TECHNOLOGIES CORPORATION
Reel/Frame 044672/0058 →
Continuity (1)
Related Publication 20190228514A1 · Jul 25, 2019
Cited By (1)
US 12,614,273