IP Library Granted Patent US 11,340,602
Granted Patent B2
US 11,340,602 · App. 15/535,909 · Granted May 24, 2022

Sensor data fusion for prognostics and health monitoring

Inventors: Michael J. Giering (Bolton, CT); Madhusudana Shashanka (Santa Clara, CA); Soumik Sarkar (Ames, IA); Vivek Venugopalan (Bridgewater, NJ)
Assignee: RAYTHEON TECHNOLOGIES CORPORATION
G05B23/0254G05B23/0294G06K9/6273G06K9/6289G06N3/0454G06N3/08G06T7/248G06V10/454G06V20/56
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,340,602
App. No.
15/535,909
Granted
May 24, 2022
Kind
B2
Abstract

A method includes converting time-series data from a plurality of prognostic and health monitoring (PHM) sensors into frequency domain data. One or more portions of the frequency domain data are labeled as indicative of one or more target modes to form labeled target data. A model including a deep neural network is applied to the labeled target data. A result of applying the model is classified as one or more discretized PHM training indicators associated with the one or more target modes. The one or more discretized PHM training indicators are output.

Claims (34)

1. A method comprising:

converting time-series data from a plurality of prognostic and health monitoring (PHM) sensors into frequency domain data, wherein the PHM sensors are heterogeneous sensors that monitor at least two uncorrelated parameters of a monitored system, and the PHM sensors include at least two kinds of sensors from a set of one or more temperature sensors, one or more strain gauges, one or more flow sensors, one or more accelerometers, and one or more debris monitors;

performing data set size normalization of the frequency domain data to support fusion of multiple sensor data sets from the heterogeneous sensors, wherein the frequency domain data comprise spectrogram data generated for each of the PHM sensors covering a same period of time and normalizing the data set sizes using a sliding window;

labeling one or more portions of the frequency domain data indicative of one or more target modes to form labeled target data;

training a model based on a plurality of weights developed in a pre-training process configured to reconstruct an input of the PHM sensors in response to a corrupted sensor input generated by adding noise as part of the pre-training process, wherein the model is trained using a supervised learning process to develop the weights in the pre-training process and tune the weights based on the labeled target data and the pre-training process comprises applying a pre-training network of a plurality of stacked Restricted Boltzmann Machines to develop the weights to remove noise from one or more inputs comprising noise;

applying the model comprising a deep neural network to the labeled target data, wherein the deep neural network comprises at least one layer of hidden units between an input layer and an output layer, and each of the hidden units uses a nonlinear mapping function to map an input to a scalar state;

classifying a result of applying the model as one or more discretized PHM training indicators associated with the one or more target modes, wherein the classifying converts an output of the output layer of the deep neural network into a class probability indexed over a plurality of classes; and

outputting the one or more discretized PHM training indicators.

2. The method of claim 1 , wherein the PHM sensors further comprise one or more pressure sensors.

3. The method of claim 1 , wherein the one or more targeted modes comprise one or more fault conditions.

4. The method of claim 1 , wherein the deep neural network is a deep belief network with a soft max layer performing classification using a nonlinear mapping.

5. The method of claim 1 , wherein applying the model to the labeled target data is performed during a training process to train the model, and further comprising:

applying the model in a testing process to unlabeled frequency domain data from one or more of the PHM sensors;

classifying a result of applying the model as one or more discretized PHM result indicators; and

outputting the one or more discretized PHM result indicators.

6. The method of claim 1 , further comprising:

creating different instances of the model for different target modes.

7. The method of claim 1 , wherein the one or more target modes comprise one or more health condition and one or more prognostic condition of the monitored system.

8. A system comprising:

a sensor system comprising a plurality of prognostic and health monitoring (PHM) sensors, wherein the PHM sensors are heterogeneous sensors that monitor at least two uncorrelated parameters of a monitored system, and the PHM sensors include at least two kinds of sensors from a set of one or more temperature sensors, one or more strain gauges, one or more flow sensors, one or more accelerometers, and one or more debris monitors; and

a PHM processor operable to:

convert time-series data from the PHM sensors into frequency domain data;

perform data set size normalization of the frequency domain data to support fusion of multiple sensor data sets from the heterogeneous sensors, wherein the frequency domain data comprise spectrogram data generated for each of the PHM sensors covering a same period of time and normalizing the data set sizes using a sliding window;

label one or more portions of the frequency domain data indicative of one or more target modes to form labeled target data;

train a model based on a plurality of weights developed in a pre-training process configured to reconstruct an input of the PHM sensors in response to a corrupted sensor input generated by adding noise as part of the pre-training process, wherein the model is trained using a supervised learning process to develop the weights in the pre-training process and tune the weights based on the labeled target data and the pre-training process comprises applying a pre-training network of a plurality of stacked Restricted Boltzmann Machines to develop the weights to remove noise from one or more inputs comprising noise;

apply the model comprising a deep neural network to the labeled target data, wherein the deep neural network comprises at least one layer of hidden units between an input layer and an output layer, and each of the hidden units uses a nonlinear mapping function to map an input to a scalar state;

classify a result of applying the model as one or more discretized PHM training indicators associated with the one or more target modes, wherein classification converts an output of the output layer of the deep neural network into a class probability indexed over a plurality of classes; and

output the one or more discretized PHM training indicators.

9. The system of claim 8 , wherein the PHM sensors further comprise one or more pressure sensors.

10. The system of claim 8 , wherein the one or more targeted modes comprise one or more fault conditions.

11. The system of claim 8 , wherein the deep neural network is a deep belief network with a soft max layer performing classification using a nonlinear mapping.

12. The system of claim 8 , wherein application of the model to the labeled target data is performed during a training process to train the model and the PHM processor is further operable to apply the model in a testing process to unlabeled frequency domain data from one or more of the PHM sensors, classify a result of applying the model as one or more discretized PHM result indicators, and output the one or more discretized PHM result indicators.

13. The system of claim 8 , wherein different instances of the model are created for different target modes.

14. The system of claim 8 , wherein the one or more target modes comprise one or more health condition and one or more prognostic condition of the monitored system.

Assignments (4)
CHANGE OF NAME Recorded Sep 30, 2024
From: RAYTHEON TECHNOLOGIES CORPORATION
To: RTX CORPORATION
Reel/Frame 069073/0814 →
CORRECTIVE ASSIGNMENT TO CORRECT THE AND REMOVE PATENT APPLICATION NUMBER 11886281 AND ADD PATENT APPLICATION NUMBER 14846874. TO CORRECT THE RECEIVING PARTY ADDRESS PREVIOUSLY RECORDED AT REEL: 054062 FRAME: 0001. ASSIGNOR(S) HEREBY CONFIRMS THE CHANGE OF ADDRESS. Recorded Mar 4, 2021
From: UNITED TECHNOLOGIES CORPORATION
To: RAYTHEON TECHNOLOGIES CORPORATION
Reel/Frame 055659/0001 →
CHANGE OF NAME Recorded Sep 4, 2020
From: UNITED TECHNOLOGIES CORPORATION
To: RAYTHEON TECHNOLOGIES CORPORATION
Reel/Frame 054062/0001 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 14, 2017
From: GIERING, MICHAEL J.; SHASHANKA, MADHUSUDANA; SARKAR, SOUMIK; VENUGOPALAN, VIVEK
To: UNITED TECHNOLOGIES CORPORATION
Reel/Frame 042707/0061 →