IP Library Granted Patent US 11,403,551
Granted Patent B2
US 11,403,551 · App. 16/007,592 · Granted Aug 2, 2022

System and method for validating unsupervised machine learning models

Inventors: David Lavid Ben Lulu (Nesher, IL); Eitan Vesely (Kfar-Vradim, IL)
Assignee: Presenso, Ltd.
G06N20/00G05B13/027G07C3/00
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,403,551
App. No.
16/007,592
Granted
Aug 2, 2022
Kind
B2
Abstract

A system and method for validating unsupervised machine learning models. The method includes: analyzing, via unsupervised machine learning, a plurality of sensory inputs associated with a machine, wherein the unsupervised machine learning outputs at least one normal behavior pattern of the machine; generating, based on the at least one normal behavior pattern, at least one artificial anomaly, wherein each artificial anomaly deviates from the at least one normal behavior pattern; injecting the at least one artificial anomaly into the plurality of sensory inputs to create an artificial dataset; and analyzing the artificial dataset to determine whether a candidate model is a valid representation of operation of the machine, wherein analyzing the artificial dataset further comprises running the candidate model using the artificial dataset as an input.

Claims (52)

1. A method for validating unsupervised machine learning models, comprising:

preprocessing a plurality of sensory inputs associated with the a machine, wherein the preprocessing includes extracting at least one feature from raw sensory data;

analyzing, via unsupervised machine learning, the plurality of sensory inputs associated with the machine, wherein the unsupervised machine learning outputs at least one normal behavior pattern of the machine;

generating, based on the at least one normal behavior pattern, at least one artificial anomaly, wherein each artificial anomaly deviates from the at least one normal behavior pattern, wherein each generated artificial anomaly includes a randomized time for injection and an at least partially randomized magnitude;

injecting the at least one artificial anomaly into the plurality of sensory inputs to create an artificial dataset; and

analyzing the artificial dataset to determine whether a candidate model is a valid representation of operation of the machine, wherein analyzing the artificial dataset further comprises running the candidate model using the artificial dataset as an input.

2. The method of claim 1 , further comprising:

determining, based on the analysis of the artificial dataset, a score representing an accuracy of the candidate model in detecting anomalies, wherein the candidate model is valid when the determined score is above a predetermined threshold.

3. The method of claim 1 , further comprising:

selecting a new model, when it is determined that the candidate model is not valid.

4. The method of claim 1 , wherein the analysis of the artificial dataset includes unsupervised machine learning, wherein the unsupervised machine learning analysis of the artificial dataset outputs at least one detected anomaly, wherein determining whether the candidate model is valid further comprises:

determining whether the at least one detected anomaly includes the at least one artificial anomaly.

5. The method of claim 1 , further comprising:

generating, based on the at least one normal behavior pattern, at least one adaptive threshold, wherein each artificial anomaly does not meet at least one of the at least one adaptive threshold.

6. The method of claim 5 , wherein each adaptive threshold includes at least one threshold value, wherein the at least one threshold value varies over time.

7. The method of claim 1 , wherein the plurality of sensory inputs associated with the machine are captured by at least one sensor in proximity to the machine, wherein each proximate sensor is within a predetermined distance of the machine.

8. A non-transitory computer readable medium having stored thereon instructions for causing a processing circuitry to perform a process, the process comprising:

preprocessing a plurality of sensory inputs associated with the a machine, wherein the preprocessing includes extracting at least one feature from raw sensory data;

analyzing, via unsupervised machine learning, the plurality of sensory inputs associated with the machine, wherein the unsupervised machine learning outputs at least one normal behavior pattern of the machine;

generating, based on the at least one normal behavior pattern, at least one artificial anomaly, wherein each artificial anomaly deviates from the at least one normal behavior pattern, wherein each generated artificial anomaly includes a randomized time for injection and an at least partially randomized magnitude;

injecting the at least one artificial anomaly into the plurality of sensory inputs to create an artificial dataset; and

analyzing the artificial dataset to determine whether a candidate model is a valid representation of operation of the machine, wherein analyzing the artificial dataset further comprises running the candidate model using the artificial dataset as an input.

9. A system for unsupervised prediction of machine failures, comprising:

a processing circuitry; and

a memory, the memory containing instructions that, when executed by the processing circuitry, configure the system to:

preprocessing a plurality of sensory inputs associated with the a machine, wherein the preprocessing includes extracting at least one feature from raw sensory data;

analyze, via unsupervised machine learning, the plurality of sensory inputs associated with the machine, wherein the unsupervised machine learning outputs at least one normal behavior pattern of the machine;

generate, based on the at least one normal behavior pattern, at least one artificial anomaly, wherein each artificial anomaly deviates from the at least one normal behavior pattern, wherein each generated artificial anomaly includes a randomized time for injection and an at least partially randomized magnitude;

inject the at least one artificial anomaly into the plurality of sensory inputs to create an artificial dataset; and

analyze the artificial dataset to determine whether a candidate model is a valid representation of operation of the machine, wherein analyzing the artificial dataset further comprises running the candidate model using the artificial dataset as an input.

10. The system of claim 9 , wherein the system is further configured to:

determine, based on the analysis of the artificial dataset, a score representing an accuracy of the candidate model in detecting anomalies, wherein the candidate model is valid when the determined score is above a predetermined threshold.

11. The system of claim 9 , wherein the system is further configured to:

select a new model, when it is determined that the candidate model is not valid.

12. The system of claim 9 , wherein the analysis of the artificial dataset includes unsupervised machine learning, wherein the unsupervised machine learning analysis of the artificial dataset outputs at least one detected anomaly, wherein the system is further configured to:

determine whether the at least one detected anomaly includes the at least one artificial anomaly.

13. The system of claim 9 , wherein the system is further configured to:

generate, based on the at least one normal behavior pattern, at least one adaptive threshold, wherein each artificial anomaly does not meet at least one of the at least one adaptive threshold.

14. The system of claim 13 , wherein each adaptive threshold includes at least one threshold value, wherein the at least one threshold value varies over time.

15. The system of claim 9 , wherein the plurality of sensory inputs associated with the machine are captured by at least one sensor in proximity to the machine, wherein each proximate sensor is within a predetermined distance of the machine.

16. A method for validating unsupervised machine learning models, comprising:

analyzing, via unsupervised machine learning, a plurality of sensory inputs associated with a machine, wherein the unsupervised machine learning outputs at least one normal behavior pattern of the machine;

generating, based on the at least one normal behavior pattern, at least one artificial anomaly, wherein each artificial anomaly deviates from the at least one normal behavior pattern, wherein each generated artificial anomaly includes a randomized time for injection and an at least partially randomized magnitude;

injecting the at least one artificial anomaly into the plurality of sensory inputs to create an artificial dataset; and

analyzing the artificial dataset to determine whether a candidate model is a valid representation of operation of the machine, wherein analyzing the artificial dataset further comprises running the candidate model using the artificial dataset as an input.

17. A system for unsupervised prediction of machine failures, comprising:

a processing circuitry; and

a memory, the memory containing instructions that, when executed by the processing circuitry, configure the system to:

analyze, via unsupervised machine learning, a plurality of sensory inputs associated with a machine, wherein the unsupervised machine learning outputs at least one normal behavior pattern of the machine;

generate, based on the at least one normal behavior pattern, at least one artificial anomaly, wherein each artificial anomaly deviates from the at least one normal behavior pattern, wherein each generated artificial anomaly includes a randomized time for injection and an at least partially randomized magnitude;

inject the at least one artificial anomaly into the plurality of sensory inputs to create an artificial dataset; and

analyze the artificial dataset to determine whether a candidate model is a valid representation of operation of the machine, wherein analyzing the artificial dataset further comprises running the candidate model using the artificial dataset as an input.

Assignments (5)
CORRECTIVE ASSIGNMENT TO CORRECT THE CONVEYING PARTY EXECUTION DATE PREVIOUSLY RECORDED ON REEL 057468 FRAME 0522. ASSIGNOR(S) HEREBY CONFIRMS THE CHANGE OF NAME. Recorded Dec 28, 2023
From: PRESENSO, LTD.
To: SKF PRESENSO, LTD.
Reel/Frame 066254/0864 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Sep 10, 2021
From: SKF AI LTD.
To: AKTIEBOLAGET SKF
Reel/Frame 057450/0220 →
CHANGE OF NAME Recorded Sep 10, 2021
From: PRESENSO, LTD.
To: SKF PRESENSO, LTD.
Reel/Frame 057468/0522 →
CHANGE OF NAME Recorded Sep 10, 2021
From: SKF PRESENSO, LTD.
To: SKF AI LTD.
Reel/Frame 057469/0325 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 13, 2018
From: LAVID BEN LULU, DAVID; VESELY, EITAN
To: PRESENSO, LTD.
Reel/Frame 046077/0507 →
Continuity (3)
Continuation PCTUS2017012708 · Jan 9, 2017
Provisional Application 62276875 · Jan 10, 2016
Related Publication 20180293516A1 · Oct 11, 2018
Cited By (1)
US 12,223,337