IP Library › Granted Patent US 12,651,196
Granted Patent B2
US 12,651,196 · App. 17/122,401 · Granted Jun 9, 2026

Automatically change anomaly detection threshold based on probabilistic distribution of anomaly scores

Inventors: Amin Suzani (Vancouver, CA); Matteo Casserini (Zurich, CH); Milos Vasic (Zurich, CH); Saeid Allahdadian (Vancouver, CA); Andrew Brownsword (Vancouver, CA); Hamed Ahmadi (Burnaby, CA); Felix Schmidt (Baden-Daettwil, CH); Nipun Agarwal (Saratoga, CA)
Assignee: Oracle International Corporation
G06N20/00G06F17/18G06N7/01
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,651,196
App. No.
17/122,401
Filed
Dec 15, 2020
Granted
Jun 9, 2026
Kind
B2
Art Unit
2129
USPC
706/12
Abstract

Approaches herein relate to model decay of an anomaly detector due to concept drift. Herein are machine learning techniques for dynamically self-tuning an anomaly score threshold. In an embodiment in a production environment, a computer receives an item in a stream of items. A machine learning (ML) model hosted by the computer infers by calculation an anomaly score for the item. Whether the item is anomalous or not is decided based on the anomaly score and an adaptive anomaly threshold that dynamically fluctuates. A moving standard deviation of anomaly scores is adjusted based on a moving average of anomaly scores. The moving average of anomaly scores is then adjusted based on the anomaly score. The adaptive anomaly threshold is then adjusted based on the moving average of anomaly scores and the moving standard deviation of anomaly scores.

Claims (38)

1 . A method comprising:

receiving a feature aggregation in a stream of feature aggregations;

calculating, by a machine learning (ML) model, an anomaly score for said feature aggregation;

deciding said feature aggregation is anomalous when said anomaly score exceeds an adaptive anomaly threshold that is less than one; and

sequentially:

adjusting a moving standard deviation of anomaly scores based on a moving average of anomaly scores,

adjusting said moving average of anomaly scores based on said anomaly score, and

decreasing said adaptive anomaly threshold based on said moving average of anomaly scores and said moving standard deviation of anomaly scores;

wherein the method is performed by one or more computers.

2 . The method of claim 1 wherein said decreasing said adaptive anomaly threshold based on said moving average of anomaly scores comprises adjusting said adaptive anomaly threshold based on an exponential moving average of anomaly scores.

3 . The method of claim 1 wherein said decreasing said adaptive anomaly threshold based on said moving standard deviation of anomaly scores comprises adjusting said adaptive anomaly threshold based on an exponential moving standard deviation of anomaly scores.

4 . The method of claim 1 further comprising initializing the moving standard deviation of anomaly scores based on a standard deviation of a plurality of early anomaly scores.

5 . The method of claim 4 further comprising calculating, by the ML model, said plurality of early anomaly scores respectively for a plurality of early feature aggregations.

6 . The method of claim 1 wherein said calculating said anomaly score for said feature aggregation comprises calculating a normalized anomaly score based on a standard deviation of a plurality of early anomaly scores.

7 . The method of claim 1 wherein said calculating said anomaly score for said feature aggregation comprises calculating a normalized anomaly score based on a probability density.

8 . The method of claim 1 further comprising initializing the moving average of anomaly scores based on an average of a plurality of early anomaly scores.

9 . The method of claim 1 further comprising:

receiving a second feature aggregation in said stream of feature aggregations;

calculating, by said ML model, same said anomaly score for said second feature aggregation;

deciding said second feature aggregation is not anomalous based on said adaptive anomaly threshold.

10 . The method of claim 1 wherein said calculating said anomaly score for said feature aggregation comprises said ML model calculating said anomaly score for internet of things (IoT) telemetry.

11 . The method of claim 1 wherein said deciding said feature aggregation is anomalous comprises detecting a network intrusion based on said adaptive anomaly threshold.

12 . The method of claim 1 wherein said deciding said feature aggregation is anomalous comprises detecting a console log message is anomalous based on said adaptive anomaly threshold.

13 . One or more non-transitory computer-readable storage media storing instructions that, when executed by one or more processors, cause:

receiving a feature aggregation in a stream of feature aggregations;

calculating, by a machine learning (ML) model, an anomaly score for said feature aggregation;

deciding said feature aggregation is anomalous when said anomaly score exceeds an adaptive anomaly threshold that is less than one; and

sequentially:

adjusting a moving standard deviation of anomaly scores based on a moving average of anomaly scores,

adjusting said moving average of anomaly scores based on said anomaly score, and

decreasing said adaptive anomaly threshold based on said moving average of anomaly scores and said moving standard deviation of anomaly scores.

14 . The one or more non-transitory computer-readable storage media of claim 13 wherein said decreasing said adaptive anomaly threshold based on said moving average of anomaly scores comprises adjusting said adaptive anomaly threshold based on an exponential moving average of anomaly scores.

15 . The one or more non-transitory computer-readable storage media of claim 13 wherein said decreasing said adaptive anomaly threshold based on said moving standard deviation of anomaly scores comprises adjusting said adaptive anomaly threshold based on an exponential moving standard deviation of anomaly scores.

16 . The one or more non-transitory computer-readable storage media of claim 13 wherein the instructions further cause initializing the moving standard deviation of anomaly scores based on a standard deviation of a plurality of early anomaly scores.

17 . The one or more non-transitory computer-readable storage media of claim 13 wherein said calculating said anomaly score for said feature aggregation comprises calculating a normalized anomaly score based on a standard deviation of a plurality of early anomaly scores.

18 . The one or more non-transitory computer-readable storage media of claim 13 wherein said calculating said anomaly score for said feature aggregation comprises calculating a normalized anomaly score based on a probability density.

19 . The one or more non-transitory computer-readable storage media of claim 13 wherein said deciding said feature aggregation is anomalous comprises detecting a network intrusion based on said adaptive anomaly threshold.

20 . The one or more non-transitory computer-readable storage media of claim 13 wherein said deciding said feature aggregation is anomalous comprises detecting a console log message is anomalous based on said adaptive anomaly threshold.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 5, 2021
From: SUZANI, AMIN; CASSERINI, MATTEO; VASIC, MILOS; ALLAHDADIAN, SAEID; BROWNSWORD, ANDREW; AHMADI, HAMED; SCHMIDT, FELIX; AGARWAL, NIPUN
To: ORACLE INTERNATIONAL CORPORATION
Reel/Frame 055507/0840 →
Continuity (1)
Related Publication 20220188694A1 · Jun 16, 2022
References Cited (108)
US 10069900B2 · Poola et al. · 2018 [cited by applicant]
US 20090091443A1 · Chen · 2009 [cited by examiner]
US 20160092516A1 · Poola et al. · 2016 [cited by applicant]
US 20170169360A1 · Veeramachaneni et al. · 2017 [cited by applicant]
US 20180095004A1 · Ide et al. · 2018 [cited by applicant]
US 20180109589A1 · Ozaki et al. · 2018 [cited by applicant]
US 20190095300A1 · Oba · 2019 [cited by examiner]
US 20190370610A1 · Batoukov · 2019 [cited by examiner]
US 20200045064A1 · Bindal · 2020 [cited by applicant]
US 20200076841A1 · Hajimirsadeghi · 2020 [cited by applicant]
US 20200201727A1 · Nie · 2020 [cited by applicant]
US 20220138504A1 · Moghadam et al. · 2022 [cited by applicant]
US 20220156578A1 · Allahdadian et al. · 2022 [cited by applicant]
US 20220188410A1 · Allahdadian et al. · 2022 [cited by applicant]
KR 20200164008 · 2020 [cited by applicant]
WO WO2020185101A9 · 2021 [cited by applicant]
WO WO2021176460A1 · 2021 [cited by examiner]
Li et al. “Dlog: diagnosing router events with syslogs for anomaly detection.” The Journal of Supercomputing 74 (2018) (Year: 2018). [cited by examiner]
Ross et al. “Exponentially weighted moving average charts for detecting concept drift.” Pattern recognition letters 33.2 (2012) (Year: 2012). [cited by examiner]
Li et al. “Dlog: diagnosing router events with syslogs for anomaly detection.” The Journal of Supercomputing 7 4 (Year: 2018). [cited by examiner]
Hinton, Geoffrey, “A Practical Guide to Training Restricted Boltzmann Machines”, Version 1, dated Aug. 2, 2010, 21 pages. [cited by applicant]
Ackerman et al., “Measures of Clustering Quality: AWorking Set of Axioms for Clustering”, dated 2009, 8 pages. [cited by applicant]
Ghosh et al., “Detecting Anomalous and Unknown Intrusions Against Programs”, dated 1998, 9 pages. [cited by applicant]
Gody, Daniel, “Understanding binary cross-entropy / log loss: a visual explanation”, dated Nov. 21, 2018, 13 pages. [cited by applicant]
Goix, Nicolas, “How to Evaluate the Quality of Unsupervised Anomaly Detection Algorithms?”, Presented at ICML2016 Anomaly Detection Workshop, New York, NY, USA, 2016, 13 pages. [cited by applicant]
Golan et al., “Deep Anomaly Detection Using Geometric Transformations”, 32nd Conference on Neural Information Processing Systems (NeurIPS dated 2018), Montréal, Canada, 12 pages. [cited by applicant]
HaddadPajouh et al, “A Two-layer Dimension Reduction and Two-tier Classification Model for Anomaly-Based Intrusion Detection in IoT Backbone Networks”, dated 2016, 12 pages. [cited by applicant]
Du et al. DeepLog: Anomaly Detection and Diagnosis from System Logs through Deep Learning, CCS'17, Oct. 30-Nov. 3, 2017, 14 pages. [cited by applicant]
Hill et al., “Learning Distributed Representations of Sentences from Unlabelled Data”, dated Feb. 10, 2016, 11 pages. [cited by applicant]
Dong et al., “Quality-Based Dynamic Threshold for Iris Matching”, IEEE, dated 2009, 4 pages. [cited by applicant]
Hochreiter, Sepp, “Long Short Term Memory”, Nural Computation, dated Nov. 15, 1997, 46 pages. [cited by applicant]
Hu et al., “Anomalous User Activity Detection in Enterprise Multi-Source Logs”, dated Nov. 2017, 8 pages. [cited by applicant]
Jain et al., “Score normalization in multimodal biometric systems”, Pattern Recognition 38, dated Jan. 2005, 16 pages. [cited by applicant]
Kim et al., “Behavior-based anomaly detection on big data”, Edith Cowan University, Research Online, dated 2015, 9 pages. [cited by applicant]
Kiros et al., “Skip-Thought Vectors”, dated 2015, 9 pages. [cited by applicant]
Koenker et al. “Quantile Regression”, Journal of Economic Perspectives, vol. 15, No. 4, dated 2001, 129 pages. [cited by applicant]
Kolosnjaji et al., “Deep Learning for Classification of Malware System Call Sequences”, dated 2016, 12 pages. [cited by applicant]
Halkidi et al., “On Clustering Validation Techniques”, Journal of Intelligent Information Systems, dated 2001, 39 pages. [cited by applicant]
Buczak et al., “A Survey of Data Mining and Machine Learning Methods for Cyber Security Intrusion Detection”, IEEE Communications Surveys & Tutorials, vol. 18, No. 2, Second Quarter 2016, 24 pages. [cited by applicant]
Ackerman, Margareta, “A Theoretical Study of Clusterability and Clustering Quality”, Waterloo, Ontario, Canada, dated 2007, 76 pages. [cited by applicant]
Alexey Tsymbal, “The Problem of Concept Drift: Definitions and Related Work”, dated Apr. 29, 2004, 7 pages. [cited by applicant]
Anonymous authors, “Versatile Outlier Detection With Outlier Preserving Distribution Mapping Autoencoders”, conference paper at ICLR 2020, dated 2019, 13 pages. [cited by applicant]
Bach et al., A Bayesian Approach to Concept Drift, dated 2010, 9 pages. [cited by applicant]
Bengio et al., “A Neural Probabilistic Language Model”, Journal of Machine Learning Research 3, dated Feb. 2003, 19 pages. [cited by applicant]
Berlin et al., “Malicious Behavior Detection using Windows Audit Logs”, dated Aug. 25, 2015, 10 pages. [cited by applicant]
Gama et al., “A Survey on Concept Drift Adaptation”, ACM Computing Surveys, vol. 1, No. 1, Article 1, Publication date: Jan. 2013, 44 pages. [cited by applicant]
Browniee_Jason, “A Gentle Introduction to Cross-Entropy for Machine Learning”, dated Oct. 21, 2019, https://machinelearningmastery.com/cross-entropy-for-machine-learning/, 34 pages. [cited by applicant]
Liu et al., “Detecting and Preventing Cyber Insider Threats: A Survey”, IEEE Communications Survey & Tutorials, dated 2018, 21 pages. [cited by applicant]
Chakraborty et al., “Early detection of faults in HVAC systems using an XGBoost model with a dynamic threshold”, Energy and Buildings, dated 2019, pp. 326-344. [cited by applicant]
Chandola et al., “Anomaly Detection : A Survey”, ACM Computing Surveys, dated Sep. 2009, 75 pages. [cited by applicant]
Chang et al., “A Dynamic Threshold Decision System for Stock Trading Signal Detection”, Applied Soft Computing 11, dated 2011, 13 pages. [cited by applicant]
Clemencon et al., “Scoring anomalies: a M-estimation formulation”, Proceedings of the 16th International Conference on Artifical Intelligence and Statistics (AISTATS) 2013, vol. 31 of JMLR, 9 pgs. [cited by applicant]
Conneau et al. “Supervised Learning of Universal Sentence Representations from Natural Language Inference Data”, Dated Jul. 21, 2017, 12 pages. [cited by applicant]
Dai et al., “Semi-supervised Sequence Learning”, dated 2015, 9 pages. [cited by applicant]
Dal Pozzolo et al., “Learned lessons in credit card fraud detection from a practitioner perspective”, dated 2014 Elsevier Ltd., 14 pages. [cited by applicant]
Bontemps et al., “Collective Anomaly Detection based on Long Short Term Memory Recurrent Neural Network”, dated Mar. 2017, 13 pages. [cited by applicant]
Sutskever et al., “Sequence to Sequence Learning with Neural Networks”, dated 2014, 9 pages. [cited by applicant]
Ruder, Sebastian, “An Overview of Gradient Descent Optimization Algorithms”, dated Jun. 15, 2017, 14 pages. [cited by applicant]
Sabokrou et al., “Real-Time Anomaly Detection and Localization in Crowded Scenes”, dated 2015, 7 pages. [cited by applicant]
Sakurada et al., “Anomaly Detection Using Autoencoders with Nonlinear Dimensionality Reduction”, MLSDA '14, Dec. 2, 2014, Gold Coast, QLD, Australia Copyright 2014 ACM, 8 pages. [cited by applicant]
Schubert et al., “On Evaluation of Outlier Rankings and Outlier Scores”, dated 2012, 12 pages. [cited by applicant]
Seleznyov et al., “Anomaly Intrusion Detection Systems: Handling Temporal Relations between Events”, dated 1999, 12 pages. [cited by applicant]
Shen et al., “Scalable Large-Margin Mahalanobis Distance Metric Learning”, IEEE, vol. 30, No. 9, dated 2010, 7 pages. [cited by applicant]
Kolter et al., “Dynamic Weighted Majority: An Ensemble Method for Drifting Concepts”, Journal of Machine Learning Research 8, dated 2007, 36 pages. [cited by applicant]
Singh et al., “Quantitative Evaluation of Normalization Techniques of Matching Scores in Multimodal Biometric Systems”, Springer-Verlag Berlin Heidelberg, dated 2007, 10 pages. [cited by applicant]
Platt, John, “Probabilistic Outputs for Support Vector Machines and Comparisons to Regularized Likelihood Methods”, dated Mar. 6, 1999, 11 pages. [cited by applicant]
Tuor et al., “Deep Learning for Unsupervised Insider Threat Detection in Structured Cybersecurity Data Streams”, dated Dec. 15, 2017, 9 pages. [cited by applicant]
Usama et al., “Unsupervised Machine Learning for Networking: Techniques, Applications and Research Challenges”, dated Sep. 19, 2017, 37 pages. [cited by applicant]
Wang et al.,Towards a Hierarchical Bayesian Model of Multi-View Anomaly Detection, Twenty-Ninth International Joint Conference on Artificial Intelligence, dated Jul. 11, 2020, 7 pages. [cited by applicant]
Webb et al., Characterizing Concept Drift, Data Mining and Knowledge Discovery, dated Jul. 2016, 30 pages. [cited by applicant]
Xiang et al., “Learning a Mahalanobis distance metric for data clustering and classification”, Pattern Recognition 41, dated 2008, 13 pages. [cited by applicant]
Xu et al., “Detecting Large-Scale System Problems by Mining Console Logs”, dated 2009, 16 pages. [cited by applicant]
Yousefi-Azar et al., “Autoencoder-based Feature Learning for Cyber Security Applications”, dated 2017, 8 pages. [cited by applicant]
Shipmon et al., “Time Series Anomaly Detection”, Detection of Anomalous Drops with Limited Features and Sparse Examples in Noisy Highly Periodic Data, dated 2017, 9 pages. [cited by applicant]
Mirza Ali H et al., “Computer Network Intrusion Detection Using Sequwnrial LSTM Neural Networks Autoencoders”, dated May 2, 2018, 2018 26th Signal Processing and Communicaitons Applications Con. 4 pgs. [cited by applicant]
YuanZhong, Zhu, “Intrusion Detection Method based on Improved BP Neural Network Research”, International Journal of Security and Its Applications vol. 10, No. 5 (2016) pp. 193-202. [cited by applicant]
Loganathan et al., “Sequence to Sequence Pattern Learning Algorithm for Real-time Anomaly Detection in Network Traffic”, conference paper dated May 2018. [cited by applicant]
Luo et al., “A Revisit of Sparse Coding Based Anomaly Detection in Stacked RNN Framework”, dated Oct. 2017, 9 pages. [cited by applicant]
Malhotra et al., “Long Short Term Memory Networks for Anomaly Detection in Time Series”, ESANN dated Apr. 22, 2015 proceedings, European Symposium on Artificial Neural Networks, 6 pages. [cited by applicant]
Malhotra et al., “LSTM-based Encoder-Decoder for Multi-sensor Anomaly Detection”, Presented at ICML 2016 Anomaly Detection Workshop, New York, NY, USA, 2016. Copyright 2016—5 pages. [cited by applicant]
Mathuranathan, Q Function and Error functions : Demystified, dated Jul. 16, 2012, https://www.gaussianwaves.com/2012/07/q-function-and-error-functions/, 7 pages. [cited by applicant]
Mei et al., “Learning a Mahalanobis Distance based Dynamic Time Warping Measure for Multivariate Time Series Classification”, IEEE, dated 2015, 12 pages. [cited by applicant]
Rayana et al., “Sequential Ensemble Learning for Outlier Detection: A Bias-Variance Perspective”, dated Sep. 18, 2016, 11 pages. [cited by applicant]
Mikolov et al., “Efficient Estimation of Word Representations in Vector Space”, dated Sep. 7, 2013, 12 pages. [cited by applicant]
Polonik, Wolfgang, “Measuring Mass Concentrations and Estimating Density Contour Clusters—An Excess Mass Approach”, The Annals of Statstics, dated 1995, vol. 23, No. 3, 27 pages. [cited by applicant]
Mnih et al., “A Scalable Hierarchical Distributed Language Model”, dated 2009, 8 pages. [cited by applicant]
Moustafa et al., “A holistic review of Network Anomaly Detection Systems: A comprehensive survey”, Journal of Network and Computer Applications vol. 128, Feb. 15, 2019, pp. 33-55. [cited by applicant]
N.Krishnavardhan, “A Framework to Identify Cybercrime Using Data Analytics”, International Journal of Pure and Applied Mathematics, vol. 120, No. 6 dated Jun. 11, 2018, 14 pages. [cited by applicant]
Naseer et al., “Enhanced Network Anomaly Detection Based on Deep Neural Networks”, Journal of Latex Class Files, vol. 14, No. 8, Aug. 2015, 16 pages. [cited by applicant]
Nguyen et al., “An Evaluation Method for Unsupervised Anomaly Detection Algorithms”, Journal of Computer Science and Cybernetics, V.32, N.3, dated 2016, 14 pages. [cited by applicant]
Palacio-Nino et al., “Evaluation Metrics for Unsupervised Learning Algorithms”, dated May 23, 2019, 9 pages. [cited by applicant]
Le et al., “Distributed Representations of Sentences and Documents”, Proceedings of the 31 st International Conference on Machine Learning, Beijing, China, dated 2014, 9 pages. [cited by applicant]
Mikolov et al., “Distributed Representations of Words and Phrases and their Compositionality”, dated 2013, 9 pages. [cited by applicant]
Zhou, Junlin, et al., “Unsupervised Learning Based Distributed Detection of Global Anomalies”, 2010, International Journal of Information Technology and Decision Making, vol. 2010, Nov. 2010, pp. 1-11. [cited by applicant]
Vikram, Adiya, et al., “Anomaly detection in Network Traffic Using Unsupervised Machine learning Approach”, 2020 5th Intl Conf on Communication and Electronics Systems (ICCES), vol. 5 (2020), pp. 476-479, Jul. 10, 2020,… [cited by applicant]
Angiulli, Fabrizio, “Concentration Free Outlier Detection”, 2017, Machine Learning and Knowledge Discovery in Databases, vol. 2017, pp. 3-19. [cited by applicant]
Neuberg, Richard, et al., “Detective Relative Anomaly”, 2015 18th Intl Conf, on Mach Learning and Data Mining in Pattern Recognition, Lecture Notes in Computer Science, LNAI 10358. Springer, doi.org/10.1007/978-3-319-62… [cited by applicant]
Gao, Jing, et al., “Converting Output Scores from Outlier Detection Algorithms into Probability Estimates”, 6th Intl Conf on Data Mining (ICDM'06), pp. 212-221, doi: 10.1109/ICDM.2006.43, Dec. 18, 2006, 10pgs. [cited by applicant]
Siffer et al., “Anomaly Detection in Streams with Extreme Value Theory”, Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, https://hal.archives-ouvertes.fr/hal-01640325,… [cited by applicant]
Eskin, Eleazar, “Anomaly Detection Over Noisy Data Using Learned Probability Distributions”, https://academiccommons.columbia.edu/doi/10.7916/D8C53SKF, dated 2000, 8 pages. [cited by applicant]
Dykes, Sandra, “Poster: An Extreme Value Theory Approach to Anomaly Detection (EVT-AD)”, https://www.ieee-security.org/TC/SP2012/posters/An%20Extreme%20Value%20Theory%20Approach.pdf, dated 2012, 2 pages. [cited by applicant]
Davis et al., “LSTM-Based Anomaly Detection: Detection Rules from Extreme Value Theory”, EPIA Conference on Artificial Intelligenc https://arxiv.org/pdf/1909.06041.pdf, dated Sep. 13, 2019, 12 pages. [cited by applicant]
Angiulli et al., “Distance-Based Detection and Prediction of Outliers”, IEEE Transactions on Knowledge and Data Engineering, vol. 18, No. 2, dated Feb. 2006, 16 pages. [cited by applicant]
Ross, Gordon J., et al. “Exponentially weighted moving average charts for detecting concept drift”, Pattern Recognition Letters 33.2 (Year: 2012). [cited by applicant]
Siffer et al., “Anomaly Detection in Streams with Extreme Value Theory” KDD Research Paper (2017). [cited by applicant]
Ross et al., “Exponentially weighted moving average charts for detecting concept drift.” Pattern Recognition Letters 33.2 (Year: 2012). [cited by applicant]
Qian et al., “A Rank-SVM Approach to Anomaly Detection”, A New One-Class SVM for Anomaly Detection (Year: 2014). [cited by applicant]
Jaworski et al., “Concept drift detection using autoencoers in data streams processing”. In International Conference on Artificial Intelligence and Soft Computing (Year 2020). [cited by applicant]