Anomaly score adjustment across anomaly generators
Techniques are disclosed for generating an anomaly score for a neuro-linguistic model of input data obtained from one or more sources. According to one embodiment, generating an anomaly score comprises receiving a score indicating how often a characteristic is observed in the input data. Upon receiving the score, comparing the score with an unusual score model to determine an unusualness score and comparing the unusualness score with an anomaly score model based on one or more unusual score models to generate the anomaly score indicating an overall unusualness for the input data.
1. A computer-implemented method for anomaly detection, the method comprising:
executing a neuro-linguistic model to generate a raw unusualness score based on input data obtained from at least one of a video data source, a supervisory control and data acquisition (SCADA) system, or a data network security system, the raw unusualness score indicating how often a characteristic is observed in the input data;
comparing the raw unusualness score with an unusualness score model to generate an unusualness score, the unusualness score model produced from historical raw unusualness scores;
comparing the unusualness score with an anomaly score model to generate an anomaly score that represents, as a percentile, an unusualness of the input data, the anomaly score model produced from historical unusualness scores; and
sending alert data based on the anomaly score.
2. The method of claim 1 , wherein the unusualness score model uses a probabilistic histogram based on previously received scores.
3. The method of claim 1 , wherein the anomaly score model uses a probabilistic histogram based on unusualness scores from the unusualness score model.
4. The method of claim 3 , wherein comparing the unusualness score includes:
decoding the anomaly score model to reconstruct a set of previously determined unusualness scores; and
comparing the unusualness score to the set of previously determined unusualness scores.
5. The method of claim 1 , further comprising encoding the unusualness score in the unusualness score model.
6. The method of claim 1 , further comprising encoding the unusualness score in the anomaly score model.
7. A non-transitory processor-readable medium storing code representing instructions to be executed by a processor, the code comprising code to cause the processor to:
execute a neuro-linguistic model to generate a raw unusualness score based on input data obtained from at least one of a video data source, a supervisory control and data acquisition (SCADA) system, or a data network security system, the raw unusualness score indicating how often a characteristic is observed in the input data;
compare the raw unusualness score with an unusualness score model to generate an unusualness score, the unusualness score model produced from historical raw unusualness scores;
compare the unusualness score with an anomaly score model to generate an anomaly score that represents, as a percentile, an unusualness of the input data, the anomaly score model produced from historical unusualness scores; and
send alert data based on the anomaly score.
8. The non-transitory processor-readable medium of claim 7 , wherein the unusualness score model uses a probabilistic histogram based on previously received scores.
9. The non-transitory processor-readable medium of claim 7 , wherein the anomaly score model uses a probabilistic histogram based on unusualness scores from the unusualness score model.
10. The non-transitory processor-readable medium of claim 9 , wherein the code to cause the processor to compare the unusualness score includes code to cause the processor to:
decode the anomaly score model to reconstruct a set of previously determined unusualness scores; and
compare the unusualness score to the set of previously determined unusualness scores.
11. The non-transitory processor-readable medium of claim 7 , wherein the code further comprises code to cause the processor to encode the usualness score in the unusualness score model.
12. The non-transitory processor-readable medium of claim 7 , wherein the code further comprises code to cause the processor to encode the unusualness score in the anomaly score model.
13. The non-transitory processor-readable medium of claim 7 , wherein the unusualness score represents a percentile ranking indicating the unusualness of the raw unusualness score as compared to previously received scores.
14. A system, comprising:
a memory; and
a processor operatively coupled to the memory, the processor configured to:
execute a neuro-linguistic model to generate a raw unusualness score based on input data obtained from at least one of a video data source, a supervisory control and data acquisition (SCADA) system, or a data network security system, the raw unusualness score indicating how often a characteristic is observed in the input data;
compare the raw unusualness score with an unusualness score model to generate an unusualness score, the unusualness score model produced from historical raw unusualness scores;
compare the unusualness score with an anomaly score model to generate an anomaly score that represents, as a percentile, an unusualness of the input data, the anomaly score model produced from historical unusualness scores; and
send alert data based on the anomaly score.
15. The system of claim 14 , wherein the unusualness score model comprises a probabilistic histogram based on previously received scores.
16. The system of claim 14 , wherein the anomaly score model comprises a probabilistic histogram based on unusualness scores from the unusualness score model.
17. The system of claim 16 , wherein the processor is configured to compare the unusualness score by:
decoding the anomaly score model to reconstruct a set of previously determined unusualness scores; and
comparing the unusualness score to the set of previously determined unusualness scores.
18. The system of claim 14 , wherein the processor is configured to encode the unusualness score in the unusualness score model.
19. The system of claim 14 , wherein the processor is configured to encode the usualness score in the anomaly score model.