Techniques for monitoring a data stream
View Patent ↗Systems and methods for detecting an interesting event in a data stream. More specifically, a monitoring system is configured to monitor a data stream and establish a sensitivity parameter based on sequences generated from values in a first portion of the data stream. A detector may be trained using the sensitivity parameter to detect the occurrence of an interesting event in the data stream.
1. A processor-based method comprising:
receiving a data stream comprising a plurality of temporally ordered data points;
generating a plurality of sequences from a first portion of the data stream; and
training a detector by determining a value for a sensitivity parameter using the plurality of sequences.
2. The method, as set forth in claim 1 , comprising running the detector on a second portion of the data stream.
3. The method, as set forth in claim 2 , wherein running the detector comprises:
generating a score corresponding to the second portion of the data stream;
comparing the score to the determined value for the sensitivity parameter; and
signaling detection of an interesting event in the data stream if the score crosses the sensitivity parameter.
4. The method, as set forth in claim 1 , wherein training the detector by determining the value for the sensitivity parameter comprises selecting the value for the sensitivity parameter based on a target level for an estimated performance characteristic of the detector.
5. The method, as set forth in claim 1 , wherein training the detector by determining the value for the sensitivity parameter comprises:
generating a score for each of the plurality of sequences; and
selecting the value for the sensitivity parameter based on the scores.
6. The method, as set forth in claim 1 , wherein generating the plurality of sequences comprises:
inferring a statistical distribution of a known type to characterize the first portion of the data stream; and
generating the plurality of sequences from the statistical distribution.
7. The method, as set forth in claim 6 , wherein the statistical distribution is a discrete distribution containing data points from the first portion of the data stream, and wherein generating the plurality of sequences from the statistical distribution comprises selecting data points from the discrete distribution.
8. The method, as set forth in claim 6 , wherein inferring a known type of distribution comprises determining a set of parameters corresponding to the known type of statistical distribution.
9. The method, as set forth in claim 1 , wherein generating the plurality of sequences comprises:
selecting a change based on a distribution of changes; and
generating a changed sequence based on the selected change.
10. The method, as set forth in claim 1 , wherein determining the value of the sensitivity parameter comprises determining a plurality of values for the sensitivity parameter using the plurality of sequences.
11. The method, as set forth in claim 10 , wherein determining one of the plurality of values for the sensitivity parameter comprises calculating a transformation of a second of the plurality of values for the sensitivity parameter.
12. The method, as set forth in claim 1 ,
wherein receiving a data stream comprises receiving a plurality of data streams;
wherein generating the sequences comprises generating a respective plurality of sequences from a respective first portion of each of the plurality of data streams; and
wherein determining the sensitivity parameter comprises determining a respective sensitivity parameter for each of the plurality of sequences.
13. The method, as set forth in claim 1 , wherein determining the value for the sensitivity parameter comprises determining the value for the sensitivity parameter based at least partially on cost parameters.
14. The method, as set forth in claim 12 , comprising raising an alarm when a respective detector signals detection when parameterized by the respective sensitivity parameter and run on a respective second portion of a sufficient set of data streams.
15. A processor-based method comprising:
training a detector using a plurality of sequences generated from a first portion of a data stream, wherein the detector is configured to detect an interesting event in the data stream; and
testing a second portion of the data stream using the trained detector.
16. The method, as set forth in claim 15 , comprising transforming the data stream before training the detector.
17. The method, as set forth in claim 15 , comprising:
generating a plurality of sequences from a third portion of the data stream; and
retraining the detector using the plurality of sequences generated from the third portion of the data stream.
18. The method, as set forth in claim 15 , wherein training the detector comprises determining one or more sensitivity parameters from the plurality of sequences.
19. The method, as set forth in claim 18 , wherein testing the second portion of the data stream comprises:
generating a score associated with the second portion of the data stream; and
comparing the score with the one or more sensitivity parameters.
20. The method, as set forth in claim 15 , comprising raising an alarm only if an interesting event is detected in the data stream a predetermined number of times within a predetermined amount of time.
21. The method, as set forth in claim 15 , comprising raising an alarm if the detector detects an interesting event in the data stream.
22. A system comprising:
a trainer configured to generate a plurality of sequences from a first portion of a data stream and further configured to determine one or more sensitivity parameters based on the sequences; and
a detector configured to detect an interesting event in the data stream using the one or more sensitivity parameters.
23. The system, as set forth in claim 22 , comprising an alarm coupled to the detector and configured to engage when an interesting event in the data stream is detected.
24. The system, as set forth in claim 22 , comprising an input device coupled to the trainer, wherein the device is configured to allow a user to set cost parameters for use in determining the one or more sensitivity parameters.
25. A computer-readable medium comprising computer instructions for:
generating a plurality of sequences from a first portion of a data stream;
determining a sensitivity parameter using the plurality of sequences; and
training a detector to detect an interesting event in the data stream using the sensitivity parameter.
26. The computer-readable medium, as set forth in claim 25 , further comprising computer instructions for:
generating a score corresponding to a second portion of the data stream; and
signaling detection of an interesting event in the data stream if the score crosses the sensitivity parameter.
27. A system comprising:
means for generating a plurality of sequences from a first portion of a data stream;
means for determining a sensitivity parameter based on the plurality of sequences; and
means for detecting an interesting event in a second portion of the data stream using the sensitivity parameter.
28. The system, as set forth in claim 27 , wherein means for determining comprises means for generating a plurality of interesting sequences from the data stream, wherein the interesting sequences have a different statistical distribution than a statistical distribution of the first portion of the data stream.
29. The system, as set forth in claim 27 , wherein means for detecting comprises means for detecting an interesting event in a parameter of the plurality of distributions.
30. The system, as set forth in claim 27 , comprising means for injecting a change into the first portion of the data stream.