Analysis of smart meter data based on frequency content
Analysis of smart meter and/or similar data based on frequency content is disclosed. In various embodiments, for each of a plurality of resource consumption nodes a time series data including for each of a series of observation times a corresponding resource consumption data associated with that observation time is received. At least a portion of the time series data, for each of at least a subset of the plurality of resource consumption nodes, is transformed into a frequency domain. A feature set based at least in part on the resource consumption data as transformed into the frequency domain is used to detect that resource consumption data associated with a particular resource consumption node is anomalous.
1. A method, comprising:
receiving for each of a plurality of resource consumption nodes a time series data including for each of a series of observation times a corresponding resource consumption data associated with that observation time;
transforming at least a portion of the time series data into a frequency domain for each of at least a subset of the plurality of resource consumption nodes;
determining whether to initiate an automated responsive action in response to identifying anomalous resource consumption data associated with a particular resource consumption node, wherein identifying the anomalous consumption data includes using at least in part the resource consumption data associated with the portion of the time series data as transformed into the frequency domain to detect that resource consumption data associated with the particular resource consumption node is anomalous;
determining a type of anomaly associated with the anomalous resource consumption data corresponding to the particular resource consumption node, wherein the type of anomaly is determined based at least in part on the resource consumption data associated with the particular resource consumption node, wherein the anomalous resource consumption data includes a cluster of time series data distinguished from a set of non-anomalous time-series data; and
selecting at least one responsive action from a set of a plurality of responsive actions to be performed based at least in part on one or more anomalies and the type of anomaly, the selecting the at least one responsive action comprising selecting a type of investigation to be initiated based at least in part on the type of anomaly.
2. The method of claim 1 , wherein the type of anomaly is determined based at least in part on a cluster of other time series data that is different from a set of time series data that is determined to be non-anomalous.
3. The method of claim 1 , wherein the type of anomaly associated with the resource consumption data associated with the particular resource consumption node is determined based at least in part on resource consumption data associated with one or more other resource consumption nodes.
4. The method of claim 1 , further comprising:
using a feature set based at least in part on the resource consumption data associated with the portion of the time series data as transformed into the frequency domain to detect the anomalous resource consumption data associated with the particular resource consumption node.
5. The method of claim 4 , further comprising:
computing a degree of confidence associated with a detection that the resource consumption data associated with the particular resource consumption node is anomalous,
wherein a determination of whether to initiate an automated responsive action is based at least in part on the degree of confidence.
6. The method of claim 4 , wherein the feature set includes for each of at least a set of frequencies a corresponding magnitude.
7. The method of claim 4 , wherein the feature set includes one or more features not derived directly from the resource consumption data associated with the portion of the time series data as transformed into the frequency domain.
8. The method of claim 7 , wherein the feature set includes at least one of one or more attributes of a location with which the particular resource consumption node is associated, one or more attributes of a resource consumer with which the particular resource consumption node is associated, or one or more environmental attributes associated with the particular resource consumption node.
9. The method of claim 4 , wherein: the using the feature set based at least in part on the resource consumption data associated with the portion of the time series data as transformed into the frequency domain to detect that the resource consumption data associated with the particular resource consumption node is anomalous includes performing a cluster analysis, including identifying one or more clusters of the resource consumption nodes, and
the identification of the anomalous resource consumption data associated with the particular resource consumption node includes determining that the particular resource consumption node falls outside a prescribed threshold distance from a cluster centroid of a corresponding one of said one or more clusters of the resource consumption nodes.
10. The method of claim 1 , wherein the plurality of resource consumption nodes from which the time series data is received comprise utility users.
11. The method of claim 1 , wherein the plurality of resource consumption nodes from which the time series data is received include electric utility users, and for each resource consumption node of the plurality of resource consumption nodes, its corresponding time series data is received from a corresponding associated smart meter.
12. The method of claim 1 , further comprising:
validating the received time series data associated with the plurality of resource consumption nodes and processing the received time series data of each of the plurality of resource consumption nodes for missing data.
13. The method of claim 1 , wherein a threshold used to determine whether the resource consumption data associated with the particular resource consumption node is anomalous is user defined.
14. The method of claim 1 , further comprising:
receiving contextual data from one or more contextual sources, wherein the detection that the resource consumption data associated with the particular resource consumption node is anomalous is based at least in part on the contextual data.
15. The method of claim 14 , wherein the contextual data includes one or more of weather information, an environmental variable, economic activity, character of land use associated with a corresponding resource consumption node, and income of a user or account associated with the corresponding resource consumption node.
16. The method of claim 1 , wherein the type of anomaly is determined based at least in part on one or more characteristics or dimensions of the resource consumption data associated with the portion of the time series data as transformed into a frequency domain and a cluster analysis relating to consumption data from the plurality of the resource consumption nodes.
17. The method of claim 1 , wherein the determining the type of anomaly comprises determining whether the type of anomaly is a vegetation-related type or a fraudulent usage type.
18. A system, comprising:
a communication interface; and
one or more processors coupled to the communication interface and configured to:
receive via the communication interface for each of a plurality of resource consumption nodes a time series data including for each of a series of observation times a corresponding resource consumption data associated with that observation time;
transform at least a portion of the time series data into a frequency domain for each of at least a subset of the plurality of resource consumption nodes;
determine whether to initiate an automated responsive action in response to identifying anomalous resource consumption data associated with a particular resource consumption node, wherein identifying the anomalous consumption data includes using at least in part the resource consumption data associated with the portion of the time series data as transformed into the frequency domain to detect that resource consumption data associated with the particular resource consumption node is anomalous;
determine a type of anomaly associated with the anomalous resource consumption data corresponding to the particular resource consumption node, wherein the type of anomaly is determined based at least in part on the resource consumption data associated with the particular resource consumption node, wherein the anomalous resource consumption data includes a cluster of time series data distinguished from a set of non-anomalous time-series data; and
selecting at least one responsive action from a set of a plurality of responsive actions to be performed based at least in part on one or more anomalies and the type of anomaly, the selecting the at least one responsive action comprising selecting a type of investigation to be initiated based at least in part on the type of anomaly.
19. The system of claim 18 , wherein the type of anomaly is determined based at least in part on a cluster of other time series data that is different from a set of time series data that is determined to be non-anomalous.
20. The system of claim 18 , wherein the type of anomaly associated with the resource consumption data associated with the particular resource consumption node is determined based at least in part on resource consumption data associated with one or more other resource consumption nodes.
21. The system of claim 18 , wherein the one or more processors are further configured to receive contextual data from one or more contextual sources, and the detection that the resource consumption data associated with the particular resource consumption node is anomalous is based at least in part on the contextual data.
22. The system of claim 21 , wherein the contextual data includes one or more of weather information, an environmental variable, economic activity, character of land use associated with a corresponding resource consumption node, and income of a user or account associated with the corresponding resource consumption node.
23. A computer program product embodied in a non-transitory computer-readable storage medium and comprising computer instructions for:
receiving for each of a plurality of resource consumption nodes a time series data including for each of a series of observation times a corresponding resource consumption data associated with that observation time;
transforming at least a portion of the time series data into a frequency domain for each of at least a subset of the plurality of resource consumption nodes;
determining whether to initiate an automated responsive action in response to identifying anomalous resource consumption data associated with a particular resource consumption node, wherein identifying the anomalous consumption data includes using at least in part the resource consumption data associated with the portion of the time series data as transformed into the frequency domain to detect that resource consumption data associated with the particular resource consumption node is anomalous;
determining a type of anomaly associated with the anomalous resource consumption data corresponding to the particular resource consumption node, wherein the type of anomaly is determined based at least in part on the resource consumption data associated with the particular resource consumption node, wherein the anomalous resource consumption data includes a cluster of time series data distinguished from a set of non-anomalous time-series data; and
selecting at least one responsive action from a set of a plurality of responsive actions to be performed based at least in part on one or more anomalies and the type of anomaly, the selecting the at least one responsive action comprising selecting a type of investigation to be initiated based at least in part on the type of anomaly.
24. The computer program product of claim 23 , wherein the type of anomaly is determined based at least in part on a cluster of other time series data that is different from a set of time series data that is determined to be non-anomalous.
25. The computer program product of claim 23 , wherein the type of anomaly associated with the resource consumption data associated with the particular resource consumption node is determined based at least in part on resource consumption data associated with one or more other resource consumption nodes.
26. The computer program product of claim 23 , further comprising instructions for receiving contextual data from one or more contextual sources, wherein the detection that the resource consumption data associated with the particular resource consumption node is anomalous is based at least in part on the contextual data.
27. The computer program product of claim 23 , wherein the contextual data includes one or more of weather information, an environmental variable, economic activity, character of land use associated with a corresponding resource consumption node, and income of a user or account associated with the corresponding resource consumption node.