SYSTEMS AND METHODS FOR PROCESSING DATA RELATING TO ENERGY USAGE
Processing of data relating to energy usage. First data relating to energy usage is loaded for analysis by an energy management platform. Second data relating to energy usage is stream processed by the energy management platform. Third data relating to energy usage is batch parallel processed by the energy management platform. Additional computing resources, owned by a third party separate from an entity that owns the computer system that supports the energy management platform, are provisioned based on increasing computing demand. Existing computing resources owned by the third party are released based on decreasing computing demand.
1 . A computer implemented method comprising:
loading, by a computer system, first data relating to energy usage for analysis by an energy management platform;
stream processing, by the computer system, second data relating to energy usage; and
batch parallel processing, by the computer system, third data relating to energy usage.
2 . The method of claim 1 , wherein the loading includes converting the first data relating to energy usage from a canonical form according to a data model of the energy management platform.
3 . The method of claim 1 , wherein the loading includes performing validation on the first data relating to energy usage.
4 . The method of claim 3 , wherein the performing validation on the first data relating to energy usage includes at least one of:
examining structure of the first data to ensure that required fields are present;
detecting gaps in the first data;
detecting anomalies in the first data;
identifying outliers in the first data; and
conducting referential integrity checks on the first data.
5 . The method of claim 1 , wherein the loading includes storing a first type of the first data relating to energy usage in a relational database.
6 . The method of claim 5 , wherein the first type of the first data relating to energy usage includes customer data.
7 . The method of claim 1 , wherein the loading includes storing a second type of the first data relating to energy usage in a key/value store.
8 . The method of claim 7 , wherein the second type of the first data relating to energy usage includes meter data.
9 . The method of claim 7 , wherein the loading includes normalizing the second type of the first data relating to energy usage.
10 . The method of claim 9 , wherein the loading includes storing the normalized second type of the first data relating to energy usage.
11 . The method of claim 1 , wherein the stream processing includes converting the second data relating to energy usage according to a data model of the energy management platform.
12 . The method of claim 11 , wherein the stream processing includes persisting the converted second data relating to energy usage.
13 . The method of claim 12 , wherein the stream processing includes scheduling processing of the second data relating to energy usage based on receipt of an event notification.
14 . The method of claim 13 , wherein the stream processing includes, using stream processing logic specified by at least one of the energy management platform and a user of the energy management platform, processing the second data relating to energy usage.
15 . The method of claim 14 , wherein the stream processing includes at least one of:
providing an alert based on the processed second data relating to energy usage;
persisting the processed second data relating to energy usage; and
providing a new stream based on the processed second data relating to energy usage.
16 . The method of claim 1 , wherein the batch parallel processing includes:
receiving a request for a batch job;
identifying a data set for the batch job associated with the third data relating to energy usage; and
dividing the batch job into batches.
17 . The method of claim 16 , wherein the batch parallel processing includes scheduling processing of the batches.
18 . The method of claim 1 , further comprising:
provisioning additional computing resources owned by a third party separate from an entity that owns the computer system based on increasing computing demand; and
releasing existing computing resources owned by the third party based on decreasing computing demand.
19 . A computer-storage medium storing computer-executable instructions that, when executed, cause a computer system to perform a computer-implemented method comprising:
loading first data relating to energy usage for analysis by an energy management platform;
stream processing second data relating to energy usage; and
batch parallel processing third data relating to energy usage.
20 . A system comprising:
at least one processor; and
a memory storing instructions configured to instruct the at least one processor to perform:
loading first data relating to energy usage for analysis by an energy management platform;
stream processing second data relating to energy usage; and
batch parallel processing third data relating to energy usage.