Multi-dimensional time series datasets
A method may comprise receiving a query for performing one or more computational operations on one or more multi-dimensional data sets representing multi-dimensional time series data collected in real-time from one or more sensors associated with one or more technical systems. The method may also comprise identifying the location of the one or more multi-dimensional time series data sets in one or more databases, retrieving the one or more multi-dimensional time series data sets from the identified one or more databases, and performing the one or more computational operations on the retrieved one or more multi-dimensional time series data sets. The method may also comprise generating output based on the result of the one or more computational operations indicative of one or more states of the one or more technical systems with respect to time.
1 . A method, comprising:
receiving real-time streaming data originating from a plurality of sensors associated with one or more technical systems, the real-time streaming data representing one or more multi-dimensional time series data sets;
prior to parsing and cleaning the real-time streaming data, storing the real-time streaming data in a cold storage as raw data received from the plurality of sensors, the raw data comprising unparsed and uncleaned data;
cleaning the real-time streaming data;
structuring the real-time streaming data according to a format associated with an ontology associated with the plurality of sensors;
storing the structured real-time streaming data in one or more time-series databases;
in response to identifying missing data or erroneous data stored in the one or more time-series databases, updating the structured real-time streaming data in the one or more time-series databases with data from the cold storage;
receiving a query for performing one or more computational operations on the structured real-time streaming data representing the one or more multi-dimensional time series data sets collected in real-time from the plurality of sensors associated with the one or more technical systems;
identifying a location of the one or more multi-dimensional time series data sets in one or more databases based on accessing metadata associated with the one or more multi-dimensional time series data sets in the one or more databases, said one or more databases being pre-registered with a middleware analysis platform, the metadata including identifiers of the one or more multi-dimensional time series data sets and their respective storage locations in the one or more databases;
retrieving the one or more multi-dimensional time series data sets from the one or more databases;
performing the one or more computational operations on the one or more multi-dimensional time series data sets retrieved from the one or more databases to generate a resultant time series data set;
displaying, via an interactive graphical user interface, a multi-dimensional visualization of the resultant time series data set to permit a user to analyze one or more states of the one or more technical systems;
monitoring the resultant time series data set to detect a predetermined condition of the resultant time series data set, wherein the predetermined condition is based on a relationship between the multi-dimensional time series data sets;
in response to detecting the predetermined condition of the resultant time series data set, displaying, via the interactive graphical user interface:
information relating to the predetermined condition of the resultant time series data set and the one or more technical systems, and
indications of one or more system operations to be performed on the one or more technical systems; and
in response to receiving one or more user selections via the interactive graphical user interface of the indications of the one or more system operations, performing one or more system operations on the one or more technical systems according to the one or more user selections.
2 . The method of claim 1 further comprising:
accessing diagnostic data comprising information relating to a historical predetermined condition and an indication of how the historical predetermined condition was resolved, wherein the historical predetermined condition comprises a historical data outlier corresponding a data outlier of the resultant time series data set;
determining the information relating to the predetermined condition from the information relating to the historical predetermined condition; and
determining the one or more system operations from the indication of how the historical predetermined condition was resolved.
3 . The method of claim 1 , further comprising performing an automatic operation on the one or more technical systems responsive to the predetermined condition being detected.
4 . The method of claim 1 , wherein the multi-dimensional visualization of the resultant time series data set comprises one or more multi-dimensional graphs.
5 . The method of claim 1 , wherein the multi-dimensional visualization of the resultant time series data set includes one or more scatter plots.
6 . The method of claim 1 , wherein the multi-dimensional visualization of the resultant time series data set includes a sequence of time slices.
7 . Non-transitory computer readable media including computer-executable instructions which, when executed by a computing system, cause the computing system to perform operations comprising:
receiving real-time streaming data originating from a plurality of sensors associated with one or more technical systems, the real-time streaming data representing one or more multi-dimensional time series data sets;
prior to parsing and cleaning the real-time streaming data, storing the real-time streaming data in a cold storage as raw data received from the plurality of sensors, the raw data comprising unparsed and uncleaned data;
cleaning the real-time streaming data;
structuring the real-time streaming data according to a format associated with an ontology associated with the plurality of sensors;
storing the structured real-time streaming data in one or more time-series databases;
in response to identifying missing data or erroneous data stored in the one or more time-series databases, updating the structured real-time streaming data in the one or more time-series databases with data from the cold storage;
receiving a query for performing one or more computational operations on the structured real-time streaming data representing the one or more multi-dimensional time series data sets collected in real-time from the plurality of sensors associated with the one or more technical systems;
identifying a location of the one or more multi-dimensional time series data sets in one or more databases based on accessing metadata associated with the one or more multi-dimensional time series data sets in the one or more databases, said one or more databases being pre-registered with a middleware analysis platform, the metadata including identifiers of the one or more multi-dimensional time series data sets and their respective storage locations in the one or more databases;
retrieving the one or more multi-dimensional time series data sets from the one or more databases;
performing the one or more computational operations on the one or more multi-dimensional time series data sets retrieved from the one or more databases to generate a resultant time series data set;
displaying, via an interactive graphical user interface, a multi-dimensional visualization of the resultant time series data set to permit a user to analyze one or more states of the one or more technical systems;
monitoring the resultant time series data set to detect a predetermined condition of the resultant time series data set, wherein the predetermined condition is based on a relationship between the multi-dimensional time series data sets;
in response to detecting the predetermined condition of the resultant time series data set, displaying, via the interactive graphical user interface:
information relating to the predetermined condition of the resultant time series data set and the one or more technical systems, and
indications of one or more system operations to be performed on the one or more technical systems; and
in response to receiving one or more user selections via the interactive graphical user interface of the indications of the one or more system operations, performing one or more system operations on the one or more technical systems according to the one or more user selections.
8 . The non-transitory computer readable media of claim 7 , wherein the computer-executable instructions, when executed by the computing system, cause the computing system to perform operations comprising:
accessing diagnostic data comprising information relating to a historical predetermined condition, and an indication of how the historical predetermined condition was resolved, wherein the historical predetermined condition comprises a historical data outlier corresponding to a data outlier of the resultant time series data set;
determining the information relating to the predetermined condition from the information relating to the historical predetermined condition; and
determining the one or more system operations from the indication of how the historical predetermined condition was resolved.
9 . The non-transitory computer readable media of claim 7 , wherein the computer-executable instructions, when executed by the computing system, cause the computing system to perform operations comprising performing an automatic operation on the one or more technical systems responsive to the predetermined condition being detected.
10 . The non-transitory computer readable media of claim 7 , wherein the multi-dimensional visualization of the resultant time series data set comprises one or more multi-dimensional graphs.
11 . The non-transitory computer readable media of claim 7 , wherein the multi-dimensional visualization of the resultant time series data set includes one or more scatter plots.
12 . The non-transitory computer readable media of claim 7 , wherein the multi-dimensional visualization of the resultant time series data set includes a sequence of time slices.
13 . A computing system comprising one or more hardware processors configured to:
receive real-time streaming data originating from a plurality of sensors associated with one or more technical systems, the real-time streaming data representing one or more multi-dimensional time series data sets;
prior to parsing and cleaning the real-time streaming data, store the real-time streaming data in a cold storage as raw data received from the plurality of sensors, the raw data comprising unparsed and uncleaned data;
clean the real-time streaming data;
structure the real-time streaming data according to a format associated with an ontology associated with the plurality of sensors;
store the structured real-time streaming data in one or more time-series databases;
in response to identifying missing data or erroneous data stored in the one or more time-series databases, update the structured real-time streaming data in the one or more time-series databases with data from the cold storage;
receive a query for performing one or more computational operations on the structured real-time streaming data representing the one or more multi-dimensional time series data sets collected in real-time from the plurality of sensors associated with the one or more technical systems;
identify a location of the one or more multi-dimensional time series data sets in one or more databases based on accessing metadata associated with the one or more multi-dimensional time series data sets in the one or more databases, said one or more databases being pre-registered with a middleware analysis platform, the metadata including identifiers of the one or more multi-dimensional time series data sets and their respective storage locations in the one or more databases;
retrieve the one or more multi-dimensional time series data sets from the one or more databases;
perform the one or more computational operations on the one or more multi-dimensional time series data sets retrieved from the one or more databases to generate a resultant time series data set;
display, via an interactive graphical user interface, a multi-dimensional visualization of the resultant time series data set to permit a user to analyze one or more states of the one or more technical systems;
monitor the resultant time series data set to detect a predetermined condition of the resultant time series data set, wherein the predetermined condition is based on a relationship between the multi-dimensional time series data sets;
in response to detecting the predetermined condition of the resultant time series data set, display, via the interactive graphical user interface:
information relating to the predetermined condition of the resultant time series data set and the one or more technical systems, and
indications of one or more system operations to be performed on the one or more technical systems; and
in response to receiving one or more user selections via the interactive graphical user interface of the indications of the one or more system operations, perform one or more system operations on the one or more technical systems according to the one or more user selections.
14 . The computing system of claim 13 , wherein the one or more hardware processors are configured to:
access diagnostic data comprising information relating to a historical predetermined condition and an indication of how the historical predetermined condition was resolved, wherein the historical predetermined condition comprises a historical data outlier corresponding to a data outlier of the resultant time series data set;
determine the information relating to the predetermined condition from the information relating to the historical predetermined condition; and
determine the one or more system operations from the indication of how the historical predetermined condition was resolved.
15 . The computing system of claim 13 , wherein the one or more hardware processors are configured to perform an automatic operation on the one or more technical systems responsive to the predetermined condition being detected.
16 . The computing system of claim 13 , wherein the multi-dimensional visualization of the resultant time series data set comprises one or more multi-dimensional graphs.
17 . The computing system of claim 13 , wherein the multi-dimensional visualization of the resultant time series data set includes one or more scatter plots.
18 . The computing system of claim 13 , wherein the multi-dimensional visualization of the resultant time series data set includes a sequence of time slices.