SYSTEM AND METHOD FOR LARGE-SCALE ACCELERATED PARALLEL PREDICTIVE MODELLING AND CONTROL

1 . A method for parallel predictive modelling, the method comprising:

receiving a configuration file associated with a predictive concept at a production layer of a predictive modelling platform, the predictive modelling platform comprising the production layer and a consumption layer, wherein the production layer and the consumption layer are communicatively connected by a distributed messaging system;

identifying, by the production layer, a job request based on the configuration file;

sending, by the distributed messaging system, the job request to the consumption layer, as one of a plurality of job requests to be passed to a predictive model implemented by a processing container, wherein the predictive model is specified by the configuration file;

obtaining, from the processing container, a forecast as an output of the predictive model;

sending, by the distributed messaging system, the forecast to the production layer; and

determining, by the production layer, one or more values of the predictive concept based on the forecast and an operator specified by the configuration file.

2 . The method of claim 1 , wherein the predictive model is a deep learning model.

3 . The method of claim 1 , further comprising:

identifying, by the production layer, a model training request based on the configuration file;

sending, by the distributed messaging system, to the consumption layer, a request for updated training data from a data mapper provided by the predictive modelling platform; and

training the predictive model based on the updated training data.

4 . The method of claim 3 , wherein the configuration file comprises a history of predictive models used to generate the one or more values of the predictive concept.

5 . The method of claim 3 , wherein the data mapper comprises a data store for the predictive modelling platform, and

wherein the data mapper assigns a plurality of connections between a plurality of heterogeneous data storage units.

6 . The method of claim 1 , wherein the configuration file specifies a feature set for the predictive model used to generate the one or more values of the predictive concept.

7 . The method of claim 1 , further comprising:

generating, by the production layer, a control command comprising a parameter based on the determined one or more values of the predictive concept; and

sending the control command, via a network to an external system.

8 . A platform, comprising:

a processor; and

a non-transitory memory containing instructions, which, when executed by the processor, cause the platform to:

receive a configuration file associated with a predictive concept at a production layer of the platform, wherein the platform comprises the production layer and a consumption layer, wherein the production layer and the consumption layer are communicatively connected by a distributed messaging system,

identify, by the production layer, a job request based on the configuration file,

send, by the distributed messaging system, the job request to the consumption layer, as one of a plurality of job requests to be passed to a predictive model implemented by a processing container, wherein the predictive model is specified by the configuration file,

obtain, from the processing container, a forecast as an output of the predictive model,

send, by the distributed messaging system, the forecast to the production layer, and

determine, by the production layer, one or more values of the predictive concept based on the forecast and an operator specified by the configuration file.

9 . The platform of claim 8 , wherein the predictive model is a deep learning model.

10 . The platform of claim 8 , wherein the memory further contains instructions, which, when executed by the processor, cause the platform to:

identify, by the production layer, a model training request based on the configuration file,

send, by the distributed messaging system, to the consumption layer, a request for updated training data from a data mapper provided by the predictive modelling platform, and

train the predictive model based on the updated training data.

11 . The platform of claim 10 , wherein the configuration file comprises a history of predictive models used to generate the one or more values of the predictive concept.

12 . The platform of claim 10 , wherein the data mapper comprises a data store for the predictive modelling platform, and

wherein the data mapper assigns a plurality of connections between a plurality of heterogeneous data storage units.

13 . The platform of claim 8 , wherein the configuration file specifies a feature set for the predictive model used to generate the one or more values of the predictive concept.

14 . The platform of claim 8 , wherein the memory further comprises instructions, which when executed by the processor, cause the platform to:

generate, by the production layer, a control command comprising a parameter based on the determined one or more values of the predictive concept, and

send the control command, via a network to an external system.

15 . A non-transitory, computer-readable medium containing instructions, which when executed by a processor, cause a predictive modelling platform to:

receive a configuration file associated with a predictive concept at a production layer of the platform, wherein the platform comprises the production layer and a consumption layer, wherein the production layer and the consumption layer are communicatively connected by a distributed messaging system,

identify, by the production layer, a job request based on the configuration file,

send, by the distributed messaging system, the job request to the consumption layer, as one of a plurality of job requests to be passed to a predictive model implemented by a processing container, wherein the predictive model is specified by the configuration file,

obtain, from the processing container, a forecast as an output of the predictive model,

send, by the distributed messaging system, the forecast to the production layer, and

determine, by the production layer, one or more values of the predictive concept based on the forecast and an operator specified by the configuration file.

16 . The non-transitory, computer readable medium of claim 15 , wherein the predictive model is a deep learning model.

17 . The non-transitory, computer readable medium of claim 15 , further comprising instructions, which when executed by the processor cause the predictive modelling platform to:

identify, by the production layer, a model training request based on the configuration file,

send, by the distributed messaging system, to the consumption layer, a request for updated training data from a data mapper provided by the predictive modelling platform, and

train the predictive model based on the updated training data.

18 . The non-transitory, computer readable medium of claim 17 , wherein the configuration file comprises a history of predictive models used to generate the one or more values of the predictive concept.

19 . The non-transitory, computer readable medium of claim 17 , wherein the data mapper comprises a data store for the predictive modelling platform, and

wherein the data mapper assigns a plurality of connections between a plurality of heterogeneous data storage units.

20 . The non-transitory, computer readable medium of claim 17 , wherein the configuration file specifies a feature set for the predictive model used to generate the one or more values of the predictive concept.