Discovery, mapping, and scoring of machine learning models residing on an external application from within a data pipeline
A data pipeline configuration system allows industrial data pipelines to be configured using an intuitive visual interface. The pipeline configuration system allows graphical pipeline components representing data sources, data processing, analytic or machine learning models, and emitters to be selectively added to an industrial data pipeline application by selecting these components from a library. The pipeline configuration application is created by arranging and linking these selected pipeline components within a pipeline builder section of the configuration system's visual design interface. The design interface also allows analytic or machine learning models to be easily integrated into the pipeline application and mapped to incoming data items, such that the model is applied and scored against incoming data during pipeline operation. The configuration system also allows the user to configure destinations or data sinks for the pipeline data, including both the incoming industrial data and model scoring results.
1. A system, comprising:
a memory that stores executable components; and
a processor, operatively coupled to the memory, that executes the executable components, the executable components comprising:
a user interface component configured to render an interface display and to receive, via interaction with the interface display, pipeline configuration input that defines aspects of a data pipeline, the pipeline configuration input comprising at least
model selection input that selects an analytic model to be included in the data pipeline;
a pipeline configuration component configured to generate pipeline application data based on the pipeline configuration input; and
a model mapping component configured to, in response to selection of the analytic model by the model selection input, identify input fields defined by the analytic model,
wherein
the user interface component is configured to render a field mapping interface display that displays the input fields identified by the model mapping component and data input selection fields respectively corresponding to the input fields,
a data input selection field, of the data input selection fields, lists pipeline data items associated with a data source selected by the pipeline configuration input and available to be selectively mapped to one of the input fields of the analytic model,
the pipeline configuration input further comprises mapping input that selects a pipeline data item, of the pipeline data items, via interaction with one of the data input selection fields,
selection of the pipeline data item by the mapping input maps the pipeline data item to one of the input fields of the analytic model corresponding to the one of the data input selection fields,
the pipeline application data configured to execute on a hardware platform to implement the data pipeline, and
the data pipeline is configured to apply the analytic model to items of the input data, including the pipeline data item, mapped to the input fields of the analytic model.
2. The system of claim 1 , wherein the interface display comprises
a component selection section that renders a library of pipeline components as selectable icons, and
a pipeline builder section that renders icons representing pipeline components selected from the component selection section by the pipeline configuration input and the analytic model selected by the model selection input as icons, and
the pipeline configuration component is configured to generate the pipeline application data based on an arrangement of the icons rendered in the pipeline builder section.
3. The system of claim 1 , wherein the user interface component is configured to:
render, on the interface display, a connection name field that prompts for information specifying a connection path to an application on which one or more analytic models are stored, and
render, on the interface display, a list of the one or more analytic models, including the analytic model, based on the information specifying the connection path, and
wherein the model selection input selects the analytic model from the list.
4. The system of claim 1 , wherein
the pipeline configuration input further comprises at least data preparation selection input that selects a data preparation component, from a library of pipeline components, for inclusion in the pipeline application data,
the inclusion of the data preparation component in the pipeline application data configures the data pipeline to convert input data from the data source to transformed data having a format that is understandable by processing components of the data pipeline, and
the user interface component determines the pipeline data items to be listed in the data input selection field based on the transformed data.
5. The system of claim 4 , wherein
the input data is unstructured data generated by the data source,
the data preparation component configures the data pipeline to convert the unstructured data to, as the transformed data, a set of comma-separated values, and
the user interface component obtains the pipeline data items to be listed in the data input selection field from the set of comma-separated values.
6. The system of claim 4 , wherein the data preparation component is configured to detect a schema of the input data from the data source and to transform the input data to the transformed data based on the schema.
7. The system of claim 1 , further comprising a model scoring component configured to generate model scoring result data based on application of the analytic model to the items of the input data.
8. The system of claim 7 , wherein
the user interface component is configured to render a model configuration interface display that displays configurable model parameters defined by the analytic model,
the pipeline configuration input further comprises model configuration input that sets values of the configurable model parameters, and
the model scoring component is configured to generate the scoring result data in accordance with the values of the configurable model parameters.
9. The system of claim 7 , wherein
the pipeline configuration input further comprises at least emitter selection input that selects an emitter component, from a library of pipeline components, for inclusion in the pipeline application data, and
inclusion of the emitter component in the pipeline application data configures the data pipeline to publish the model scoring result data to a data sink entity represented by the emitter component.
10. The system of claim 1 , further comprising a pipeline deployment component configured to deploy the pipeline application data to the hardware platform.
11. The system of claim 1 , wherein the data source is one an industrial controller, a variable frequency drive, a data historian, a file system, an edge device, an industrial sensor, an industrial telemetry device, a data warehouse, a data lake, or a software development kit.
12. The system of claim 1 , wherein analytic model is one of a predictive model configured to generate predictions regarding operation of an industrial machine or process, an optimization model configured to determine a modification to the industrial machine or process that causes a performance metric to satisfy a conditions, or a statistical model configured to calculate statistics regarding operation of the industrial machine or process.
13. A method, comprising:
rendering, by a system comprising a processor, an interface display on a client device;
receiving, by the system via interaction with the interface display, pipeline configuration input that defines aspects of a data pipeline, wherein the receiving comprises receiving at least model selection input that selects an analytic or machine learning model to be included in the data pipeline;
in response to receiving the model section input, identifying, by the system, input fields of the analytic or machine learning model;
rendering, by the system, a field mapping interface that displays the input fields and data input selection fields respectively corresponding to the input fields, wherein the rendering of the field mapping interface comprises:
identifying pipeline data items that are available to the data pipeline based on an identity of the data source, and
rendering the pipeline data items available for selection via the one of the data input selection fields;
receiving, by the system via interaction with the field mapping interface, selection of a pipeline data item, of the pipeline data items, via interaction with one of the data input selection fields corresponding to one of the input fields of the analytic or machine learning model, wherein the receiving of the selection defines a mapping between the pipeline data item and the one of the input fields;
generating, by the system, a data pipeline application in accordance with the pipeline configuration input and the mapping,
wherein
the generating comprises, based on the model selection and the mapping, configuring the data pipeline application to apply the analytic or machine learning model to items of the input data, including the pipeline data item, mapped to the input fields of the analytic or machine learning model, and
the data pipeline application is configured to execute on a hardware device to implement the data pipeline.
14. The method of claim 13 , wherein
the rendering of the interface display comprises formatting the interface display to include
a component selection section that renders a library of pipeline objects as selectable icons representing the pipeline components, and
a pipeline builder section that renders icons representing pipeline components selected from the component selection section by the pipeline configuration input and the analytic or machine learning model selected by the model selection input, and
the generating of the data pipeline application comprises generating the data pipeline application based on an arrangement of the icons rendered in the pipeline builder section.
15. The method of claim 13 , wherein the rendering of the interface display comprises:
rendering, on the interface display, a connection name field that prompts for information specifying a connection path to an application on which one or more analytic or machine learning models are stored, and
rendering, on the interface display, a list of the one or more analytic or machine learning models based on the information specifying the connection path,
wherein the model selection input selects the analytic or machine learning model via interaction with the list.
16. The method of claim 13 , further comprising:
applying, by the system, the analytic or machine learning model to the items of the input data mapped to the input fields of the analytic or machine learning model; and
generating, by the system, model scoring result data based on the applying of the analytic or machine learning model.
17. The method of claim 16 , wherein
the rendering of the interface display comprises rendering configurable model parameters defined by the analytic or machine learning model,
the receiving of the pipeline configuration input comprises receiving model configuration input that sets values of the configurable model parameters, and
the applying of the analytic or machine learning model comprises applying the analytic or machine learning model in accordance with the values of the configurable model parameters.
18. The method of claim 16 , wherein
the receiving of the pipeline configuration input further comprises receiving emitter selection input that selects an emitter component, from a library of pipeline components, for inclusion in the pipeline configuration application, and
the generating comprises, based on the emitter selection input, configuring the data pipeline application to publish the model scoring result data to a data sink entity represented by the emitter component.
19. A non-transitory computer-readable medium having stored thereon instructions that, in response to execution, cause a system comprising a processor to perform operations, the operations comprising:
rendering an interface display on a client device;
receiving, via interaction with the interface display, pipeline configuration input that defines aspects of a data pipeline, wherein the receiving comprises receiving at least model selection input that selects an analytic or machine learning model to be included in the data pipeline;
in response to receiving the model section input, identifying input fields of the analytic or machine learning model;
rendering a field mapping interface that displays the input fields and data input selection fields respectively corresponding to the input fields, wherein the rendering of the field mapping interface comprises:
identifying pipeline data items that are available to the data pipeline based on an identity of the data source, and
rendering the pipeline data items available for selection via the one of the data input selection fields;
receiving, via interaction with the field mapping interface, selection of a pipeline data item, of the pipeline data items, via interaction with one of the data input selection fields corresponding to one of the input fields of the analytic or machine learning model, wherein the receiving of the selection defines a mapping between the pipeline data item and the one of the input fields;
generating a data pipeline application in accordance with the pipeline configuration input and the mapping,
wherein
the generating comprises, based on the model selection and the mapping, configuring the data pipeline application to apply the analytic or machine learning model to items of the input data, including the pipeline data item, mapped to the input fields of the analytic or machine learning model, and
the data pipeline application is configured to execute on a hardware platform to operate the data pipeline.
20. The non-transitory computer-readable medium of claim 19 , wherein the rendering of the interface display comprises:
rendering, on the interface display, a connection name field that prompts for information specifying a connection path to an application on which one or more analytic or machine learning models are stored, and
rendering, on the interface display, a list of the one or more analytic or machine learning models based on the information specifying the connection path,
wherein the model selection input selects the analytic or machine learning model via interaction with the list.