Development platform for image processing pipelines that use machine learning with user interface
A development platform for implementing a machine learning pipeline on a chip containing multiple hardware compute elements. The development platform includes a user interface, a library of software blocks, and a synthesis engine. The user interface facilitates a user to develop a functional description of the machine learning pipeline. The functional description specifies multiple functional modules, including a machine learning model. The synthesis engine synthesizes the pipeline of functional modules into multiple interconnected executable components of software blocks and generates an implementation package including the executable components and specifying interconnections between the executable components.
1. A development platform for implementing a machine learning pipeline on a chip containing a plurality of hardware compute elements, the development platform comprising:
a user interface that facilitates a user to develop a functional description of the machine learning pipeline, wherein the functional description specifies a plurality of functional modules that form the pipeline, and at least one functional module includes a machine learning model;
a library of software blocks that implement the functional modules on the hardware compute elements of the chip, wherein the library of software blocks includes software blocks for implementing a plurality of machine learning models, a plurality of sensor plugins, and a plurality of ethernet device plugins; and
a synthesis engine configured to:
synthesize the pipeline of functional modules into a plurality of interconnected executable components of the software blocks, wherein the machine learning pipeline is synthesized into executable components that execute on at least two different hardware compute elements of the chip; and
generate an implementation package comprising the executable components and specifying interconnections between the executable components.
2. The development platform of claim 1 further comprising:
a key performance indicator (KPI) calculator that computes a KPI based on the interconnected executable components the KPI displayed in the user interface.
3. The development platform of claim 2 wherein the KPI includes at least one of frames per second (FPS), power consumption, memory utilization, and processor utilization.
4. The development platform of claim 2 wherein the KPI includes KPIs for each of the hardware compute elements.
5. The development platform of claim 2 wherein, responsive to the user's modification of the pipeline of functional modules, the KPI calculator updates the KPI.
6. The development platform of claim 1 wherein the user interface includes a graphical user interface (GUI) that displays a catalog of functional modules and allows the user to select functional modules from the catalog.
7. The development platform of claim 6 wherein the GUI further includes a canvas area; and, responsive to the user dragging functional modules from the catalog to the canvas area, the GUI displays the dragged functional modules in the canvas area.
8. The development platform of claim 6 wherein one of the selected functional modules includes a plurality of submodules that execute on corresponding hardware compute elements, and the GUI displays the plurality of submodules and the corresponding hardware compute elements.
9. The development platform of claim 6 wherein the GUI displays one or more parameters of a hardware compute element, and allows the user to modify the one or more parameters of the hardware compute element.
10. The development platform of claim 1 wherein the synthesis engine is further configured to
generate source code files for software blocks implementing the pipeline of functional modules, and
compile the source code files into the plurality of interconnected executable components.
11. The development platform of claim 1 wherein the synthesis engine is further configured to:
determine whether a connection between two functional modules is correct, and
responsive to determining an incorrect connection, generate an error alert.
12. The development platform of claim 1 further comprising a model training engine configured to:
train a custom machine learning model based on user input, and
include software blocks for implementing the custom machine learning model in the library.
13. The development platform of claim 1 wherein the library of software blocks includes proprietary software blocks, open source software blocks, and user-generated software blocks.
14. The development platform of claim 1 further comprising a library of different chips comprising different processors, and a user is allowed to select a target chip from the library of chips, and the synthesis engine generates the executable components based on the processors on the target chip.
15. The development platform of claim 1 wherein the library of software blocks comprises software blocks for implementing a plurality of machine learning models.
16. The development platform of claim 1 further comprising:
a library of reference pipelines.
17. The development platform of claim 1 wherein the development platform supports a concurrent design of multiple different chips and multiple different versions of chips.
18. The development platform of claim 1 wherein the development platform is a cloud-based service.
19. The development platform of claim 1 wherein the development platform is connected to the chip, and configured to receive a KPI of the chip when the executable components are executed by the hardware compute elements of the chip.
20. A development platform for implementing a machine learning pipeline on a chip containing a plurality of hardware compute elements, the development platform comprising:
a user interface that facilitates a user to develop a functional description of the machine learning pipeline, wherein the functional description specifies a plurality of functional modules that form the pipeline, and at least one functional module includes a machine learning model;
a library of software blocks that implement the functional modules on the hardware compute elements of the chip;
a library of different chips comprising different processors; and
a synthesis engine configured to:
synthesize the pipeline of functional modules into a plurality of interconnected executable components of the software blocks, wherein the machine learning pipeline is synthesized into executable components that execute on at least two different hardware compute elements of the chip; and
generate an implementation package comprising the executable components and specifying interconnections between the executable components, wherein a user is allowed to select a target chip from the library of chips, and the synthesis engine generates the executable components based on the processors on the target chip.
21. A development platform for implementing a machine learning pipeline on a chip containing a plurality of hardware compute elements, the development platform comprising:
a user interface that facilitates a user to develop a functional description of the machine learning pipeline, wherein the functional description specifies a plurality of functional modules that form the pipeline, and at least one functional module includes a machine learning model;
a library of software blocks that implement the functional modules on the hardware compute elements of the chip, wherein the development platform supports a concurrent design of multiple different chips and multiple different versions of chips; and
a synthesis engine configured to:
synthesize the pipeline of functional modules into a plurality of interconnected executable components of the software blocks, wherein the machine learning pipeline is synthesized into executable components that execute on at least two different hardware compute elements of the chip; and
generate an implementation package comprising the executable components and specifying interconnections between the executable components.