Hardware-aware machine learning model search mechanisms
The present disclosure is related to framework for automatically and efficiently finding machine learning (ML) architectures that generalize well across multiple artificial intelligence (AI) and/or ML domains, AI/ML tasks, and datasets. The ML architecture search framework accepts a list of tasks and corresponding datasets as inputs, and may also include relevancy scores/weights for each item in the input. A combined performance metric is generated, where this combined performance metric quantifies the performance of the ML architecture across all the specified AI/ML domains, AI/ML tasks, and datasets. The system then performs a multi-objective ML architecture search with the combined performance metric, along with hardware-specific performance metrics as the objectives. Other embodiments may be described and/or claimed.
1. An apparatus for providing a machine learning (ML) architecture search (MLAS) engine, the apparatus comprising:
interface circuitry to obtain a set of specified tasks from a client device;
machine-readable instructions; and
one or more processor circuits to be programmed by the machine-readable instructions to:
generate a task list from the set of specified tasks, and
group a set of datasets into a first sub-group of generalized datasets and a second sub-group of task-specific datasets, the task-specific datasets with a single task in the task list and the generalized datasets usable with more than one task in the task list;
generate one or more data batches from the set of datasets based on ones of the task-specific datasets and the generalized datasets;
determine respective performance metrics for tasks in the task list, and
generate a combined performance metric (CPM) based on the performance metrics for the tasks;
determine a set of generalized machine learning architectures (GMLAs) and corresponding platform-based performance metrics based on the CPM and the one or more data batches, wherein a GMLA of the set of GMLAs is a machine learning (ML) architecture that can be used for more than one ML task and with the task-specific datasets and the generalized datasets; and
provide the set of GMLAs and the corresponding performance metrics for deployment on a hardware platform.
2. The apparatus of claim 1 , wherein the CPM and the platform-based performance metrics are objectives of a multi-objective function and the set of GMLAs are a result of application of the multi-objective function.
3. The apparatus of claim 2 , wherein at least one of the one or more processor circuits is to determine task-specific weights for corresponding tasks in the task list, wherein the task-specific weights indicate a preference of the corresponding tasks with respect to other tasks in the task list, and the task-specific weights are coefficients that correspond to respective ones of the platform-based performance metrics in the multi-objective function.
4. The apparatus of claim 3 , wherein the interface circuitry is to obtain an ML configuration from the client device, the ML configuration including the set of specified tasks, the set of datasets, the task-specific weights, and the platform-based performance metrics.
5. The apparatus of claim 3 , wherein at least one of the one or more processor circuits is to:
generate individual datasets from the task-specific datasets and the generalized datasets such that the individual datasets correspond to a respective task in the task list.
6. The apparatus of claim 2 , wherein the multi-objective function includes one or more of a predetermined multi-objective evolutionary algorithm (MOEA), a predetermined Neural Architecture Search (NAS) algorithm, or a configured multi-objective function included in an ML configuration provided by the client device.
7. The apparatus of claim 2 , wherein the the multi-objective function is Strength Pareto Evolutionary Algorithm 2 (SPEA-2) or Non-dominated Sorting Genetic Algorithm-II (NSGA-II).
8. The apparatus of claim 2 , wherein the set of GMLAs includes a set of Pareto optimal solutions of the multi-objective function.
9. The apparatus of claim 1 , wherein at least one of the one or more processor circuits is to group the set of datasets into the first sub-group of generalized datasets or the second sub-group of task-specific datasets using a predetermined or configured ruleset, the ruleset including one or more rules for grouping the set of datasets to the first sub-group of generalized datasets and the second sub-group of task-specific datasets.
10. The apparatus of claim 1 , wherein at least one of the one or more processor circuits is to apply one or more pre-processing techniques on individual datasets of the set of datasets based on a predefined or configured ruleset, wherein the ruleset defines rules for translating the individual datasets into a format that is readable by a multi-objective MLAS function.
11. The apparatus of claim 1 , wherein the at least one of the one or more processor circuits is to determine the respective performance metrics for the tasks in the task list based on a predefined or configured ruleset.
12. The apparatus of claim 11 , wherein, to generate the CPM, at least one of the one or more processor circuits is to calculate the CPM from the respective performance for the tasks using a predefined or configured statistical function.
13. The apparatus of claim 12 , wherein the configured statistical function is included in an ML configuration provided by the client device.
14. The apparatus of claim 1 , wherein the at least one of the one or more processor circuits is to:
operate respective GMLAs in the set of GMLAs using a test dataset; and
measure the corresponding performance metrics based on the operation of the respective GMLAs in the set of GMLAs.
15. The apparatus of claim 1 , wherein at least one of the one or more processor circuits is to predict the corresponding platform-based performance metrics of each GMLA using one or more proxy functions, wherein the one or more proxy functions include one or more of associative arrays, mapping functions, dictionaries, hash tables, look-up tables (LUTs), linked lists, ML classifiers, parameter counting, computational throughput metrics, Jacobian covariance functions, saliency pruning functions, channel pruning functions, heuristic functions, and hyper-heuristic functions.
16. One or more non-transitory computer readable media (NTCRM) comprising instructions to cause one or more processor circuits to at least:
serve a machine learning architecture search interface (MLASI) to a client device;
after receipt of an individual machine learning (ML) configuration from the client device via the MLASI, determine a set of ML tasks and a set of datasets indicated by the individual ML configuration;
generate a task list from the set of ML tasks;
group the set of datasets into a first sub-group of generalized datasets and a second sub-group of task-specific datasets, the task-specific datasets usable with a single task in the task list and the generalized datasets usable with more than one task in the task list;
generate one or more data batches from individual datasets based on the task-specific datasets and the generalized datasets;
determine respective performance metrics for tasks in the task list;
generate a combined performance metric (CPM) based on the performance metrics for the tasks;
determine a set of generalized machine learning architectures (GMLAs) and corresponding platform-based performance metrics based on the CPM and the one or more data batches, wherein a GMLA of the set of GMLAs is a machine learning (ML) architecture that can be used for more than one ML task and with the task-specific datasets and the generalized datasets; and
indicate, via the MLASI, the set of GMLAs and the corresponding performance metrics for deployment on a hardware platform.
17. The one or more NTCRM of claim 16 , wherein the CPM and the platform-based performance metrics are objectives of a multi-objective function and the one or more GMLAs that are represented as decision variables of the multi-objective function.
18. The one or more NTCRM of claim 17 , wherein the instructions are to cause at least one of the one or more processor circuits to determine task-specific weights for corresponding tasks in the task list, wherein the task-specific weights indicate a preference of the corresponding tasks with respect to other tasks in the task list, and the task-specific weights are coefficients that correspond to respective ones of the decision variables in the multi-objective function.
19. The one or more NTCRM of claim 18 , wherein the individual ML configuration includes the set of specified tasks, the set of datasets, the task-specific weights, and the platform-based performance metrics.
20. The one or more NTCRM of claim 18 , wherein the instructions are to cause at least one of the one or more processor circuits to generate individual datasets from the task-specific datasets and the generalized datasets such that the individual datasets correspond to a respective task in the task list.
21. The one or more NTCRM of claim 17 , wherein the multi-objective function includes one or more of a predetermined multi-objective evolutionary algorithm (MOEA), a predetermined Neural Architecture Search (NAS) algorithm, or a configured multi-objective function included in an ML configuration provided by the client device.
22. The one or more NTCRM of claim 16 , wherein the instructions are to cause at least one of the one or more processor circuits to group the set of datasets to the first sub-group of generalized datasets the second sub-group of task-specific datasets using a predetermined or configured ruleset, the ruleset including one or more rules for grouping the set of datasets to the first sub-group of generalized datasets and the second sub-group of task-specific datasets.
23. The one or more NTCRM of claim 16 , wherein the instructions are to cause at least one of the one or more processor circuits to apply one or more pre-processing techniques on individual datasets of the datasets based on a predefined or configured ruleset, wherein the ruleset defines rules for translating the individual datasets into a format that is readable by the a multi-objective MLAS function.
24. The one or more NTCRM of claim 16 , wherein the instructions to cause the at least one of the one or more processor circuits to determine the respective performance metrics for the tasks in the task list based on a predefined or configured ruleset.
25. The one or more NTCRM of claim 24 , wherein, to generate the CPM, the instructions are to cause at least one of the one or more processor circuits to calculate the CPM from the respective performance for the tasks using a predefined or configured statistical function.