Method and system for performing workloads in a data cluster
Techniques described herein relate to systems, methods, and non-transitory computer readable mediums for performing workloads on hardware configurations. The method includes determining compliant hardware configurations of the data cluster using workload features associated with the workload, hardware specification information associated with hardware of data nodes of a data cluster, and a first machine learning model; generating performance predictions associated with the compliant hardware configurations using the workload features, a portion of the hardware specification information associated with the compliant hardware configurations, and a second machine learning model; and generating a recommendation using the performance predictions, and the recommendation specifies a hardware configuration of the compliant hardware configurations. The workload is performed using the recommendation.
1. A method, comprising:
obtaining, by a processor of a first computing device, a workload, wherein the workload comprises training a deep learning model;
generating workload features associated with the workload;
obtaining hardware specification information associated with hardware of computing devices of a data cluster, wherein the first computing device is operatively connected to the data cluster;
executing, by the processor, a first machine learning model to determine compliant hardware configurations of the data cluster using the workload features and the hardware specification information, wherein the workload features comprise a deep learning model type associated with the deep learning model and workload properties associated with the deep learning model, and wherein the compliant hardware configurations meet requirements of the workload;
executing, by the processor, a second machine learning model to generate performance predictions associated with the compliant hardware configurations using the workload features and a portion of the hardware specification information associated with the compliant hardware configurations, wherein the performance predictions comprise predicted training times of the deep learning model on the compliant hardware configurations, wherein the first machine learning model is different than the second machine learning model;
generating a recommendation using the performance predictions, wherein the recommendation specifies a hardware configuration of the compliant hardware configurations; and
sending the recommendation to the data cluster, wherein the recommendation specifies a second computing device of the data cluster, and a second processor of the second computing device executes the workload using the recommendation.
2. The method of claim 1 , further comprising:
following the sending of the recommendation to the data cluster:
obtaining, by the processor, feedback training data from the data cluster; and
updating the second machine learning model based on the feedback training data to generate an updated second machine learning model.
3. The method of claim 1 , wherein generating the recommendation comprises:
identifying a set of compliant hardware configurations that meet a user requirement;
ranking the set of compliant hardware configurations based on a user preference to generate ranked compliant hardware configurations; and
generating the recommendation that specifies a top ranked compliant hardware configuration of the ranked compliant hardware configurations, wherein the top ranked compliant hardware configuration is the hardware configuration of the second computing device.
4. The method of claim 1 , wherein the hardware specification information specifies components and component characteristics associated with the hardware of the computing devices of the data cluster.
5. The method of claim 1 , wherein the first machine learning model is a neural network classification model.
6. The method of claim 1 , wherein the second machine learning model is a neural network regression model.
7. The method of claim 1 , wherein the workload properties associated with the deep learning model comprise one or more of: a complexity of the deep learning model, a number of convolution layers, a number of dense layers, a number of neurons per layer, a number of filters, a size of filters, and input and output sizes.
8. A system, comprising:
a data cluster comprising computing devices, wherein a first computing device of the computing devices comprises a processor and memory; and
a second computing device, operatively connected to the data cluster, comprising a second processor and second memory and configured to:
obtain a workload, wherein the workload comprises training a deep learning model;
generate workload features associated with the workload;
obtain hardware specification information associated with hardware of the computing devices;
execute a first machine learning model to determine compliant hardware configurations of the data cluster using the workload features and the hardware specification information, wherein the workload features comprise a deep learning model type associated with the deep learning model and workload properties associated with the deep learning model, and wherein the compliant hardware configurations meet requirements of the workload;
execute a second machine learning model to generate performance predictions associated with the compliant hardware configurations using the workload features and a portion of the hardware specification information associated with the compliant hardware configurations, wherein the performance predictions comprise predicted training times of the deep learning model on the compliant hardware configurations, wherein the first machine learning model is different than the second machine learning model;
generate a recommendation using the performance predictions, wherein the recommendation specifies a hardware configuration of the compliant hardware configurations; and
send the recommendation to the first computing device, wherein the recommendation specifies the first computing device, and the processor of the first computing device executes the workload using the recommendation.
9. The system of claim 8 , wherein the second computing device is further configured to:
following the sending of the recommendation to the data cluster:
obtain feedback training data from the data cluster; and
update the second machine learning model based on the feedback training data to generate an updated second machine learning model.
10. The system of claim 8 , wherein generating the recommendation comprises:
identifying a set of compliant hardware configurations that meet a user requirement;
ranking the set of compliant hardware configurations based on a user preference to generate ranked compliant hardware configurations; and
generating the recommendation that specifies a top ranked compliant hardware configuration of the ranked compliant hardware configurations, wherein the top ranked compliant hardware configuration is the hardware configuration of the first computing device.
11. The system of claim 8 , wherein the hardware specification information specifies components and component characteristics associated with the hardware of the computing devices.
12. The system of claim 8 , wherein the first machine learning model is a neural network classification model.
13. The system of claim 8 , wherein the second machine learning model is a neural network regression model.
14. The system of claim 8 , wherein the workload properties associated with the deep learning model comprise one or more of: a complexity of the deep learning model, a number of convolution layers, a number of dense layers, a number of neurons per layer, a number of filters, a size of filters, and input and output sizes.
15. A non-transitory computer readable medium comprising computer readable program code, which when executed by a computer processor of a first computing device enables the computer processor to perform a method, the method comprising:
obtaining a workload, wherein the workload comprises training a deep learning model;
generating workload features associated with the workload;
obtaining hardware specification information associated with hardware of computing devices of a data cluster wherein the first computing device is operatively connected to the data cluster;
executing a first machine learning model to determine compliant hardware configurations of the data cluster using the workload features and the hardware specification information, wherein the workload features comprise a deep learning model type associated with the deep learning model and workload properties associated with the deep learning model, and wherein the compliant hardware configurations meet requirements of the workload;
executing a second machine learning model to generate performance predictions associated with the compliant hardware configurations using the workload features and a portion of the hardware specification information associated with the compliant hardware configurations, wherein the performance predictions comprise predicted training times of the deep learning model on the compliant hardware configurations, wherein the first machine learning model is different than the second machine learning model;
generating a recommendation using the performance predictions, wherein the recommendation specifies a hardware configuration of the compliant hardware configurations; and
sending the recommendation to the data cluster, wherein the recommendation specifies a second computing device of the data cluster, and a second processor of the second computing device executes the workload using the recommendation.
16. The non-transitory computer readable medium of claim 15 , wherein the method further comprises:
following the sending of the recommendation to the data cluster:
obtaining feedback training data from the data cluster; and
updating the second machine learning model based on the feedback training data to generate an updated second machine learning model.
17. The non-transitory computer readable medium of claim 15 , wherein generating the recommendation comprises:
identifying a set of compliant hardware configurations that meet a user requirement;
ranking the set of compliant hardware configurations based on a user preference to generate ranked compliant hardware configurations; and
generating a recommendation that specifies a top ranked compliant hardware configuration of the ranked compliant hardware configurations, wherein the top ranked compliant hardware configuration is the hardware configuration of the second computing device.
18. The non-transitory computer readable medium of claim 15 , wherein the workload properties associated with the deep learning model comprise one or more of: a complexity of the deep learning model, a number of convolution layers, a number of dense layers, a number of neurons per layer, a number of filters, a size of filters, and input and output sizes.
19. The non-transitory computer readable medium of claim 15 , wherein the hardware specification information specifies components and component characteristics associated with the hardware of the computing devices of the data cluster.
20. The non-transitory computer readable medium of claim 15 , wherein the first machine learning model is a neural network classification model and the second machine learning model is a neural network regression model.