Automatically processing batch jobs in cloud environments using artificial intelligence techniques
Methods, apparatus, and processor-readable storage media for automatically processing batch jobs in cloud environments using artificial intelligence techniques are provided herein. An example computer-implemented method includes obtaining historical job execution-related data for previous batch jobs in at least one cloud environment and resource utilization-related data for one or more pending batch jobs in the at least one cloud environment; predicting execution outcome(s) for the pending batch job(s) by processing at least a portion of the historical job execution-related data and at least a portion of the resource utilization-related data using artificial intelligence techniques; estimating temporal duration(s) associated with executing the pending batch job(s) by processing the at least a portion of the historical job execution-related data and the at least a portion of the resource utilization-related data using the artificial intelligence techniques; and performing automated actions based on the predicted execution outcome(s) and the estimated temporal duration(s).
1 . A computer-implemented method comprising:
obtaining historical job execution-related data for one or more previous batch jobs in at least one cloud environment and resource utilization-related data for one or more pending batch jobs in the at least one cloud environment;
predicting one or more execution outcomes for the one or more pending batch jobs in the at least one cloud environment by processing at least a portion of the historical job execution-related data and at least a portion of the resource utilization-related data using at least one neural network comprising multiple network branches generating respective multiple outputs of different types, wherein a first of the multiple network branches comprises a classifier and is associated with predicting the one or more execution outcomes for the one or more pending batch jobs;
estimating one or more temporal durations associated with executing the one or more pending batch jobs in the at least one cloud environment by processing the at least a portion of the historical job execution-related data and the at least a portion of the resource utilization-related data using the at least one neural network comprising the multiple network branches, wherein a second of the multiple network branches comprises a regressor and is associated with estimating the one or more temporal durations associated with executing the one or more pending batch jobs in the at least one cloud environment, and wherein the first branch and the second branch share at least a portion of a common input layer that receives a common set of input data comprising the at least a portion of the historical job execution-related data and the at least a portion of the resource utilization-related data; and
performing one or more automated actions based at least in part on the one or more predicted execution outcomes and the one or more estimated temporal durations;
wherein the method is performed by at least one processing device comprising a processor coupled to a memory.
2 . The computer-implemented method of claim 1 , wherein performing one or more automated actions comprises automatically scheduling at least a portion of the one or more pending batch jobs to be executed at one or more given times based at least in part on the one or more predicted execution outcomes and the one or more estimated temporal durations.
3 . The computer-implemented method of claim 1 , wherein performing one or more automated actions comprises automatically executing at least a portion of one or more pending batch jobs at one or more given times based at least in part on the one or more predicted execution outcomes and the one or more estimated temporal durations.
4 . The computer-implemented method of claim 1 , wherein performing one or more automated actions comprises automatically training at least a portion of the at least one neural network based at least in part on feedback to one or more of the one or more predicted execution outcomes and the one or more estimated temporal durations.
5 . The computer-implemented method of claim 1 , wherein obtaining historical job execution-related data comprises obtaining one or more of data pertaining to historical job execution outcomes and data pertaining to processing time data of one or more types of jobs.
6 . The computer-implemented method of claim 1 , wherein obtaining utilization-related data for one or more pending batch jobs comprises obtaining one or more of central processing unit data, memory data, storage utilization data, input-output information, host infrastructure availability information, information pertaining to at least one of load, volume, and seasonality, date and time information, and job name information.
7 . A non-transitory processor-readable storage medium having stored therein program code of one or more software programs, wherein the program code when executed by at least one processing device causes the at least one processing device:
to obtain historical job execution-related data for one or more previous batch jobs in at least one cloud environment and resource utilization-related data for one or more pending batch jobs in the at least one cloud environment;
to predict one or more execution outcomes for the one or more pending batch jobs in the at least one cloud environment by processing at least a portion of the historical job execution-related data and at least a portion of the resource utilization-related data using at least one neural network comprising multiple network branches generating respective multiple outputs of different types, wherein a first of the multiple network branches comprises a classifier and is associated with predicting the one or more execution outcomes for the one or more pending batch jobs;
to estimate one or more temporal durations associated with executing the one or more pending batch jobs in the at least one cloud environment by processing the at least a portion of the historical job execution-related data and the at least a portion of the resource utilization-related data using the at least one neural network comprising the multiple network branches, wherein a second of the multiple network branches comprises a regressor and is associated with estimating the one or more temporal durations associated with executing the one or more pending batch jobs in the at least one cloud environment, and wherein the first branch and the second branch share at least a portion of a common input layer that receives a common set of input data comprising the at least a portion of the historical job execution-related data and the at least a portion of the resource utilization-related data; and
to perform one or more automated actions based at least in part on the one or more predicted execution outcomes and the one or more estimated temporal durations.
8 . The non-transitory processor-readable storage medium of claim 7 , wherein performing one or more automated actions comprises automatically scheduling at least a portion of the one or more pending batch jobs to be executed at one or more given times based at least in part on the one or more predicted execution outcomes and the one or more estimated temporal durations.
9 . The non-transitory processor-readable storage medium of claim 7 , wherein performing one or more automated actions comprises automatically executing at least a portion of one or more pending batch jobs at one or more given times based at least in part on the one or more predicted execution outcomes and the one or more estimated temporal durations.
10 . The non-transitory processor-readable storage medium of claim 7 , wherein performing one or more automated actions comprises automatically training at least a portion of the at least one neural network based at least in part on feedback to one or more of the one or more predicted execution outcomes and the one or more estimated temporal durations.
11 . The non-transitory processor-readable storage medium of claim 7 , wherein obtaining historical job execution-related data comprises obtaining one or more of data pertaining to historical job execution outcomes and data pertaining to processing time data of one or more types of jobs.
12 . The non-transitory processor-readable storage medium of claim 7 , wherein obtaining utilization-related data for one or more pending batch jobs comprises obtaining one or more of central processing unit data, memory data, storage utilization data, input-output information, host infrastructure availability information, information pertaining to at least one of load, volume, and seasonality, date and time information, and job name information.
13 . An apparatus comprising:
at least one processing device comprising a processor coupled to a memory;
the at least one processing device being configured:
to obtain historical job execution-related data for one or more previous batch jobs in at least one cloud environment and resource utilization-related data for one or more pending batch jobs in the at least one cloud environment;
to predict one or more execution outcomes for the one or more pending batch jobs in the at least one cloud environment by processing at least a portion of the historical job execution-related data and at least a portion of the resource utilization-related data using at least one neural network comprising multiple network branches generating respective multiple outputs of different types, wherein a first of the multiple network branches comprises a classifier and is associated with predicting the one or more execution outcomes for the one or more pending batch jobs;
to estimate one or more temporal durations associated with executing the one or more pending batch jobs in the at least one cloud environment by processing the at least a portion of the historical job execution-related data and the at least a portion of the resource utilization-related data using the at least one neural network comprising the multiple network branches, wherein a second of the multiple network branches comprises a regressor and is associated with estimating the one or more temporal durations associated with executing the one or more pending batch jobs in the at least one cloud environment, and wherein the first branch and the second branch share at least a portion of a common input layer that receives a common set of input data comprising the at least a portion of the historical job execution-related data and the at least a portion of the resource utilization-related data; and
to perform one or more automated actions based at least in part on the one or more predicted execution outcomes and the one or more estimated temporal durations.
14 . The apparatus of claim 13 , wherein performing one or more automated actions comprises automatically scheduling at least a portion of the one or more pending batch jobs to be executed at one or more given times based at least in part on the one or more predicted execution outcomes and the one or more estimated temporal durations.
15 . The apparatus of claim 13 , wherein performing one or more automated actions comprises automatically executing at least a portion of one or more pending batch jobs at one or more given times based at least in part on the one or more predicted execution outcomes and the one or more estimated temporal durations.
16 . The apparatus of claim 13 , wherein performing one or more automated actions comprises automatically training at least a portion of the at least one neural network based at least in part on feedback to one or more of the one or more predicted execution outcomes and the one or more estimated temporal durations.
17 . The apparatus of claim 13 , wherein obtaining historical job execution-related data comprises obtaining one or more of data pertaining to historical job execution outcomes and data pertaining to processing time data of one or more types of jobs.
18 . The apparatus of claim 13 , wherein obtaining utilization-related data for one or more pending batch jobs comprises obtaining one or more of central processing unit data, memory data, storage utilization data, input-output information, host infrastructure availability information, information pertaining to at least one of load, volume, and seasonality, date and time information, and job name information.