System and method for prediction of job times within an analytics environment
Described herein are systems and methods for prediction of job times within an analytics environment. As jobs are executed, for example, within an application analytics environment, and the complexity of the jobs grow, it is becoming increasingly important to provide an accurate estimation of job completion time. Artificial intelligence and/or machine learning can be utilized, together with one or more of a pre-trained model and a transferred-learning model, to develop a job prediction time model that can be utilized to provide an estimation for a job runtime prediction. In addition, an alert can be provided in the event that a job runs longer than predicted.
1 . A system for prediction of job times within an analytics environment, comprising:
a computer including one or more processors, that provides access to an analytic applications environment;
a machine learning model provided at the analytics applications environment, the machine learning model comprising a set of data associated with job runtimes within the analytic applications environment;
wherein a request for a transaction is received at the analytic applications environment, the requested transaction comprising a plurality of jobs, each of the plurality of jobs comprising a job type;
wherein upon a job of the plurality of jobs being started at the analytic applications environment, an estimated runtime for the job, the job comprising a job type, is determined based upon the machine learning model;
wherein the determination of the estimated runtime for the job of the plurality of jobs is based at least upon a plurality of factors, the plurality of factors comprising components associated with the job of the request; factors affecting performance of the job of the request, data sources associated with the job of the request, a schedule of the job, and inputs associated with the job of the request;
wherein the determination of the estimated runtime for the job of the plurality of jobs is further based at least upon dependencies of the job of the plurality of jobs external to the analytic applications environment;
wherein the estimated runtime is displayed at a user interface accessible by the analytic applications environment; and
wherein, upon the estimated runtime being determined, the machine learning model is updated based at least upon the plurality of factors utilized in determining the estimated runtime for the job and the dependencies of the job of the plurality of jobs external to the analytics application environment.
2 . The system of claim 1 , wherein the machine learning model is at least based on a pre-trained model.
3 . The system of claim 2 , wherein the pre-trained model comprises training data associated with operations other than the requested job type.
4 . The system of claim 1 , wherein the machine learning model is at least based upon a transferred learning model.
5 . The system of claim 1 , wherein the analytics learning environment further comprises a monitoring component, the monitoring component monitoring data and metrics associated with the job of the requested job type.
6 . The system of claim 5 , wherein upon completion of the job of the requested job type, the machine learning model is updated based upon the monitored data and metrics associated with the job of the requested type.
7 . The system of claim 5 , wherein upon the job of the requested job type running longer that the estimated runtime, an alert is published by the applications analytics environment.
8 . A method for prediction of job times within an analytics environment, comprising:
providing a computer including one or more processors, that provides access to an analytic applications environment;
providing a machine learning model at the analytics applications environment, the machine learning model comprising a set of data associated with job runtimes within the analytic applications environment;
receiving a request for a transaction at the analytic applications environment, the requested transaction comprising a plurality of jobs, each of the plurality of jobs comprising a job type;
upon a job of the plurality of jobs being started at the analytic applications environment, the job comprising a job type, determining an estimated runtime for the job of the requested job type based upon the machine learning model, wherein the determination of the estimated runtime for the job is based at least upon a plurality of factors, the plurality of factors comprising components associated with the job of the request; factors affecting performance of the job of the request, data sources associated with the job of the request, a schedule of the job, and inputs associated with the job of the request, wherein the determination of the estimated runtime for the job of the plurality of jobs is further based at least upon dependencies of the job of the plurality of jobs external to the analytic applications environment;
displaying the estimated runtime at a user interface accessible by the analytic applications environment; and
upon the estimated runtime being determined, updating the machine learning model based at least upon the plurality of factors utilized in determining the estimated runtime for the job and the dependencies of the job of the plurality of jobs external to the analytics application environment.
9 . The method of claim 8 , wherein the machine learning model is at least based on a pre-trained model.
10 . The method of claim 9 , wherein the pre-trained model comprises training data associated with operations other than the requested job type.
11 . The method of claim 8 , wherein the machine learning model is at least based upon a transferred learning model.
12 . The method of claim 8 , wherein the analytics learning environment further comprises a monitoring component, the monitoring component monitoring data and metrics associated with the job of the requested job type.
13 . The method of claim 12 , wherein upon completion of the job of the requested job type, the machine learning model is updated based upon the monitored data and metrics associated with the job of the requested type.
14 . The method of claim 12 , wherein upon the job of the requested job type running longer that the estimated runtime, an alert is published by the applications analytics environment.
15 . A non-transitory computer readable storage medium having instructions thereon for prediction of job times within an analytics environment, which when read and executed by a computer including one or more processors cause the computer to perform steps comprising:
providing a computer including one or more processors, that provides access to an analytic applications environment;
providing a machine learning model at the analytics applications environment, the machine learning model comprising a set of data associated with job runtimes within the analytic applications environment;
receiving a request for a transaction at the analytic applications environment, the requested transaction comprising a plurality of jobs, each of the plurality of jobs comprising a job type;
upon a job of the plurality of jobs being started at the analytic applications environment, the job comprising a job type, determining an estimated runtime for the job of the requested job type based upon the machine learning model, wherein the determination of the estimated runtime for the job is based at least upon a plurality of factors, the plurality of factors comprising components associated with the job of the request; factors affecting performance of the job of the request, data sources associated with the job of the request, a schedule of the job, and inputs associated with the job of the request, wherein the determination of the estimated runtime for the job of the plurality of jobs is further based at least upon dependencies of the job of the plurality of jobs external to the analytic applications environment;
displaying the estimated runtime at a user interface accessible by the analytic applications environment; and
upon the estimated runtime being determined, updating the machine learning model based at least upon the plurality of factors utilized in determining the estimated runtime for the job and the dependencies of the job of the plurality of jobs external to the analytics application environment.
16 . The non-transitory computer readable storage medium of claim 15 , wherein the machine learning model is at least based on a pre-trained model.
17 . The non-transitory computer readable storage medium of claim 16 , wherein the pre-trained model comprises training data associated with operations other than the requested job type.
18 . The non-transitory computer readable storage medium of claim 15 , wherein the machine learning model is at least based upon a transferred learning model.
19 . The non-transitory computer readable storage medium of claim 15 , wherein the analytics learning environment further comprises a monitoring component, the monitoring component monitoring data and metrics associated with the job of the requested job type.
20 . The non-transitory computer readable storage medium of claim 19 , wherein upon completion of the job of the requested job type, the machine learning model is updated based upon the monitored data and metrics associated with the job of the requested type.