Semantic-aware and user-aware admission control for performance management in data analytics and data storage systems

1. A computer program product comprising a computer-readable storage medium having program instructions embodied therewith, the embodied program instructions executable by a processor to cause the processor to:

receive, by the processor, a first job request;

analyze, by the processor, the first job request to determine a user skill level of a user that submitted the first job request; and

admit, by the processor, the first job request to a data analytics system and/or a data storage system in a specified order with respect to other received job requests based on at least the user skill level of the user that submitted the first job request.

2. The computer program product as recited in claim 1 , wherein the embodied program instructions are executable by the processor to cause the processor to temporarily store, by the processor, the first job request into a first queue of a plurality of queues, the first queue being configured to store job requests having a first range of user skill levels different from those of other queues of the plurality of queues, and wherein the user skill level of the user that submitted the first job request is within the first range.

3. The computer program product as recited in claim 2 , wherein the embodied program instructions are executable by the processor to cause the processor to drop, by the processor, a job request that has a lowest dispatch priority with respect to all other job requests in the first queue in response to a determination that the first queue lacks space to store the first job request prior to temporarily storing the first job request into the first queue of the plurality of queues.

4. The computer program product as recited in claim 2 , wherein the embodied program instructions are executable by the processor to cause the processor to determine, by the processor, from which queue of the plurality of queues a next job request is admitted based on a state of each of the queues relative to one another and a current performance state of the data analytics system and/or data storage system, wherein the performance state of the data analytics system and/or data storage system is determined based on an overall workload indicator or an aggregate of workload indicators per node.

5. The computer program product as recited in claim 1 , wherein the embodied program instructions executable to analyze, by the processor, the first job request comprise program instructions executable to:

analyze, by the processor, the first job request to determine an estimated complexity of the first job request based on one or more attributes of the first job request;

analyze, by the processor, semantics of the first job request to determine a size of the first job request and identify the user that submitted the first job request; and

identify, by the processor, one or more patterns in the first job request, each pattern implying a complex operation within the first job request, and

wherein the embodied program instructions executable by the processor to cause the processor to admit, by the processor, the first job request to the data analytics system and/or data storage system in the specified order with respect to other received job requests is further based on the estimated complexity of the first job request.

6. The computer program product as recited in claim 5 , wherein the semantics are selected from the group comprising: keywords in the first job request, a combination of keywords in the first job request, and data set sizes of data sets referenced in the first job request, and wherein the estimated complexity of the first job request is higher when more patterns within the first job request are identified relative to the estimated complexity of the first job request when less or no patterns within the first job request are identified.

7. The computer program product as recited in claim 1 , wherein the embodied program instructions are executable by the processor to cause the processor to classify, by the processor, the first job request into a service class based on one or more attributes of the first job request, and wherein the embodied program instructions executable by the processor to cause the processor to admit, by the processor, the first job request to the data analytics system and/or data storage system in the specified order with respect to other received job requests is further based on the service class of the first job request.

8. The computer program product as recited in claim 7 , wherein the embodied program instructions are executable by the processor to cause the processor to:

temporarily store, by the processor, all received job requests into one of a plurality of queues, each queue being configured to store job requests having different service classes and user skill levels with respect to job requests stored in other queues; and

monitor, by the processor, a set of system resources configured to process admitted job requests in the data analytics system and/or data storage system.

9. The computer program product as recited in claim 8 , wherein each queue represents a dispatch priority and corresponds to one or more tuples having the following dimensions: service class of the job request and user skill level.

10. The computer program product as recited in claim 1 , wherein the embodied program instructions executable by the processor to analyze, by the processor, the first job request to determine the user skill level of the user that submitted the first job request determines the user skill level based on factors selected from the group comprising: user experience level, user job request submission history, and corresponding data analytics system and/or data storage system response history to the user job request submission history.

11. A computer-implemented method, comprising:

receiving a first job request;

analyzing the first job request to determine a user skill level of a user that submitted the first job request; and

admitting the first job request to a data analytics system and/or a data storage system in a specified order with respect to other received job requests based on at least the user skill level of the user that submitted the first job request.

12. The method as recited in claim 11 , further comprising temporarily storing the first job request into a first queue of a plurality of queues, the first queue being configured to store job requests having a first range of user skill levels different from those of other queues of the plurality of queues, and wherein the user skill level of the user that submitted the first job request is within the first range.

13. The method as recited in claim 12 , further comprising dropping a job request that has a lowest dispatch priority with respect to all other job requests in the first queue in response to a determination that the first queue lacks space to store the first job request prior to temporarily storing the first job request into the first queue of the plurality of queues.

14. The method as recited in claim 12 , further comprising determining from which queue of the plurality of queues a next job request is admitted based on a state of each of the queues relative to one another and a current performance state of the data analytics system and/or data storage system, wherein the performance state of the data analytics system and/or data storage system is determined based on an overall workload indicator or an aggregate of workload indicators per node.

15. The method as recited in claim 11 , wherein the analyzing the first job request further comprises:

analyzing the first job request to determine an estimated complexity of the first job request based on one or more attributes of the first job request;

analyzing semantics of the first job request to determine a size of the first job request and identify the user that submitted the first job request; and

identifying one or more patterns in the first job request, each pattern implying a complex operation within the first job request,

wherein the admitting the first job request to the data analytics system and/or data storage system in the specified order with respect to other received job requests is further based on the estimated complexity of the first job request, and

wherein the semantics are selected from the group comprising: keywords in the first job request, a combination of keywords in the first job request, and data set sizes of data sets referenced in the first job request, and wherein the estimated complexity of the first job request is higher when more patterns within the first job request are identified relative to the estimated complexity of the first job request when less or no patterns within the first job request are identified.

16. The method as recited in claim 11 , further comprising classifying the first job request into a service class based on one or more attributes of the first job request, and wherein the admitting the first job request to the data analytics system and/or data storage system in the specified order with respect to other received job requests is further based on the service class of the first job request.

17. The method as recited in claim 16 , further comprising:

temporarily storing all received job requests into one of a plurality of queues, each queue being configured to store job requests having different service classes and user skill levels with respect to job requests stored in other queues; and

monitoring a set of system resources configured to process admitted job requests in the data analytics system and/or data storage system, and

wherein each queue represents a dispatch priority and corresponds to one or more tuples having the following dimensions: service class of the job request and user skill level.

18. The method as recited in claim 11 , wherein the analyzing the first job request to determine the user skill level of the user that submitted the first job request determines the user skill level based on factors selected from the group comprising: user experience level, user job request submission history, and corresponding data analytics system and/or data storage system response history to the user job request submission history.

19. A system, comprising:

one or more hardware processors;

a computer-readable storage medium having software modules embodied therewith that are executable by the one or more hardware processors, the software modules comprising:

a job request analyzer module configured to analyze a job request to determine a user skill level of a user that submitted the job request; and

a job dispatcher module configured to admit the job request to a data analytics system and/or a data storage system in a specified order with respect to other received job requests based on the user skill level of the user that submitted the job request.

20. The system as recited in claim 19 , wherein the job request analyzer module is further configured to analyze the job request to determine an estimated complexity of the job request based on one or more attributes of the job request;

analyze semantics of the job request to determine a size of the job request and identify the user that submitted the job request; and

identify one or more patterns in the job request, each pattern implying a complex operation within the job request,

wherein the semantics are selected from the group comprising: keywords in the job request, a combination of keywords in the job request, and data set sizes of data sets referenced in the job request, and

wherein the estimated complexity of the job request is higher when more patterns within the job request are identified relative to the estimated complexity of the job request when less or no patterns within the job request are identified.