Ticket queue for controlling compute process access to shared data and compute resources
View Patent ↗Controlling compute process access to shared data and compute resources includes, responsive to a compute process determining that access to at least one of shared resources and shared data is necessary to perform a compute task, creating, by the compute process, a ticket file belonging to the compute process in a ticket queue directory. The compute process is allowed to proceed performing the compute task upon determining that the ticket file is first in line in a ticket queue of the ticket queue directory, according to a ticket ordering algorithm independently applied by the compute process. Subsequent to completing the compute task, the compute process removes the ticket from the ticket queue directory.
1 . A computer-implemented method for compute process shared access management, the computer-implemented method, comprising:
responsive to a compute process determining that access to at least one of shared resources and shared data is necessary to perform a compute task, creating, by the compute process, a ticket file belonging to the compute process in a ticket queue directory, wherein the ticket file includes an identifier of the compute process, a creation time of the ticket file, a priority level of the compute process, and a timeout threshold for the compute process, wherein a user is enabled to perform a visual inspection of the ticket queue directory and create, delete and rearrange ticket files within the ticket queue directory as desired;
waiting a predefined amount of time before allowing the compute process to proceed performing the compute task in order to ensure the compute process is independent from network delays;
allowing the compute process to proceed performing the compute task after the predefined amount of time and upon determining that the ticket file is first in line in a ticket queue of the ticket queue directory by independently applying a deterministic, filename-based ticket ordering algorithm, wherein the compute process does not coordinate with an external ticketing service;
implementing a resource allocation policy that allows allocation to the compute process at a current point in time, wherein the resource allocation policy considers resource availability, process priority, resource fairness and resource buffers and limits, and the resource allocation policy constraining the ticket queue directory, wherein, responsive to determining that the ticket file belonging to the compute process is the first in line in the ticket queue however allowing the compute process to be performed would violate the resource allocation policy at the current point in time, performing:
disallowing the compute task from being performed;
removing the ticket file belonging to the compute process from the ticket queue directory;
creating a new ticket file for the compute process; and
placing the new ticket file in the ticket queue of the ticket queue directory according to the deterministic, filename-based ticket ordering algorithm applied by the compute process; and
removing, by the compute process, the ticket file from the ticket queue directory upon completing the compute task.
2 . The computer-implemented method of claim 1 , wherein
an ordering of the ticket queue directory is based on a deterministic function of the information according to the deterministic, filename-based ticket ordering algorithm respectively applied by each ticket file in the ticket queue directory.
3 . The computer-implemented method of claim 1 , further comprising: defining a timeout threshold for the compute process, wherein, an alternative compute process is allowed to perform an alternative compute task, notwithstanding whether the ticket file belonging to the alternative compute process is the first in line in the ticket queue, when the timeout threshold for the compute process has been exceeded.
4 . The computer-implemented method of claim 1 , further comprising:
assigning the compute process to one of a plurality of process classes defined for the ticket queue directory, wherein the deterministic, filename-based ticket ordering algorithm factors in a priority assigned to the compute process according to which of the plurality of process classes the compute process is assigned to; and
applying a resource throttling policy to the compute process, wherein the resource throttling policy consists of a dynamic limit of resources available for use by the compute process to perform the compute task relative to a total resource availability, and a buffer of a minimum number of resources that must be kept available for alternative compute processes.
5 . The computer-implemented method of claim 1 , wherein
the ticket file is an empty file stored in a common writable directory and visible to all other operating processes, or
the ticket file is stored local to the compute process and is shared with all other operating processes.
6 . A system for compute process shared access management, the system comprising:
a hardware memory; and
a hardware processor executing instructions stored in the hardware memory; wherein, when executed, the instructions cause the hardware processor to:
responsive to a compute process determining that access to at least one of shared resources and shared data is necessary to perform a compute task, create, by the compute process, a ticket file belonging to the compute process in a ticket queue directory, wherein the ticket file includes an identifier of the compute process, a creation time of the ticket file, a priority level of the compute process, and a timeout threshold for the compute process, wherein a user is enabled to perform a visual inspection of the ticket queue directory and create, delete and rearrange ticket files within the ticket queue directory as desired;
wait a predefined amount of time before allowing the compute process to proceed performing the compute task in order to ensure the compute process is independent from network delays;
allow the compute process to proceed performing the compute task after the predefined amount of time and upon determining that the ticket file is first in line in a ticket queue of the ticket queue directory by independently applying a deterministic, filename-based ticket ordering algorithm, wherein the compute process does not coordinate with an external ticketing service;
implement a resource allocation policy that allows allocation to the compute process at a current point in time, wherein the resource allocation policy considers resource availability, process priority, resource fairness and resource buffers and limits, and the resource allocation policy constraining the ticket queue directory, wherein, responsive to determining that the ticket file belonging to the compute process is the first in line in the ticket queue however allowing the compute process to be performed would violate the resource allocation policy at the current point in time, performing:
disallowing the compute task from being performed;
removing the ticket file belonging to the compute process from the ticket queue directory;
creating a new ticket file for the compute process; and
placing the new ticket file in the ticket queue of the ticket queue directory according to the deterministic, filename-based ticket ordering algorithm applied by the compute process; and
remove, by the compute process, the ticket file from the ticket queue directory upon completing the compute task.
7 . The system of claim 6 , wherein
an ordering of the ticket queue directory is based on a deterministic function of the information according to the deterministic, filename-based ticket ordering algorithm respectively applied by each ticket file in the ticket queue directory.
8 . The system of claim 6 , wherein, when executed, the instructions further cause the hardware processor to:
define a timeout threshold for the compute process, wherein, an alternative compute process is allowed to perform an alternative compute task, notwithstanding whether the ticket file belonging to the alternative compute process is the first in line in the ticket queue, when the timeout threshold for the compute process has been exceeded.
9 . The system of claim 6 , wherein, when executed, the instructions further cause the hardware processor to:
assign the compute process to one of a plurality of process classes defined for the ticket queue directory, wherein the deterministic, filename-based ticket ordering algorithm factors in a priority assigned to the compute process according to which of the plurality of process classes the compute process is assigned to; and
apply a resource throttling policy to the compute process, wherein the resource throttling policy consists of a dynamic limit of resources available for use by the compute process to perform the compute task relative to a total resource availability, and a buffer of a minimum number of resources that must be kept available for alternative compute processes.
10 . The system of claim 6 , wherein
the ticket file is an empty file stored in a common writable directory and visible to all other operating processes, or
the ticket file is stored local to the compute process and is shared with all other operating processes.
11 . A computer program product for compute process shared access management, the computer program product comprising a non-transitory computer-readable storage medium having program instructions embodied thereon, the program instructions executable by a processor to cause the processor to:
responsive to a compute process determining that access to at least one of shared resources and shared data is necessary to perform a compute task, create, by the compute process, a ticket file belonging to the compute process in a ticket queue directory, wherein the ticket file includes an identifier of the compute process, a creation time of the ticket file, a priority level of the compute process, and a timeout threshold for the compute process, wherein a user is enabled to perform a visual inspection of the ticket queue directory and create, delete and rearrange ticket files within the ticket queue directory as desired;
wait a predefined amount of time before allowing the compute process to proceed performing the compute task in order to ensure the compute process is independent from network delays;
allow the compute process to proceed performing the compute task after the predefined amount of time and upon determining that the ticket file is first in line in a ticket queue of the ticket queue directory by applying a deterministic, filename-based ticket ordering algorithm, wherein the compute process does not coordinate with an external ticketing service;
implement a resource allocation policy that allows allocation to the compute process at a current point in time, wherein the resource allocation policy considers resource availability, process priority, resource fairness and resource buffers and limits, and the resource allocation policy constraining the ticket queue directory, wherein, responsive to determining that the ticket file belonging to the compute process is the first in line in the ticket queue however allowing the compute process to be performed would violate the resource allocation policy at the current point in time, performing:
disallowing the compute task from being performed;
removing the ticket file belonging to the compute process from the ticket queue directory;
creating a new ticket file for the compute process; and
placing the new ticket file in the ticket queue of the ticket queue directory according to the deterministic, filename-based ticket ordering algorithm applied by the compute process; and
remove, by the compute process, the ticket file from the ticket queue directory upon completing the compute task.
12 . The computer program product of claim 11 , wherein
an ordering of the ticket queue directory is based on a deterministic function of the information according to the deterministic, filename-based ticket ordering algorithm respectively applied by each ticket file in the ticket queue directory.
13 . The computer program product of claim 11 , wherein the program instructions executable by the processor further cause the processor to:
define a timeout threshold for the compute process, wherein, an alternative compute process is allowed to perform an alternative compute task, notwithstanding whether the ticket file belonging to the alternative compute process is the first in line in the ticket queue, when the timeout threshold for the compute process has been exceeded;
assign the compute process to one of a plurality of process classes defined for the ticket queue directory, wherein the deterministic, filename-based ticket ordering algorithm factors in a priority assigned to the compute process according to which of the plurality of process classes the compute process is assigned to; and
apply a resource throttling policy to the compute process, wherein the resource throttling policy consists of a dynamic limit of resources available for use by the compute process to perform the compute task relative to a total resource availability, and a buffer of a minimum number of resources that must be kept available for alternative compute processes.
14 . The computer program product of claim 11 , wherein
the ticket file is an empty file stored in a common writable directory and visible to all other operating processes, or
the ticket file is stored local to the compute process and is shared with all other operating processes.