Propagating job contexts to a job execution environment
In a job management environment comprising a plurality of job systems and a scheduler for scheduling a job submitted to the job management environment to a job system for running, a processor, in a first job system, intercepts, from outside of a first container in the first job system, a first job from being sent to the scheduler. A processor, in the first job system, determines whether the first job is submitted from a container in the first job system. In response to a first determination that the first job is submitted from a container in the first job system, a processor, in the first job system, determines contexts of the first job, the contexts of the first job including a context related to the first container. A processor, in the first job system, sends the first job together with the contexts of the first job to the scheduler.
1. A computer-implemented method in a job management environment comprising a plurality of job systems and a scheduler for scheduling a job submitted to the job management environment to a job system for running, the method comprising:
intercepting, by one or more processing units in a first job system and job context agent, from outside of a first container in the first job system, a first job from being sent to the scheduler;
determining, by one or more processing units in the first job system and the job context agent, whether the first job is submitted from a container in the first job system; and
in response to determining that the first job is submitted from a container in the first job system,
determining, by one or more processing units in the first job system, contexts of the first job, the contexts of the first job including a context related to the first container and includes an ID (identification) of a first user in the first job system submitting the first job and a privilege of the first user in the first job system and
the privilege of the first user in the first job system includes at least one of a mounted network file system accessible to the first user and a mounted local file system in the first job system accessible to the first user, and
sending, by one or more processing units in the first job system, the first job together with the contexts of the first job to the scheduler.
2. The method of claim 1 , wherein a broker module is configured to receive the job from the scheduler and initiate an execution of the job.
3. The method of claim 1 , wherein the job context agent is configured to intercept the first job instead of being received by the scheduler and makes a determination whether to submit the first job by querying an operational system via an API.
4. The method of claim 1 , wherein the determining the contexts of the first job comprises:
determining, by one or more processing units in the first job system, customized contexts of the first job, the context related to the first container, the ID of the first user submitting the first job in the first job system, and the privilege of the first user in the first job system, respectively; and
combining, by one or more processing units in the first job system, the customized contexts of the first job, the context related to the first container, the ID of the first user in the first job system submitting the first job, and the privilege of the first user in the first job system into the contexts of the first job.
5. The method of claim 1 , further comprising:
in response to determining that the privilege of the first user in the first job system includes a mounted local file system in the first job system accessible to the first user,
monitoring, by one or more processing units in the first job system, a register request from a third job system for accessing the mounted local file system in the first job system, and
in response to a successful register, enabling, by one or more processing units in the first job system, the third job system to access the mounted local file system in the first job system.
6. The method of claim 1 , further comprising:
in response to receiving a second job for running together with contexts of the second job sent from the scheduler,
determining, by one or more processing units in the first job system, whether the contexts of the second job comprise a context related to a second container, and
in response to determining that the contexts of the second job comprise the context related to the second container,
deploying, by one or more processing units in the first job system, the second container in the first job system based on the context related to the second container, and
running, by one or more processing units in the first job system, the second job in the deployed second container.
7. The method of claim 6 , wherein the deploying the second container in the first job system based on the context related to the second container further comprises:
in response to determining that a privilege of a second user submitting the second job in a second job system comprises a mounted network file system accessible to the second user, mounting, by one or more processing units in the first job system, the mounted network file system for the second job running in the second container in the first job system.
8. The method of claim 6 , wherein the deploying the second container in the first job system based on the context related to the second container further comprises:
in response to determining that a privilege of a second user submitting the second job in the second job system comprises a mounted local file system in the second job system accessible to the second user,
registering, by one or more processing units in the first job system, the second job in the first job system with the second job system, and
in response to a successful register, accessing, by one or more processing units in the first job system, the mounted local file system in the second job system when needed.
9. The method of claim 1 , wherein a job system of the plurality of job system is a cloud environment.
10. A first job system in a job management environment comprising a plurality of job systems and a scheduler for scheduling a job submitted to the job management environment to a job system for running, comprising:
at least one processing unit; and
a memory coupled to the at least one processing unit and storing instructions thereon, the instructions, when executed by the at least one processing unit, performing actions including:
intercepting, by one or more processing units in a first job system and job context agent, from outside of a first container in the first job system, a first job from being sent to the scheduler;
determining, by one or more processing units in the first job system and the job context agent, whether the first job is submitted from a container in the first job system; and
in response to determining that the first job is submitted from a container in the first job system,
determining, by one or more processing units in the first job system, contexts of the first job, the contexts of the first job including a context related to the first container and includes an ID (identification) of a first user in the first job system submitting the first job and a privilege of the first user in the first job system and
the privilege of the first user in the first job system includes at least one of a mounted network file system accessible to the first user and a mounted local file system in the first job system accessible to the first user, and
sending, by one or more processing units in the first job system, the first job together with the contexts of the first job to the scheduler.
11. The first job system of claim 10 , wherein a job context agent is configured to intercept the first job instead of being received by the scheduler and makes a determination whether to submit the first job by querying an operational system via an API.
12. The first job system of claim 10 , further comprising:
in response to determining that the privilege of the first user in the first job system includes a mounted local file system in the first job system accessible to the first user,
monitoring a register request from a third job system for accessing the mounted local file system in the first job system, and
in response to a successful register, enabling the third job system to access the mounted local file system in the first job system.
13. The first job system of claim 12 , further comprising:
in response to receiving a second job for running together with contexts of the second job sent from the scheduler,
determining whether the contexts of the second job comprise a context related to a second container; and
in response to determining that the contexts of the second job comprise the context related to the second container,
deploying the second container in the first job system based on the context related to the second container; and
running the second job in the deployed second container.
14. The first job system of claim 13 , wherein the deploying the second container in the first job system based on the context related to the second container further comprises:
in response to determining that a privilege of a second user submitting the second job in a second job system comprises a mounted network file system accessible to the second user, mounting, by one or more processing units in the first job system, the mounted network file system for the second job running in the second container in the first job system.
15. The first job system of claim 13 , wherein the deploying the second container in the first job system based on the context related to the second container further comprises:
in response to determining that a privilege of a second user submitting the second job in the second job system comprises a mounted local file system in the second job system accessible to the second user,
registering the second job in the first job system with the second job system, and
in response to a successful register, accessing the mounted local file system in the second job system when needed.
16. A computer program product used by a first job system in a job management environment comprising a plurality of job systems and a scheduler for scheduling a job submitted to the job management environment to a job system for running, comprising a computer-readable storage medium having program instructions embodied therewith, the program instructions executable by a first component with at least one processing unit in a cloud computing environment to cause the at least one processing unit to perform actions including:
intercepting, by one or more processing units in a first job system and job context agent, from outside of a first container in the first job system, a first job from being sent to the scheduler;
determining, by one or more processing units in the first job system and the job context agent, whether the first job is submitted from a container in the first job system; and
in response to determining that the first job is submitted from a container in the first job system,
determining, by one or more processing units in the first job system, contexts of the first job, the contexts of the first job including a context related to the first container and includes an ID (identification) of a first user in the first job system submitting the first job and a privilege of the first user in the first job system and
the privilege of the first user in the first job system includes at least one of a mounted network file system accessible to the first user and a mounted local file system in the first job system accessible to the first user, and
sending, by one or more processing units in the first job system, the first job together with the contexts of the first job to the scheduler.
17. The computer program product of claim 16 , wherein a job context agent is configured to intercept the first job instead of being received by the scheduler and makes a determination whether to submit the first job by querying an operational system via an API.
18. The computer program product of claim 16 , the actions further comprising:
in response to determining that the privilege of the first user in the first job system includes a mounted local file system in the first job system accessible to the first user,
monitoring a register request from a third job system for accessing the mounted local file system in the first job system, and
in response to a successful register, enabling the third job system to access the mounted local file system in the first job system.
19. The computer program product of claim 18 , further comprising:
in response to receiving a second job for running together with contexts of the second job sent from the scheduler,
determining whether the contexts of the second job comprise a context related to a second container; and
in response to determining that the contexts of the second job comprise the context related to the second container,
deploying the second container in the first job system based on the context related to the second container, and
running the second job in the deployed second container.
20. The computer program product of claim 19 , further comprising:
in response to determining that a privilege of a second user submitting the second job in a second job system comprises a mounted network file system accessible to the second user, mounting, by one or more processing units in the first job system, the mounted network file system for the second job running in the second container in the first job system; and
in response to determining that a privilege of a second user submitting the second job in the second job system comprises a mounted local file system in the second job system accessible to the second user,
registering the second job in the first job system with the second job system, and
in response to a successful register, accessing the mounted local file system in the second job system when needed.