Grid job submission using shared network filesystem
A method for executing a job using a remote file system that includes obtaining a plurality of job data files by the remote file system, wherein the plurality of job data files are created on the remote file system, detecting a job request for the job in an input directory to obtain a detected job request, wherein the job request is generated by a client, and executing the job based on the job data files and the detected job request, wherein executing the job generates an output file, wherein the remote file system is accessible using a local file system of the client.
1. A method for processing a job using a remote file system comprising:
obtaining, by the remote file system located on a first node, a plurality of job data files and a job request comprising a program from a second node, the second node comprising:
a file system comprising a directory structure that includes an interface to local storage on the second node and an interface to the remote file system, and
wherein the second node is configured to transfer the job request and the plurality of job data files to the remote file system using the interface to the remote file system, and
wherein transferring the job request comprises storing the job request as a file in an input directory in the remote file system;
detecting, by the first node, that the job request for the job is stored in the input directory of the remote file system;
receiving at least one parameter to execute the program, wherein the at least one parameter specifies at least one from a group consisting of a time to execute the program, an operating system, a cost, and a priority; and
executing, by the first node, using the plurality of job data files, the program based on detecting that the job request is stored in the input directory, wherein executing the program generates an output file and
wherein executing the program is performed according to the parameter, wherein the first node and the second node are different computer systems.
2. The method of claim 1 , wherein the plurality of job data files are stored in a jobs directory on the remote file system.
3. The method of claim 1 , further comprising:
outputting the output file to an output directory of the remote file system.
4. The method of claim 1 , wherein the output file is viewable via the interface to the remote file system during execution of the program.
5. The method of claim 1 , wherein a listener associated with the input directory detects a change of the job request in the input directory, wherein, in response to detecting the change, the listener sends a notification of the change to a jobs engine, and wherein the job engine schedules execution of the program in response to the notification.
6. A distributed computer system having a plurality of nodes for performing a method comprising:
obtaining, by a remote file system located on a first node, a plurality of job data files and a job request comprising a program from a second node, the second node comprising:
a file system comprising a directory structure that includes an interface to local storage on the second node and an interface to the remote file system, and
wherein the second node is configured to transfer the job request and the plurality job data files to the remote file system using the interface to the remote file system, and
wherein transferring the job request comprises storing the job request as a file in an input directory in the remote file system;
detecting, by the first node, that the job request for a job is stored in the input directory of the remote file system;
receiving, by the first node, at least one parameter to execute the program, wherein the at least one parameter specifies at least one from a group consisting of a time to execute the program, an operating system, a cost, and a priority; and
executing, by the first node, using the plurality of job data files, the program based on detecting that the job request is stored in the input directory, wherein executing the program generates an output file and wherein executing the program is performed according to the parameter,
wherein the first node and the second node are different computer systems.
7. The distributed computer system of claim 6 , wherein the output file is viewable via the interface to the remote file system during execution of the program.
8. The distributed computer system of claim 6 , wherein a listener associated with the input directory detects a change of the job request in the input directory, wherein, in response to detecting the change, the listener sends a notification of the change to a jobs engine, and wherein the job engine schedules execution of the program in response to the notification.