System and method for secure multitenant operations of a distributed computing cluster

1. A method of creating an ingest job agent for secure multitenant operations of a distributed computing cluster, comprising:

receiving a network service call from a client computer containing credentials of an operator, in which the client computer is located in a network security domain coupled to the distributed computing cluster;

initiating the network service call to a centralized directory server, located in the network security domain, to authenticate the operator using the transmitted credentials;

presenting a user interface to collect a plurality of ingest job agent configuration options from the operator;

storing the ingest job agent configuration options to a data store in the centralized directory server;

initiating a series of asynchronous operations to create the ingest job agent; and

updating a metadata store in the distributed computing cluster to indicate to the operator that the creation of the ingest job agent is completed, wherein the series of asynchronous operations comprises:

creating an ingest source folder in an ingest server where files shall be transferred from;

creating an ingest sink folder on a distributed file system where files are transferred to;

creating an access control list entry which grants and limits access of the ingest source folder and the ingest sink folder to the operator of the ingest job agent and other operators specified by the operator; and

initiating an in-memory process which monitors and moves files an authorized operator deposits inside the ingest source folder into the ingest sink folder via network file transfer.

2. The method of claim 1 , the series of asynchronous operations further comprises creating a shadow account on every node of the distributed computing cluster.

3. The method of claim 1 , further comprising:

producing ingest job creation task start tracking messages after the initiating a series of asynchronous operations for the creation of ingest job agent;

creating an in-memory thread for each tracking message;

setting a thread execution time limit;

checking a task status for completion in the in-memory thread; and

producing an asynchronous computing cluster command module task completion message.

4. The method of claim 3 , further comprising:

adding a security layer entry to grant an access control authorization of the ingest job folder to the operator; and

writing to database completion state for the ingest job creation.

5. A non-transitory computer readable medium in a computing device that has a processor, the computing device coupled to a distributed computing cluster, the non-transitory computer readable medium having instructions for creating an ingest job agent for secure multitenant operations of a distributed computing cluster, the instructions causing the processor to perform operations comprising:

receiving a network service call from a client computer containing credentials of an operator, in which the client computer is located in a network security domain coupled to the distributed computing cluster;

initiating the network service call to a centralized directory server, located in the network security domain, to authenticate the operator using the transmitted credentials;

presenting a user interface to collect a plurality of ingest job agent configuration options from the operator;

storing the ingest job agent configuration options to a data store in the centralized directory server;

initiating a series of asynchronous operations to create the ingest job agent; and

updating a metadata store in the distributed computing cluster to indicate to the operator that the creation of the ingest job agent is completed, wherein the series of asynchronous operations comprises:

creating an ingest source folder in an ingest server where files shall be transferred from;

creating an ingest sink folder on a distributed file system where files are transferred to;

creating an access control list entry which grants and limits access of the ingest source folder and the ingest sink folder to the operator of the ingest job agent and other operators specified by the operator; and

initiating an in-memory process which monitors and moves files an authorized operator deposits inside the ingest source folder into the ingest sink folder via network file transfer.

6. The non-transitory computer readable medium of claim 5 , the series of asynchronous operations further comprises creating a shadow account on every node of the distributed computing cluster.

7. The non-transitory computer readable medium of claim 5 , the operations further comprising:

producing ingest job creation task start tracking messages after the initiating a series of asynchronous operations for the creation of ingest job agent;

creating an in-memory thread for each tracking message;

setting a thread execution time limit;

checking a task status for completion in the in-memory thread; and

producing an asynchronous computing cluster command module task completion message.

8. The non-transitory computer readable medium of claim 7 , the operations further comprising:

adding a security layer entry to grant an access control authorization of the ingest job folder to the operator; and

writing to database completion state for the ingest job creation.

9. The method of claim 1 , further comprising collecting ingest job metadata and storing the ingest job metadata in the data store in the centralized directory server.

10. The non-transitory computer readable medium of claim 5 , wherein the operations further comprise collecting ingest job metadata and storing the ingest job metadata in the data store in the centralized directory server.