Communication architecture for distributed computing environment
A communication architecture for performing work in a distributed computing environment involves client processes on client nodes sending work requests to a director. The director examines the work requests to determine one or more resources required to perform the work as well as one or more server nodes that can provide the required resources. The director then forwards the work requests to server processes on the server nodes that can provide the required resources. The work requests provided to the server processes specify that the work results are to be provided directly from the server processes to the client processes that made the work requests. The director maintains a table of resource data that specifies the services and/or resources that are provided by and/or made available on the server nodes in the distributed computing environment.
1. A method for processing data on a distributed computing system that includes a plurality of nodes, the method comprising:
maintaining mapping data that specifies work that can be performed by the plurality of nodes;
in response to receiving, from a first process on a first node from the plurality of nodes, a first work request to perform first work, determining based upon the first work and the mapping data, that a first portion of the first work is to be performed on a second node from the plurality of nodes and that a second portion of the first work is to be performed on a third node from the plurality of nodes;
providing a second work request to a second process on the second node, wherein the second work request specifies that the first portion of the first work is to be performed and that results of performing the first portion of the first work are to be provided directly to the first process; and
providing a third work request to a third process on the third node, wherein the third work request specifies that the second portion of the first work is to be performed and that results of performing the second portion of the work are to be provided directly to the first process.
2. The method as recited in claim 1 , further comprising
in response to receiving from the first process a second request to perform second work, determining based upon the second work and the mapping data, that the second work is to be performed on the third node from the plurality of nodes, and
providing the second request to the third process on the third node, wherein the second request specifies that the first process is to receive results of the second work directly from the third process.
3. The method as recited in claim 1 , further comprising
in response to receiving, from the third process on the third node, second request to perform second work, determining based upon the second work and the mapping data, that the second work is to be performed on the second node, and
providing the second request to the second process, wherein the second request specifies that the third process is to receive results of the second work directly from the second process.
4. The method as recited in claim 1 , further comprising
in response to receiving a second request to perform second work from the third process on the third node from the plurality of nodes, determining based upon the second work and the mapping data, a fourth node from the plurality of nodes on which the second work is to be performed, and
providing the second request to a fourth process on the fourth node, wherein the second request specifies that the third process is to receive results of the second work directly from the fourth process.
5. The method as recited in claim 1 , wherein the step of determining that the first portion of the first work is to be performed on the second node includes
determining one or more resources required to perform the first portion of the portion of the first work on the one or more resources.
6. The method as recited in claim 1 , wherein:
determining that the first portion of the first work is to be performed on a second node from the plurality of nodes includes the step of a director determining that the first portion of the first work is to be performed on a second node from the plurality of nodes, and
providing the second work request to a second process on the second node includes the director providing the second work request to a second process on the second node.
7. The method as recited in claim 1 , further comprising upon completion of the first portion of the first work, the second process providing the results of performing the portion of the first work directly to the first process.
8. The method as recited in claim 1 , wherein the second work request is a remote procedure call.
9. A distributed computing system for performing work, the distributed computing system comprising:
a plurality of nodes; and
a director communicatively coupled to the plurality of nodes, the director being configured to
maintain mapping data that specifies work that can be performed by the plurality of nodes,
in response to receipt, from a first process on a first node from the plurality of nodes, of a first work request to perform first work, derermine based upon the first work and the mapping data, that a first portion of the first work is to be performed on a second node from the plurality of nodes and that a second portion of the first work is to be performed on a third node from the plurality of nodes and
provide a second work request to a second process on the second node, wherein the second request specifies that the first portion of the first work is to be performed and that results of performing the first portion of the first work are to be provided directly to the first process; and
provide a third work request to a third process on the third node, wherein the third work request specifies that the second portion of the first work is to be performed and that results of performing the second portion of the first work are to be provided derectly to the first process.
10. The distrbuted computing system as recited in claim 9 , wherein the director is further configured to provide the second work request to the second process.
11. The distributed computing system as recited in claim 9 , further comprising resource data that specifies the access rights of the plurality of nodes relative to resources.
12. A computer-readable medium carrying instructions for processing data on a distributed computing system that includes a plurality of nodes, wherein processing of the instructions by one or more processors, cause:
maintaining mapping data that specifies work that can be performed by the plurality of nodes;
in response to receiving, from a first process on a first node from the plurality of nodes, a first work request to perform first work, determining based upon the first work and the mapping data, that a first portion of the first work is to be performed on a second node from the plurality of nodes and that a second portion of the first work is to be performed on a third node from the plurality of nodes;
providing a second work request to a second process on the second node, wherein the second work request specifies that the first portion of the first work is to be performed and that results of performing the first portion of the first work are to be provided directly to the first process;
providing a third work request to a third process on the third node, wherein the third work request specifies that the second portion of the first work is to be performed and that results of performing the second portion of the first work are to be provided directly from the third node to the first process.
13. The computer-readable medium as recited in claim 12 , further comprising additional instructions which, when executed by the one or more processors, cause
in response to receiving from the first process a second request to perform second work, determining based upon the second work and the mapping data, that the second work is to be performed on the third node from the plurality of nodes, and
providing the second request to the third process on the third node, wherein the second request specifies that the first process is to receive results of the second work directly from the third process.
14. The computer-readable medium as recited in claim 12 , further comprising additional instructions which, when executed by the one or more processors, cause
in response to receiving, from a third process on a third node from the plurality of nodes, a second request to perform second work, determining based upon the second work and the mapping data, that the second work is to be performed on the second node, and
providing the second request to the second process, wherein the second request specifies that the third process is to receive results of the second work directly from the second process.
15. The computer-readable medium as recited in claim 12 , further comprising additional instructions which, when executed by the one or more processors, cause
in response to receiving a second request to perform second work from a third process on a third node from the plurality of nodes, determining based upon the second work and the mapping data, a fourth node from the plurality of nodes on which the second work is to be performed, and
providing the second request to a fourth process on the fourth node, wherein the second request specifies that the third process is to receive results of the second work directly from the fourth process.
16. The computer-readable medium as recited in claim 12 , wherein determining that the first portion of the first work is to be performed on a second node includes
determining one or more resources required to perform the first portion of the first work, and
determining which of the plurality of nodes is allowed to perform the first portion of the first work on the one or more resources.
17. The computer-readable medium as recited in claim 12 , wherein:
determining that the first portion of the first work is to be performed on a second node from the plurality of nodes includes the step of a director determining that the first portion of the first work is to be performed on a second node from the plurality of nodes, and
providing the first work request to a second process on the second node includes the step of the director providing the first work request to a second process on the second node.
18. The computer-readable medium as recited in claim 12 , further comprising additional instructions which, when executed by the one or more processors, cause upon completion of the first portion of the first work, the second process providing the results of performing the first portion of the first work directly to the first process.
19. The computer-readable medium as recited in claim 12 , wherein the second work request is a remote procedure call.
20. A method for processing data on a distributed computing system that includes a plurality of nodes, the method comprising:
maintaining mapping data that specifies work that can be performed by the plurality of nodes; and
in response to receiving a first work request to perform first work from a first process on a first node from the plurality of nodes,
determining based upon the first work and the mapping data, that the first work is to be performed on a second node from the plurality of nodes,
generating an updated first work request that specifies that the first process is to directly receive results of performing the first work, and
providing the updated first work request to a second process on the second node.
21. An apparatus for processing data on a distributed computing system, the apparatus comprising a memory carrying instructions which, when executed by one or more processors, cause:
maintaining mapping data that specifies work that can be performed by the plurality of nodes; and
in response to receiving a first work request to perform first work from a first process on a first node from the plurality of nodes,
determining based upon the first work and the mapping data, that the first work is to be performed on a second node from the plurality of nodes,
generating an updated first work request that specifies that the first process is to directly receive results of performing the first work, and
providing the updated first work request to a second process on the second node.
22. A computer-readable medium for processing data on a distributed computing system, the computer readable medium carrying instructions which, when executed by one or more processors, cause:
maintaining mapping data that specifies work that can be performed by the plurality of nodes; and
in response to receiving a first work request to perform first work from a first process on a first node from the plurality of nodes,
determining based upon the first work and the mapping data, that the first work is to be performed on a second node from the plurality of nodes,
generating an updated first work request that specifies that the first process is to directly receive results of performing the first work, and
providing the updated first work request to a second process on the second node.