Coordinated garbage collection in distributed systems
Fast modern interconnects may be exploited to control when garbage collection is performed on the nodes (e.g., virtual machines, such as JVMs) of a distributed system in which the individual processes communicate with each other and in which the heap memory is not shared. A garbage collection coordination mechanism (a coordinator implemented by a dedicated process on a single node or distributed across the nodes) may obtain or receive state information from each of the nodes and apply one of multiple supported garbage collection coordination policies to reduce the impact of garbage collection pauses, dependent on that information. For example, if the information indicates that a node is about to collect, the coordinator may trigger a collection on all of the other nodes (e.g., synchronizing collection pauses for batch-mode applications where throughput is important) or may steer requests to other nodes (e.g., for interactive applications where request latencies are important).
1. A system, comprising:
a plurality of computing nodes interconnected via a network, each comprising at least one processor and one or more heap memories and hosting one or more virtual machine instances, wherein each of the virtual machine instances executes a respective process of a distributed application that communicates over the network with one or more other processes of the distributed application executing on respective other virtual machine instances, and wherein a node of the plurality of computing nodes is configured to:
request, from a garbage collection coordinator, to perform a garbage collection on the respective one or more heap memories, and responsive to receiving a reply, from the garbage collection coordinator, granting the performing of the garbage collection:
stop execution of the respective one or more virtual machine instances hosted by the node; and
perform the garbage collection on the respective one or more heap memories; and
the garbage collection coordinator, configured to:
receive the request from the node to perform the garbage collection on the respective one or more heap memories;
determine, responsive to receiving the request, that a number of granted garbage collections is below a number of garbage collections allowed to be performed at a same time, and responsive to the determining:
send a reply to the node granting the performing of the garbage collection; and
cause work directed to the node to be steered to one or more other nodes of the plurality of computing nodes.
2. The system of claim 1 ,
wherein the garbage collection coordinator comprises a pool of zero or more tokens for granting garbage collections;
wherein to determine that the number of granted garbage collections is below a number of garbage collections allowed to be performed at the same time, the garbage collection coordinator is configured to determine that at least one token for granting garbage collections exists in the pool; and
wherein to send the reply to the node granting the performing of the garbage collection, the garbage collection coordinator is configured to allocate a token from the pool and send the allocated token to the node.
3. The system of claim 2 , wherein the garbage collection coordinator enforces an upper bound on the number of computing nodes allowed to perform garbage collections at the same time.
4. The system of claim 2 , wherein the node of the plurality of computing nodes is further configured to return the token to the garbage collection coordinator responsive to completion of the garbage collection on the respective one or more heap memories, and wherein the garbage collection coordinator is further configured to return the token to the pool responsive to receiving the token from the node.
5. The system of claim 4 , wherein to determine that at least one token for granting garbage collections exists in the pool, the garbage collection coordinator is configured to:
wait for a token to be returned to the pool responsive to determining that no tokens for granting garbage collections exist in the pool.
6. The system of claim 1 , wherein the garbage collection coordinator comprises a single garbage collection coordinator component on one of the plurality of computing nodes.
7. The system of claim 1 ,
wherein the distributed application is an application that was written in a garbage collected programming language; and
wherein the request to perform a garbage collection is based, at least in part, on determining that a garbage collection should be performed on the node and that execution of the distributed application on the node should be paused or stopped while the garbage collection is performed.
8. A method, comprising:
sending a request, by a computing node to a garbage collection coordinator, to perform a garbage collection on one or more heap memories, wherein the computing node is one of a plurality of computing nodes, each comprising at least one processor and one or more heap memories and hosting one or more virtual machine instances, wherein each of the virtual machine instances executes a respective process of a distributed application that communicates over a network with one or more other processes of the distributed application executing on respective other virtual machine instances;
determining, by the garbage collection coordinator responsive to receiving the request, that a number of granted garbage collections is below a number allowed to be performed at a same time, and responsive to the determining:
sending a reply to the computing node granting the performing of the garbage collection; and
causing work directed to the computing node to be steered to one or more other computing nodes of the plurality of computing nodes; and
responsive to receiving a reply granting the performing of the garbage collection:
stopping execution of the respective one or more virtual machine instances hosted by the computing node; and
performing the garbage collection on the one or more heap memories by the computing node.
9. The method of claim 8 ,
wherein the garbage collection coordinator comprises a pool of zero or more tokens for granting garbage collections;
wherein determining that the number of granted garbage collections is below the number allowed to perform garbage collections at the same time comprises determining that at least one token for granting garbage collections exists in the pool; and
wherein sending the reply to the computing node granting the performing of the garbage collection comprises allocating a token from the pool and send the allocated token to the computing node.
10. The method of claim 9 , wherein the garbage collection coordinator enforces an upper bound on the number of computing nodes allowed to perform garbage collections at the same time.
11. The method of claim 8 , further comprising:
returning, by the computing node, the token to the garbage collection coordinator responsive to completion of the garbage collection on the one or more heap memories; and
returning, by the garbage collection coordinator, the token to the pool responsive to receiving the token from the computing node.
12. The method of claim 8 , wherein the determining that at least one token for granting garbage collections exists in the pool comprises waiting for a token to be returned to the pool responsive to determining that no tokens for granting garbage collections exist in the pool.
13. The method of claim 8 , wherein the garbage collection coordinator comprises a single garbage collection coordinator component on one of the plurality of computing nodes.
14. The method of claim 8 ,
wherein the distributed application is an application that was written in a garbage collected programming language; and
wherein the request to perform a garbage collection is based, at least in part, on determining that a garbage collection should be performed on the computing node and that execution of the distributed application on the computing node should be paused or stopped while the garbage collection is performed.
15. One or more non-transitory computer-readable storage media storing program instructions that when executed on or across one or more computing nodes cause the one or more computing nodes to implement a garbage collection coordinator to perform:
receiving a request, from a computing node, to grant a garbage collection on one or more heap memories, wherein the computing node is one of a plurality of computing nodes, each comprising at least one processor and one or more heap memories and hosting one or more virtual machine instances, wherein each of the virtual machine instances executes a respective process of a distributed application that communicates over a network with one or more other processes of the distributed application executing on respective other virtual machine instances;
determining, responsive to receiving the request, that a number of granted garbage collections is below a number allowed to be performed at a same time, and responsive to the determining:
sending a reply to the computing node granting the performing of the garbage collection; and
causing work directed to the computing node to be steered to one or more other computing nodes of the plurality of computing nodes.
16. The one or more non-transitory computer-readable storage media of claim 15 ,
wherein the garbage collection coordinator comprises a pool of zero or more tokens for granting garbage collections;
wherein determining that the number of granted garbage collections is below the number allowed to perform garbage collections at the same time comprises determining that at least one token for granting garbage collections exists in the pool; and
wherein sending the reply to the computing node granting the performing of the garbage collection comprises allocating a token from the pool and send the allocated token to the computing node.
17. The one or more non-transitory computer-readable storage media of claim 15 , wherein the garbage collection coordinator enforces an upper bound on the number of computing nodes allowed to perform garbage collections at the same time.
18. The one or more non-transitory computer-readable storage media of claim 15 , wherein the garbage collection coordinator further performs:
receiving, from the computing node, the token to the garbage collection coordinator responsive to completion of the garbage collection on the one or more heap memories; and
returning the token to the pool responsive to receiving the token from the computing node.
19. The one or more non-transitory computer-readable storage media of claim 15 , wherein the determining that at least one token for granting garbage collections exists in the pool comprises waiting for a token to be returned to the pool responsive to determining that no tokens for granting garbage collections exist in the pool.
20. The one or more non-transitory computer-readable storage media of claim 15 ,
wherein the distributed application is an application that was written in a garbage collected programming language; and
wherein the request to perform a garbage collection is based, at least in part, on determining that a garbage collection should be performed on the computing node and that execution of the distributed application on the computing node should be paused or stopped while the garbage collection is performed.