Lazy lock queue reduction for cluster group changes
Described herein are technologies directed to lazy lock queue reduction in computing clusters. The disclosed lazy lock queue reduction techniques can be performed in preparation for cluster group changes. Prior to a cluster group change operation, such as a merge or a split of a node with a group, a notification of a planned group change operation can be sent to the nodes of a group. In response to the notification, the nodes of the group can perform lazy lock queue reduction techniques disclosed herein. In one disclosed lazy lock queue reduction technique, a node can set a drain goal for a lazy lock queue, and the node can drain the lazy lock queue according to the drain goal. In another disclosed lazy lock queue reduction technique, a node can set an age limit for lazy lock queue entries, and the node can remove lazy lock queue entries which are expired or over the age limit.
1. A method, comprising:
receiving, by a node device of a group of node devices, a notification of a planned group change operation, wherein, in the planned group change operation, the group of node devices is to change by increasing or decreasing a number of node devices in the group of node devices;
in response to receiving the notification of the planned group change operation, reducing, by the node device, a drain goal at the node device, wherein the drain goal defines a size of a cached lazy lock queue at the node device, and wherein the cached lazy lock queue comprises a cached queue of locks held for the node device; and
in response to receiving the notification of the planned group change operation, comparing, by the node device, ages of entries in the cached lazy lock queue to an age limit in order to identify and delete at least one entry that is older than the age limit from the cached lazy lock queue.
2. The method of claim 1 , wherein the group of node devices is within a cluster of node devices.
3. The method of claim 1 , wherein the planned group change operation comprises a merge operation to add an additional node device to the group of node devices, or a split operation to split an existing node device from the group of node devices.
4. The method of claim 1 , further comprising deleting, by the node device, at least one entry from the cached lazy lock queue in order to achieve the drain goal at the node device.
5. The method of claim 4 , wherein the deleting the at least one entry from the cached lazy lock queue comprises notifying a lock coordinator device within the group of node devices that a lock associated with the at least one entry is released.
6. The method of claim 4 , further comprising raising, by the node device, a block flag to block the planned group change operation during the deleting the at least one entry from the cached lazy lock queue, and dropping, by the node device, the block flag after the deleting the at least one entry from the cached lazy lock queue.
7. The method of claim 1 , wherein the age limit is between one minute and one hundred twenty minutes.
8. A node device in a group of node devices, comprising:
at least one processor; and
at least one memory that stores executable instructions that, when executed by the at least one processor, facilitate performance of operations, comprising:
maintaining a cached lazy lock queue comprising a cached queue of locks held for the node device;
comparing an age of a first entry in the cached lazy lock queue to an age limit;
in response to the age of the first entry being determined to be older than the age limit, deleting the first entry from the cached lazy lock queue and comparing an age of a next entry in the cached lazy lock queue to the age limit; and
in response to the age of the next entry being younger than the age limit, allowing the next entry to remain in the cached lazy lock queue.
9. The device of claim 8 , wherein the age limit is between one minute and one hundred twenty minutes.
10. The device of claim 8 , wherein the operations further comprise releasing a lock by a thread at the node device, and wherein, in response to releasing the lock, the thread at the node device performs at least the comparing the age of the first entry in the cached lazy lock queue to the age limit and the deleting the first entry in the cached lazy lock queue from the cached lazy lock queue.
11. The device of claim 8 , wherein the deleting the first entry from the cached lazy lock queue comprises notifying a lock coordinator device within the group of node devices that a lock associated with the first entry is released.
12. The device of claim 8 , wherein the operations further comprise establishing or reducing the age limit.
13. The device of claim 12 , wherein the operations further comprise receiving a notification of a planned group change operation, and wherein establishing or reducing the age limit is in response to the notification of the planned group change operation.
14. The device of claim 8 , wherein the maintaining the cached lazy lock queue comprises inserting respective entries in the cached lazy lock queue, each respective entry comprising a respective lock held for the node device and respective time information to indicate a time at which the respective entry is inserted in the cached lazy lock queue.
15. A machine-readable storage medium, comprising executable instructions that, when executed by a processor of a first node device of a group of node devices, facilitate performance of operations, comprising:
selecting a second node device of the group of node devices;
requesting the second node device to serve as a lock coordinator for a lock placed on a resource by the first node device;
placing a first entry in a cached lazy lock queue at the first node device, wherein the first entry identifies the lock placed on the resource;
receiving a notification of a planned group operation, wherein the planned group operation comprises reselection of the lock coordinator by the first node device;
in response to the receiving the notification of the planned group operation, setting a drain goal at the first node device, wherein the drain goal defines a size of the cached lazy lock queue; and
deleting at least one second entry from the cached lazy lock queue in order to achieve the drain goal at the first node device.
16. The machine-readable storage medium of claim 15 , wherein the planned group operation comprises merging a node device with the group of node devices.
17. The machine-readable storage medium of claim 15 , wherein the planned group operation comprises splitting a node device with the group of node devices.
18. The machine-readable storage medium of claim 15 , wherein the operations further comprise raising a block flag to block the planned group change operation during the deleting the at least one second entry from the cached lazy lock queue.
19. The machine-readable storage medium of claim 18 , wherein the operations further comprise dropping the block flag after the drain goal is achieved.
20. The machine-readable storage medium of claim 15 , wherein the operations further comprise identifying at least one old entry in the cached lazy lock queue that is older than an age limit, and deleting the at least one old entry from the cached lazy lock queue.