IP Library › Granted Patent US 12688156
Granted Patent B2
US 12688156 · App. 17/483,984 · Granted Jul 21, 2026

Garbage collection of redundant partitions

Inventors: Srikumar Venugopal (Dublin, IE); Stefano Braghin (Dublin, IE)
Assignee: International Business Machines Corporation
G06F16/1737
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12688156
App. No.
17/483,984
Granted
Jul 21, 2026
Kind
B2
Abstract

A method, system, and computer program product for garbage collection of redundant partitions in distributed data management systems are provided. The method stores data across a set of nodes with the data being stored using one or more partitions and the data and the one or more partitions are replicated across the set of nodes. A first partition is determined to be stale at a first node of the set of nodes. The first partition is marked for deletion locally at the first node. A set of deletion votes are determined for the first partition with each node being associated with a deletion vote. The method determines a deletion decision for the first partition on the first node based on the set of deletion votes.

Claims (65)

1 . A computer-implemented method for garbage collection of redundant partitions in distributed data management systems, the method comprising:

storing data across a set of nodes in the distributed data management systems, the data being stored using one or more partitions and the data and the one or more partitions being replicated across the set of nodes;

determining, at a first node of the set of nodes, a first partition is stale by

identifying a set of access metrics for the first partition, wherein the first partition includes data stored at the first node of the set of nodes,

determining a freshness value for the first partition based on the set of access metrics, wherein the freshness value is determined by computing a function of usage statistics of the first partition, conditions of the first partition, or attributes of the first partition, wherein the freshness value selected from a group consisting of an exponential decay of access counts, access counts over time, access counts since a last access of the first partition, access counts of the first partition, and replicas of the first partition on other nodes, and

comparing the freshness value for the first partition with a freshness threshold, wherein the freshness threshold is dynamically generated based on the usage statistics;

marking the first partition for deletion, the first partition being marked for deletion locally at the first node;

determining a set of deletion votes for the first partition, each node of the set of nodes being associated with a deletion vote of the set of deletion votes,

wherein the determining the set of deletion votes further comprises:

transmitting a deletion indicator from the first node to a remaining subset of nodes of the set of nodes; and

receiving the set of deletion votes from the remaining subset of nodes, each node of the remaining subset of nodes comparing the freshness value for the first partition to a freshness value for a local partition associated with the first partition; and

determining a deletion decision for the first partition on the first node based on the set of deletion votes.

2 . The method of claim 1 , wherein the determining the deletion decision further comprises:

determining a threshold number of nodes of the set of nodes are associated with a positive deletion vote; and

deleting the first partition from the first node.

3 . The method of claim 1 , wherein the determining the deletion decision further comprises:

determining a number of nodes associated with a positive deletion vote is below a deletion threshold; and

reverting a deletion marking for the first partition on the first node.

4 . The method of claim 1 , further comprising:

analyzing, by each node of the set of nodes, the one or more partitions stored locally on the first node to determine staleness of the one or more partitions.

5 . The method of claim 1 , wherein the set of access metrics includes requests or queries for the first partition, requests or queries for the first partition within a specified time period, hits per time period, the usage statistics, access counts, last access time, query alteration, replication of data of the first partition to other nodes, and number of nodes to which the first partition is cached.

6 . A system for garbage collection of redundant partitions in distributed data management systems, the system comprising:

one or more processors; and

a computer-readable storage medium, coupled to the one or more processors, storing program instructions that, when executed by the one or more processors, cause the one or more processors to perform operations comprising:

storing data across a set of nodes in the distributed data management systems, the data being stored using one or more partitions and the data and the one or more partitions being replicated across the set of nodes;

determining, at a first node of the set of nodes, a first partition is stale by

identifying a set of access metrics for the first partition, wherein the first partition includes data stored at the first node of the set of nodes,

determining a freshness value for the first partition based on the set of access metrics, wherein the freshness value is determined by computing a function of usage statistics of the first partition, conditions of the first partition, or attributes of the first partition, wherein the freshness value is selected from a group consisting of an exponential decay of access counts, access counts over time, access counts since a last access of the first partition, access counts of the first partition, and replicas of the first partition on other nodes, and

comparing the freshness value for the first partition with a freshness threshold, wherein the freshness threshold is dynamically generated based on the usage statistics;

marking the first partition for deletion, the first partition being marked for deletion locally at the first node;

determining a set of deletion votes for the first partition, each node of the set of nodes being associated with a deletion vote of the set of deletion votes,

wherein the determining the set of deletion votes further comprises:

transmitting a deletion indicator from the first node to a remaining subset of nodes of the set of nodes; and

receiving the set of deletion votes from the remaining subset of nodes, each node of the remaining subset of nodes comparing the freshness value for the first partition to a freshness value for a local partition associated with the first partition; and

determining a deletion decision for the first partition on the first node based on the set of deletion votes.

7 . The system of claim 6 , wherein the determining the deletion decision further comprises:

determining a threshold number of nodes of the set of nodes are associated with a positive deletion vote; and

deleting the first partition from the first node.

8 . The system of claim 6 , wherein the determining the deletion decision further comprises:

determining a number of nodes associated with a positive deletion vote is below a deletion threshold; and

reverting a deletion marking for the first partition on the first node.

9 . The system of claim 6 , wherein the operations further comprise:

analyzing, by each node of the set of nodes, the one or more partitions stored locally on the first node to determine staleness of the one or more partitions.

10 . The system of claim 6 , wherein the set of access metrics includes requests or queries for the first partition, requests or queries for the first partition within a specified time period, hits per time period, the usage statistics, access counts, last access time, query alteration, replication of data of the first partition to other nodes, and number of nodes to which the first partition is cached.

11 . A computer program product for garbage collection of redundant partitions in distributed data management systems comprising a computer readable storage medium having program instructions embodied therewith, the program instructions being executable by one or more processors to cause the one or more processors to perform operations comprising:

storing data across a set of nodes in the distributed data management systems, the data being stored using one or more partitions and the data and the one or more partitions being replicated across the set of nodes;

determining, at a first node of the set of nodes, a first partition is stale by

identifying a set of access metrics for the first partition, wherein the first partition includes data stored at the first node of the set of nodes,

determining a freshness value for the first partition based on the set of access metrics, wherein the freshness value is determined by computing a function of usage statistics of the first partition, conditions of the first partition, or attributes of the first partition, wherein the freshness value is selected from a group consisting of an exponential decay of access counts, access counts over time, access counts since a last access of the first partition, access counts of the first partition, and replicas of the first partition on other nodes, and

comparing the freshness value for the first partition with a freshness threshold, wherein the freshness threshold is dynamically generated based on the usage statistics;

marking the first partition for deletion, the first partition being marked for deletion locally at the first node;

determining a set of deletion votes for the first partition, each node of the set of nodes being associated with a deletion vote of the set of deletion votes,

wherein the determining the set of deletion votes further comprises:

transmitting a deletion indicator from the first node to a remaining subset of nodes of the set of nodes; and

receiving the set of deletion votes from the remaining subset of nodes, each node of the remaining subset of nodes comparing the freshness value for the first partition to a freshness value for a local partition associated with the first partition; and

determining a deletion decision for the first partition on the first node based on the set of deletion votes.

12 . The computer program product of claim 11 , wherein the determining the deletion decision comprises:

determining a threshold number of nodes of the set of nodes are associated with a positive deletion vote; and

deleting the first partition from the first node.

13 . The computer program product of claim 11 , wherein the determining the deletion decision comprises:

determining a number of nodes associated with a positive deletion vote is below a deletion threshold; and

reverting a deletion marking for the first partition on the first node.

14 . The computer program product of claim 11 , wherein the operations further comprise:

analyzing, by each node of the set of nodes, the one or more partitions stored locally on the first node to determine staleness of the one or more partitions.

15 . The computer program product of claim 11 , wherein the set of access metrics includes requests or queries for the first partition, requests or queries for the first partition within a specified time period, hits per time period, the usage statistics, access counts, last access time, query alteration, replication of data of the first partition to other nodes, and number of nodes to which the first partition is cached.