IP Library Granted Patent US 11,487,735
Granted Patent B2
US 11,487,735 · App. 17/318,725 · Granted Nov 1, 2022

Combinators

Inventors: Rich Skrenta (San Carlos, CA); Bryn Robert Dole (Sunnyvale, CA); Michael Markson (San Francisco, CA); Keith Peters (San Francisco, CA); Robert Michael Saliba (San Francisco, CA); Robert N. Truel (San Carlos, CA); Gregory B. Lindahl (Sunnyvale, CA)
Assignee: International Business Machines Corporation
G06F16/2282G06F16/2228G06F16/2462G06F16/24554G06F16/273G06F16/285G06F16/951
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,487,735
App. No.
17/318,725
Granted
Nov 1, 2022
Kind
B2
Abstract

A method, according to one embodiment, includes identifying data to be stored in one or more tables within a predetermined portion of a partitioned storage in one of a plurality of nodes, the predetermined portion having at least one replica, and where no two identical replicas reside on a single node; assigning an identifier and a data storage hierarchical level to the data; mapping the data to an index and storing the data in accordance with the index and the data storage hierarchical level, the storing including writing the data to a row in one of the one or more tables on the predetermined portion and recording a write operation into a transaction log of the node; receiving a plurality of write operations; and combining a plurality of write tasks of the predetermined portion for a predetermined time period.

Claims (45)

1. A computer-implemented method, comprising:

identifying data to be stored in one or more tables within a predetermined portion of a partitioned storage in one of a plurality of nodes, the predetermined portion having at least one replica, and where no two identical replicas reside on a single node;

assigning an identifier and a data storage hierarchical level to the data;

mapping the data to an index and storing the data in accordance with the index and the data storage hierarchical level, the storing including writing the data to a row in one of the one or more tables on the predetermined portion and recording a write operation into a transaction log of the node;

receiving a plurality of write operations; and

combining a plurality of write tasks of the predetermined portion for a predetermined time period.

2. The computer-implemented method of claim 1 , comprising delaying one or more of the plurality of write operations to collect additional write operations designated to the predetermined portion.

3. The computer-implemented method of claim 1 , wherein the predetermined time period includes a time period needed to collect a total amount of write operations that meet a predetermined file size.

4. The computer-implemented method of claim 1 , comprising:

designating a merging limit value to the one or more tables on the predetermined portion, wherein the merging limit value is indicative of one of a plurality of data storage hierarchical levels; and

limiting a merging of files in accordance with the merging limit value by refraining from merging data files in the one of the plurality of data storage hierarchical levels once the merging limit value is met.

5. The computer-implemented method of claim 1 , comprising:

storing a bloom filter for a plurality of row keys for each of a plurality of data storage hierarchical levels; and

consulting the bloom filter to determine whether one of the plurality of row keys is found in a given data storage hierarchical level of the plurality of data storage hierarchical levels.

6. The computer-implemented method of claim 1 , comprising metering the combined plurality of write tasks.

7. The computer-implemented method of claim 1 , wherein the data storage hierarchical level includes a first hierarchical level indicative of oldest data in the one or more tables on the predetermined portion.

8. The computer-implemented method of claim 7 , wherein the data storage hierarchical level includes a second hierarchical level indicative of newer stored data in the one or more tables on the predetermined portion.

9. The computer-implemented method of claim 8 , wherein the data storage hierarchical level includes a third hierarchical level indicative of most recently stored data in the one or more tables on the predetermined portion.

10. The computer-implemented method of claim 9 , wherein the plurality of write tasks are written to the third hierarchical level, data stored in the third hierarchical level is merged to the second hierarchical level after a first predetermined time period, and the data stored in the second hierarchical level is merged to the first hierarchical level after a second predetermined time period.

11. A computer program product for organizing data in a database system, the computer program product comprising a computer readable storage medium having program instructions embodied therewith, wherein the computer readable storage medium is not a transitory signal per se, the program instructions executable by a processor to cause the processor to perform a method comprising:

identifying, utilizing the processor, data to be stored in one or more tables within a predetermined portion of a partitioned storage in one of a plurality of nodes, the predetermined portion having at least one replica, and where no two identical replicas reside on a single node;

assigning, utilizing the processor, an identifier and a data storage hierarchical level to the data;

mapping, utilizing the processor, the data to an index and storing the data in accordance with the index and the data storage hierarchical level, the storing including writing the data to a row in one of the one or more tables on the predetermined portion and recording a write operation into a transaction log of the node;

receiving, utilizing the processor, a plurality of write operations; and

combining, utilizing the processor, a plurality of write tasks of the predetermined portion for a predetermined time period.

12. The computer program product of claim 11 , comprising delaying one or more of the plurality of write operations to collect additional write operations designated to the predetermined portion.

13. The computer program product of claim 11 , wherein the predetermined time period includes a time period needed to collect a total amount of write operations that meet a predetermined file size.

14. The computer program product of claim 11 , comprising:

designating a merging limit value to the one or more tables on the predetermined portion, wherein the merging limit value is indicative of one of a plurality of data storage hierarchical levels; and

limiting a merging of files in accordance with the merging limit value by refraining from merging data files in the one of the plurality of data storage hierarchical levels once the merging limit value is met.

15. The computer program product of claim 11 , comprising:

storing a bloom filter for a plurality of row keys for each of a plurality of data storage hierarchical levels; and

consulting the bloom filter to determine whether one of the plurality of row keys is found in a given data storage hierarchical level of the plurality of data storage hierarchical levels.

16. The computer program product of claim 11 , comprising metering the combined plurality of write tasks.

17. The computer program product of claim 11 , wherein the data storage hierarchical level includes a first hierarchical level indicative of oldest data in the one or more tables on the predetermined portion.

18. The computer program product of claim 17 , wherein the data storage hierarchical level includes a second hierarchical level indicative of newer stored data in the one or more tables on the predetermined portion.

19. The computer program product of claim 18 , wherein the data storage hierarchical level includes a third hierarchical level indicative of most recently stored data in the one or more tables on the predetermined portion.

20. A system, comprising:

a processor; and

logic integrated with the processor, executable by the processor, or integrated with and executable by the processor, the logic being configured to:

identify data to be stored in one or more tables within a predetermined portion of a partitioned storage in one of a plurality of nodes, the predetermined portion having at least one replica, and where no two identical replicas reside on a single node;

assign an identifier and a data storage hierarchical level to the data;

map the data to an index and storing the data in accordance with the index and the data storage hierarchical level, the storing including writing the data to a row in one of the one or more tables on the predetermined portion and recording a write operation into a transaction log of the node;

receive a plurality of write operations; and

combine a plurality of write tasks of the predetermined portion for a predetermined time period.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded May 24, 2021
From: SKRENTA, RICH; DOLE, BRYN ROBERT; MARKSON, MICHAEL; PETERS, KEITH; SALIBA, ROBERT MICHAEL; TRUEL, ROBERT N.; LINDAHL, GREGORY B.
To: INTERNATIONAL BUSINESS MACHINES CORPORATION
Reel/Frame 056333/0370 →
Continuity (6)
Continuation 16127059 · Sep 10, 2018
Continuation 15063376 · Mar 7, 2016
Continuation 13328464 · Dec 16, 2011
Continuation PCTUS2010039395 · Jun 21, 2010
Provisional Application 61218889 · Jun 19, 2009
Related Publication 20210279224A1 · Sep 9, 2021