IP Library Granted Patent US 11,544,232
Granted Patent B2
US 11,544,232 · App. 16/710,440 · Granted Jan 3, 2023

Efficient transaction log and database processing

Inventor: Maarten Foque (Steendorp, BE)
Assignee: Western Digital Technologies, Inc.
G06F16/1865G06F16/2365
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,544,232
App. No.
16/710,440
Granted
Jan 3, 2023
Kind
B2
Abstract

The described methods, systems, and other aspects can, by way of example, advantageously provide efficient metadata storage in a master-slave node-based metadata store of a distributed storage system. An example method may write a first transaction entry in a transaction log of a first node based on a first request for a first operation associated with a first storage object. The method may then request a second node write the first transaction entry in a transaction log of the second node based on the first request. While awaiting a first acknowledgment of a completion of a writing of the first transaction entry in the transaction log of the second node, the method may proceed to write one or more subsequent transaction entries in the transaction log of the first node based on one or more subsequent requests for one or more operations associated with one or more storage objects.

Claims (73)

1. A computer-implemented method, comprising:

writing a first transaction entry in a transaction log of a first node based on a first request for a first operation associated with a first storage object;

requesting that a second node writes the first transaction entry in a transaction log of the second node based on the first request;

while awaiting a first acknowledgment of a completion of a writing of the first transaction entry in the transaction log of the second node, writing one or more subsequent transaction entries in the transaction log of the first node based on one or more subsequent requests for one or more operations associated with one or more storage objects;

determining a failure to write a certain entry from the one or more subsequent transaction entries in one of the transaction log of the first node and the transaction log of the second node;

responsive to determining the failure, terminating a processing of a certain request from the one or more subsequent requests that is associated with the certain entry; and

providing a failure notification to a client that submitted the certain request associated with the certain entry.

2. The computer-implemented method of claim 1 , further comprising, responsive to receiving the first acknowledgment of the completion of the writing of the first transaction entry in the transaction log of the second node, writing a first metadata associated with the first request in a transaction database of the first node.

3. The computer-implemented method of claim 2 , further comprising receiving the one or more subsequent requests after writing the first metadata associated with the first request in the transaction database of the first node, wherein:

receiving the one or more subsequent requests includes receiving a subsequent request for an operation associated with a second storage object;

writing the one or more subsequent transaction entries includes writing a subsequent transaction entry in the transaction log of the first node based on the subsequent request; and

the computer-implemented method further comprises requesting that the second node writes the subsequent transaction entry in the transaction log of the second node based on the subsequent request, wherein:

the subsequent transaction entry is written in the transaction log of the first node after the first metadata associated with the first request is written in the transaction database of the first node; and

the subsequent transaction entry is written in the transaction log of the second node after the first metadata associated with the first request is written in a transaction database of the second node.

4. The computer-implemented method of claim 2 , further comprising:

requesting that the second node writes a second transaction entry in the transaction log of the second node based on a second request included in the one or more subsequent requests;

receiving a second acknowledgment of a completion of the writing of the second transaction entry in the transaction log of the second node; and

writing a second metadata associated with the second request in the transaction database of the first node.

5. The computer-implemented method of claim 1 , further comprising, responsive to the completion of the writing of the first transaction entry in the transaction log of the second node, writing first metadata associated with the first request in a transaction database of the second node.

6. The computer-implemented method of claim 1 , wherein the first operation comprises one of a read operation, a write operation, and a delete operation associated with the first storage object.

7. The computer-implemented method of claim 1 , further comprising:

while awaiting the first acknowledgment of the completion of the writing of the first transaction entry in the transaction log of the second node, requesting that the second node writes the one or more subsequent transaction entries in the transaction log of the second node based on the one or more subsequent requests.

8. The computer-implemented method of claim 1 , wherein:

the one or more subsequent requests includes a plurality of subsequent requests;

the one or more operations includes a plurality of operations associated with the one or more storage objects; and

writing the one or more subsequent transaction entries in the transaction log of the first node based on the one or more subsequent requests includes, while awaiting the first acknowledgment:

sequentially writing a plurality of subsequent transaction entries in the transaction log of the first node based on the plurality of subsequent requests; and

requesting that the second node sequentially writes the plurality of subsequent transaction entries in the transaction log of the second node.

9. The computer-implemented method of claim 1 , further comprising:

determining the failure to write the certain entry from the one or more subsequent transaction entries is in the transaction log of the second node;

determining a number of remaining nodes processing the certain request from the one or more subsequent requests that is associated with the certain entry; and

determining that the number of remaining nodes does not satisfy a threshold number, wherein terminating the processing of the certain request is further responsive to determining that the number of remaining nodes does not satisfy the threshold number.

10. The computer-implemented method of claim 1 , further comprising:

requesting that the client retransmit the request.

11. A system, comprising:

a cluster of storage nodes including a first node and a second node; and

one or more node controllers associated with the cluster of storage nodes, the one or more node controllers including at least one processor configured to:

write a first transaction entry in a transaction log of the first node based on a first request for a first operation associated with a first storage object;

request the second node writes the first transaction entry in a transaction log of the second node based on the first request;

while awaiting a first acknowledgment of a completion of a writing of the first transaction entry in the transaction log of the second node, write one or more subsequent transaction entries in the transaction log of the first node based on one or more subsequent requests for one or more operations associated with one or more storage objects;

determine a failure to write a certain entry from the one or more subsequent transaction entries in one of the transaction log of the first node and the transaction log of the second node;

responsive to determining the failure, terminate a processing of a certain request from the one or more subsequent requests that is associated with the certain entry; and

provide a failure notification to a client that submitted the certain request associated with the certain entry.

12. The system of claim 11 , wherein the one or more node controllers are further configured to:

responsive to receiving the first acknowledgment of the completion of the writing of the first transaction entry in the transaction log of the second node, write a first metadata associated with the first request in a transaction database of the first node.

13. The system of claim 12 , wherein:

the first request is received via a computer network from a client;

the one or more subsequent requests are received via the computer network from one or more clients, wherein a subsequent request for an operation associated with a second storage object is received after writing the first metadata associated with the first request in the transaction database of the first node;

a subsequent transaction entry is written in the transaction log of the first node based on the subsequent request; and

the one or more node controllers are further configured to request the second node writes the subsequent transaction entry in the transaction log of the second node based on the subsequent request, wherein:

the subsequent transaction entry is written in the transaction log of the first node after the first metadata associated with the first request is written in the transaction database of the first node; and

the subsequent transaction entry is written in the transaction log of the second node after the first metadata associated with the first request is written in a transaction database of the second node.

14. The system of claim 12 , wherein the one or more node controllers are further configured to:

request the second node writes a second transaction entry in the transaction log of the second node based on a second request included in the one or more subsequent requests;

receive a second acknowledgment of a completion of the writing of the second transaction entry in the transaction log of the second node; and

write a second metadata associated with the second request in the transaction database of the first node.

15. The system of claim 11 , wherein the one or more node controllers are further configured to, responsive to the completion of the writing of the first transaction entry in the transaction log of the second node, write a first metadata associated with the first request in a transaction database of the second node.

16. The system of claim 11 , wherein the one or more node controllers are further configured to, while awaiting the first acknowledgment of the completion of the writing of the first transaction entry in the transaction log of the second node, request the second node writes the one or more subsequent transaction entries in the transaction log of the second node based on the one or more subsequent requests.

17. The system of claim 11 , wherein:

the one or more subsequent requests includes a plurality of subsequent requests;

the one or more operations includes a plurality of operations associated with the one or more storage objects; and

the one or more node controllers are further configured to, while awaiting the first acknowledgment:

sequentially write a plurality of subsequent transaction entries in the transaction log of the first node based on the plurality of subsequent requests; and

request the second node sequentially writes the plurality of subsequent transaction entries in the transaction log of the second node.

18. The system of claim 11 , wherein the first node and the second node comprise a key-value store cluster.

19. The system of claim 11 , wherein the first node is a master node and the second node is a slave node.

20. A system, comprising:

means for writing a first transaction entry in a transaction log of a first node based on a first request for a first operation associated with a first storage object;

means for requesting that a second node writes the first transaction entry in a transaction log of the second node based on the first request;

means for writing one or more subsequent transaction entries in the transaction log of the first node based on one or more subsequent requests for one or more operations associated with one or more storage objects, while awaiting a first acknowledgment of a completion of a writing of the first transaction entry in the transaction log of the second node;

means for determining a failure to write a certain entry from the one or more subsequent transaction entries in one of the transaction log of the first node and the transaction log of the second node;

means for terminating, responsive to determining the failure, a processing of a certain request from the one or more subsequent requests that is associated with the certain entry; and

means for providing a failure notification to a client that submitted the certain request associated with the certain entry.

Assignments (5)
PATENT COLLATERAL AGREEMENT - A&R LOAN AGREEMENT Recorded Aug 21, 2023
From: WESTERN DIGITAL TECHNOLOGIES, INC.
To: JPMORGAN CHASE BANK, N.A.
Reel/Frame 064715/0001 →
PATENT COLLATERAL AGREEMENT - DDTL LOAN AGREEMENT Recorded Aug 21, 2023
From: WESTERN DIGITAL TECHNOLOGIES, INC.
To: JPMORGAN CHASE BANK, N.A.
Reel/Frame 067045/0156 →
RELEASE OF SECURITY INTEREST AT REEL 052025 FRAME 0088 Recorded Feb 8, 2022
From: JPMORGAN CHASE BANK, N.A.
To: WESTERN DIGITAL TECHNOLOGIES, INC.
Reel/Frame 058965/0699 →
SECURITY INTEREST Recorded Feb 26, 2020
From: WESTERN DIGITAL TECHNOLOGIES, INC.
To: JPMORGAN CHASE BANK, N.A., AS AGENT
Reel/Frame 052025/0088 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jan 10, 2020
From: FOQUE, MAARTEN
To: WESTERN DIGITAL TECHNOLOGIES, INC.
Reel/Frame 051480/0516 →
Continuity (1)
Related Publication 20210182246A1 · Jun 17, 2021