IP Library Granted Patent US 11,474,977
Granted Patent B2
US 11,474,977 · App. 16/588,960 · Granted Oct 18, 2022

Snapshot isolation in a distributed storage system

Inventors: David Wetterau (San Francisco, CA); Braden Walker (San Francisco, CA)
Assignee: Dropbox, Inc.
G06F16/1865G06F11/1474G06F16/144G06F16/1824G06F2201/84
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,474,977
App. No.
16/588,960
Granted
Oct 18, 2022
Kind
B2
Abstract

Systems and methods for snapshot isolation in a distributed storage system M are provided. In some examples, a method can include receiving a request for data from a data store comprising a plurality of storage nodes, the request comprising a key associated with the data and a request timestamp, identifying a storage node in the plurality of storage nodes from which to read the data based on the key and a mapping of keys to storage nodes, determining that the request is associated with a transaction comprising operations associated with at least one other storage node in the plurality of storage nodes, determining a status of the entry for the key in the key-value table based on a lookup of a transaction table using the transaction identifier, and providing the data when the status of the entry for the key indicates the data is ready to be read.

Claims (49)

1. A method comprising:

receiving a request for a data object stored at a data storage system comprising a plurality of storage nodes, the request comprising a key associated with the data object and a request timestamp;

identifying, based on the key and a mapping of keys to storage nodes, a first storage node in the plurality of storage nodes from which to read the data object;

determining that the data object is associated with a transaction comprising at least one operation associated with a second storage node in the plurality of storage nodes, the determining based on an entry for the data object in a key-value table, wherein the entry for the data object includes the key and a transaction identifier that identifies the transaction and a transaction table;

identifying a transaction timestamp based on a query of the transaction table using the transaction identifier;

determining that the data object is ready to be read based on a comparison of a marker of the first storage node to the transaction timestamp; and

providing the data object when the data object is ready to be read.

2. The method of claim 1 , wherein the key-value table is stored on the first storage node.

3. The method of claim 1 , wherein the key comprises a sharding value and an object identifier.

4. The method of claim 3 , wherein each storage node in the plurality of storage nodes is associated with a range of sharding values.

5. The method of claim 1 , further comprising identifying the transaction table from a plurality of transaction tables based on a transaction map, wherein each transaction table in the plurality of transaction tables is stored on a storage node in the plurality of storage nodes.

6. The method of claim 1 , further comprising identifying the transaction table from a plurality of transaction tables based on an encoding of the transaction table in the transaction identifier.

7. The method of claim 1 , further comprising:

identifying a data object timestamp from the entry for the data object; and

determining that the data object is ready to be read based on a comparison of the marker of the first storage node to the data object timestamp.

8. The method of claim 1 , further comprising:

updating a data object timestamp in the entry for the data object with the transaction timestamp.

9. A non-transitory computer-readable medium comprising instructions that, when executed by one or more processors, cause a computing device to:

receive a request to perform a transaction to write data to a data storage system comprising a plurality of storage nodes, wherein the transaction comprises at least a first write operation on a first storage node in the plurality of storage nodes and a second write operation on a second storage node in the plurality of storage nodes;

create, for the first write operation, a first entry in a first key-value table, wherein the first entry includes a transaction identifier that identifies the transaction and a transaction table;

create, for the second write operation, a second entry in a second key-value table, wherein the second entry includes the transaction identifier;

create, for the transaction, a third entry in the transaction table, wherein the third entry includes the transaction identifier;

perform the first write operation on the first storage node and the second write operation on the second storage node;

in response to a read request to read a data object associated with the data, identify a transaction timestamp by querying the transaction table using the transaction identifier;

determine that the data object is ready to be read based on a comparison of a marker of the first storage node to the transaction timestamp; and

provide the data object when the data object is ready to be read.

10. The non-transitory computer-readable medium of claim 9 , wherein the first key-value table is stored on the first storage node and the second key-value table is stored on the second storage node.

11. The non-transitory computer-readable medium of claim 9 , wherein the instructions further cause the computing device to:

identify the first storage node and the second storage node for handling the request to write data to the data storage system based on a mapping of key ranges to storage nodes.

12. The non-transitory computer-readable medium of claim 9 , wherein the first entry further includes a first timestamp generated by the first storage node, and wherein the second entry further includes a second timestamp generated by the second storage node.

13. The non-transitory computer-readable medium of claim 9 , wherein the first write operation comprises an operation to write metadata associated with a content item.

14. A distributed storage system comprising:

a plurality of storage nodes; and

at least one front end device including at least a first processor configured to:

receive a request for a data object from a client device, the request comprising a key associated with the data object, and

identify, based on the key and a mapping of keys to storage nodes, a first storage node in the plurality of storage nodes from which to read the data object; and

wherein the first storage node including at least a second processor in the plurality of storage nodes is configured to:

determine that the data object is associated with a transaction comprising at least one operation associated with a second storage node in the plurality of storage nodes, the determining based on an entry for the data object in a key-value table, wherein the entry for the data object includes the key and a transaction identifier that identifies the transaction and a transaction table;

identify a transaction timestamp based on a query of the transaction table using the transaction identifier;

determine that the data object is ready to be read based on a comparison of a marker of the first storage node to the transaction timestamp; and

provide the data object when the data object is ready to be read.

15. The distributed storage system of claim 14 , wherein the first storage node is further configured to:

identify the transaction identifier in the entry for the data object in the key-value table indicating that the data object is associated with the transaction comprising at least one operation associated with the second storage node in the plurality of storage nodes;

wherein the data object is ready to be read based on a status of the at least one operation.

16. The distributed storage system of claim 15 , wherein the key-value table is stored on the first storage node.

17. The distributed storage system of claim 15 , wherein the front end device is further configured to identify the transaction table from a plurality of transaction tables based on an encoding of the transaction table in the transaction identifier.

18. The distributed storage system of claim 15 , wherein the first storage node is further configured to:

update a data object timestamp in the entry for the data object with the transaction timestamp.

19. The distributed storage system of claim 14 , wherein the at least one front end device and the plurality of storage nodes are a part of a content management system.

Assignments (4)
RELEASE OF SECURITY INTEREST Recorded Dec 13, 2024
From: JPMORGAN CHASE BANK, N.A., AS COLLATERAL AGENT
To: DROPBOX, INC.
Reel/Frame 069635/0332 →
SECURITY INTEREST Recorded Dec 12, 2024
From: DROPBOX, INC.
To: WILMINGTON TRUST, NATIONAL ASSOCIATION, AS COLLATERAL AGENT
Reel/Frame 069604/0611 →
PATENT SECURITY AGREEMENT Recorded Mar 10, 2021
From: DROPBOX, INC.
To: JPMORGAN CHASE BANK, N.A., AS COLLATERAL AGENT
Reel/Frame 055670/0219 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Nov 27, 2019
From: WETTERAU, DAVID; WALKER, BRADEN
To: DROPBOX, INC.
Reel/Frame 051130/0958 →