IP Library › Granted Patent US 10,191,663
Granted Patent B1
US 10,191,663 · App. 15/269,935 · Granted Jan 29, 2019

Using data store accelerator intermediary nodes and write control settings to identify write propagation nodes

Inventors: Brian O'Neill (Bellevue, WA); Kevin Christen (Seattle, WA); Omer Ahmed Zaki (Bellevue, WA); Kiran Kumar Muniswamy Reddy (Sammamish, WA)
Assignee: Amazon Technologies, Inc.
G06F3/0611G06F3/0604G06F3/065G06F3/067G06F3/0619G06F3/0655G06F9/54G06F17/30575G06F17/30578G06F2009/45562
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,191,663
App. No.
15/269,935
Granted
Jan 29, 2019
Kind
B1
Abstract

An accelerator intermediary node (AIN) associated with a data store obtains an indication of a control setting to be applied with respect to a write request directed to a data item, where the control setting specifies a target for one or more of replication count, data durability, a transaction grouping with respect to a write request, or back-end synchronization node. Using the control setting, a write propagation node set is identified for the write request. The write propagation node set includes another accelerator intermediary node and/or a storage node of a data store. Respective operation requests corresponding to the write request are transmitted to one or more members of the write propagation node set. A write coordinator role may be verified prior to attempting a commit together of a plurality of write requests as part of a multi-write transaction.

Claims (50)

1. A system, comprising:

one or more accelerator intermediary nodes associated with a plurality of data stores of a provider network, including a first accelerator intermediary node implemented at least in part at a first computing device, wherein the plurality of data stores include a first data store with a first data model and a second data store with a different data model;

wherein the first accelerator intermediary node is configured to:

obtain, via a first programmatic interface, an indication of a first control setting to be applied with respect to a first write request directed to a first data item, wherein the first control setting specifies a respective first target for one or more of: (a) replication count, (b) data durability, (c) transaction grouping with respect to one or more write requests including the first write request, or (d) a back-end synchronization mode;

determine, based at least in part on the first control setting, a first write propagation node set with respect to the first write request, wherein the first write propagation node set comprises one or more of (a) a second accelerator intermediary node of the one or more accelerator intermediary nodes or (b) a first storage node of the first data store;

transmit, prior to providing an indication that the first write request has succeeded, a respective first operation request corresponding to the first write request to one or more members of the first write propagation node set;

receive an indication of a second control setting to be applied with respect to a second write request directed to a second data item, wherein at least one target indicated in the second control setting differs from a corresponding target indicated in the first control setting; and

transmit a respective second operation request corresponding to the second write request to one or more members of a second write propagation node set, wherein the second write propagation node set comprises a second storage node of the second data store.

2. The system as recited in claim 1 , wherein the first control setting indicates that a plurality of write requests, including the first write request, are to be committed together as part of a multi-write transaction, wherein the plurality of write requests includes a third write request directed to a third data item of the second data store.

3. The system as recited in claim 2 , wherein the first accelerator intermediary node is further configured to:

determine an assignee of a write coordinator role with respect to a different write request of the plurality of write requests; and

verify, prior to attempting a commit of the multi-write transaction, that the write coordinator role has not been reassigned after the different write request was processed.

4. The system as recited in claim 1 , wherein the first accelerator intermediary node is further configured to:

obtain an indication of a third control setting to be applied with respect to a third write request directed to a third data item; and

in response to determining that applying the third control setting would require a change to a set of resources allocated on behalf of one or more clients, reject the third control setting.

5. The system as recited in claim 1 , wherein the first accelerator intermediary node is further configured to:

insert, prior to transmitting a first operation request, a data entry into a local data cache of the first accelerator intermediary node, wherein the data entry indicates a modification operation included in the first write request, wherein the data entry is formatted in accordance with a binary encoding; and

wherein the first storage node is configured to store a representation of the first data item formatted in accordance with a different encoding at a persistent storage device of the first storage node, in response to the first operation request.

6. A method, comprising:

performing, at a first accelerator intermediary node of a fleet of accelerator intermediary nodes associated with one or more data stores, wherein the first accelerator intermediary node comprises one or more computing devices:

obtaining, via a first programmatic interface, an indication of a first control setting to be applied with respect to a first write request directed to a first data item, wherein the first control setting specifies a respective first target for one or more of: (a) data durability or (b) transaction grouping with respect to one or more write requests including the first write request;

identifying, based at least in part on the first control setting, a first write propagation node set with respect to the first write request, wherein the first write propagation node set comprises one or more of (a) a second accelerator intermediary node of the fleet of accelerator intermediary nodes or (b) a first storage node of a first data store of the one or more data stores; and

transmitting a respective first operation request corresponding to the first write request to one or more members of the first write propagation node set.

7. The method as recited in claim 6 , wherein the first control setting indicates that a plurality of write requests, including the first write request, are to be committed as one multi-write transaction to the one or more data stores.

8. The method as recited in claim 7 , further comprising performing, by the first accelerator intermediary node:

determining an assignee of a write coordinator role with respect to a different write request of the plurality of write requests; and

verifying, prior to attempting a commit of the multi-write transaction, that the write coordinator role has not been reassigned after the different write request was processed.

9. The method as recited in claim 6 , further comprising performing, by the first accelerator intermediary node:

obtaining an indication of a second control setting to be applied with respect to a second write request directed to a second data item; and

determining that applying the second control setting would require a change to a set of resources allocated on behalf of a client.

10. The method as recited in claim 9 , further comprising performing, by the first accelerator intermediary node:

in response to determining that applying the second control setting would require a change to a set of resources, providing an indication that the second control setting has been rejected.

11. The method as recited in claim 9 , further comprising performing, by the first accelerator intermediary node:

in response to determining that applying the second control setting would require a change to a set of resources, providing an indication of the change to a client.

12. The method as recited in claim 6 , wherein the first control setting indicates an acceptable storage device type at which a portion of at least one replica of the first data item is to be stored.

13. The method as recited in claim 6 , wherein obtaining the indication of the first control setting comprises:

determining a value of a parameter passed in an invocation of an API (application programming interface) by a client-side component of a network-accessible storage service.

14. The method as recited in claim 13 , wherein one or more other parameters are passed in the invocation, including one or more of: (a) an identifier of the first data item or (b) a write payload.

15. The method as recited in claim 6 , further comprising:

inserting, by the first accelerator intermediary node prior to transmitting a first operation request, a data entry into a local data cache of the first accelerator intermediary node, wherein the data entry indicates a modification operation included in the first write request, wherein the data entry is formatted in accordance with a binary encoding; and

in response to the first operation request, storing, at a persistent storage device of the first storage node, a representation of the first data item formatted in accordance with a different encoding.

16. A non-transitory computer-accessible storage medium storing program instructions that when executed on one or more processors implement an accelerator intermediary node of a fleet of accelerator intermediary nodes associated with one or more data stores, wherein the accelerator intermediary node is configured to:

obtain, via a first programmatic interface, an indication of a first control setting to be applied with respect to a first write request directed to a data item, wherein the first control setting specifies a transaction grouping with respect to one or more write requests including the first write request;

identify, based at least in part on the first control setting, a first write propagation node set with respect to the first write request, wherein the first write propagation node set comprises one or more of (a) a second accelerator intermediary node of the fleet of accelerator intermediary nodes or (b) a first storage node of a first data store of the one or more data stores; and

transmit a respective first operation request corresponding to the first write request to one or more members of the first write propagation node set.

17. The non-transitory computer-accessible storage medium as recited in claim 16 , wherein the first control setting includes an indication of: (a) a replication count with respect to the first write request and (b) a first acknowledgement requirement corresponding to at least a first replica indicated via the replication count.

18. The non-transitory computer-accessible storage medium as recited in claim 17 , wherein the first control setting includes an indication of: a different acknowledgement requirement corresponding to a second replica indicated via the replication count.

19. The non-transitory computer-accessible storage medium as recited in claim 16 , wherein the first programmatic interface comprises a console for the fleet of accelerator intermediary nodes.

20. The non-transitory computer-accessible storage medium as recited in claim 16 , wherein to obtain the indication, the accelerator intermediary node is configured to examine contents of a first message received via a network, wherein the accelerator intermediary node is configured to:

examine contents of a second message received via the network to obtain the first write request.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Sep 20, 2016
From: O'NEILL, BRIAN; CHRISTEN, KEVIN; ZAKI, OMER AHMED; MUNISWAMY REDDY, KIRAN KUMAR
To: AMAZON TECHNOLOGIES, INC.
Reel/Frame 039797/0879 →
Cited By (2)
US 12,423,023 US 12,443,587