IP Library Granted Patent US 10,855,754
Granted Patent B1
US 10,855,754 · App. 16/143,340 · Granted Dec 1, 2020

Isolated read channel categories at streaming data service

Inventors: Benjamin Warren Mercier (Seattle, WA); Sayantan Chakravorty (Sammamish, WA); Yasemin Avcular (Seattle, WA); Charlie Paucard (Seattle, WA)
Assignee: Amazon Technologies, Inc.
H04L67/1029G06F16/285
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,855,754
App. No.
16/143,340
Filed
Sep 26, 2018
Granted
Dec 1, 2020
Kind
B1
Art Unit
2454
USPC
709/223
Abstract

In response to a first programmatic request, metadata indicating that a first isolated read channel of a real-time category has been associated with a first target stream is stored at a stream management service. In response to another request, metadata indicating that a second isolated read channel of a non-real-time category has been associated with a second target stream is stored. In response to a read request indicating the first channel or the second channel, one or more data records of the corresponding target streams are provided.

Claims (75)

1. A system, comprising:

one or more computing devices of a data streams management service;

wherein the one or more computing devices include instructions that upon execution on a processor cause the one or more computing devices to:

provide, via a programmatic interface, an indication of a plurality of categories of isolated read channels configurable for one or more data streams, including at least a first real-time category and a first non-real-time category;

in response to a first channel establishment request received via a programmatic interface, wherein the first channel establishment request indicates (a) at least a first target data stream and (b) the first real time category, perform one or more operations to

verify that a first storage configuration of the first target data stream meets a first performance capability criterion corresponding to the first real-time category; and

store metadata indicating that a first isolated read channel of the first real-time category has been established and associated with the first target data stream;

in response to a second channel establishment request received via the programmatic interface, wherein the second channel establishment request indicates (a) at least a second target data stream and (b) the first non-real time category, perform one or more operations to:

verify that a second storage configuration of the second target data stream meets a second performance capability criterion corresponding to the first non-real-time category; and

store metadata indicating that a second isolated read channel of the first non-real-time category has been established and associated with the second target data stream;

in response to a first read request directed to the first target data stream using the first isolated read channel, provide contents of a data record of the first target data stream using at least a portion of the first storage configuration; and

in response to a second read request directed to the second target data stream using the second isolated read channel, provide contents of a data record of the second target data stream using at least a portion of the second storage configuration.

2. The system as recited in claim 1 , wherein the instructions upon execution on the processor cause the one or more computing devices to:

in response to a third channel establishment request, wherein the third channel establishment request indicates a particular category of isolated read channels optimized to enable lagging readers to catch up with writes of a data stream, store metadata indicating that a third isolated read channel of the particular category has been established; and

in response to a third read request indicating the third isolated read channel, provide contents of a plurality of data records from a particular set of storage devices in which the plurality of data records is stored contiguously.

3. The system as recited in claim 1 , wherein the instructions upon execution on the processor cause the one or more computing devices to:

in response to a third channel establishment request, wherein the third channel establishment request indicates (a) at least the first target data stream and (b) at least the first non-real-time category of isolated read channels, storing metadata indicating that a third isolated read channel of the first non-real-time category has been established and associated with the first target data stream; and

in response to a third read request directed to the first target data stream using the third isolated read channel, providing contents of a data record of the first target data stream.

4. The system as recited in claim 1 , wherein the instructions upon execution on the processor cause the one or more computing devices to:

provide, via a programmatic interface, an indication of a first storage device type associated with a particular category of isolated read channels; and

in response to a read request indicating an isolated read channel of the particular category, obtaining contents of one or more data records from a storage device of the first storage device type.

5. The system as recited in claim 1 , wherein the instructions upon execution on the processor cause the one or more computing devices to:

obtain an indication, via a programmatic interface, of one or more targeted properties of isolated read channels, including at least a first performance-related property;

identify, based at least in part on the one or more targeted properties, a particular isolated read channel category from among a plurality of supported isolated read channel categories of a stream management service; and

provide an indication of the particular isolated read channel category via a programmatic interface.

6. A method, comprising:

performing, by one or more computing devices:

in response to a first programmatic channel establishment request, wherein the first programmatic channel establishment request indicates (a) at least a first target data stream and (b) at least a first real-time category of isolated read channels, storing metadata indicating that a first isolated read channel of the first real-time category has been established and associated with the first target data stream;

in response to a second programmatic channel establishment request, wherein the second programmatic channel establishment request indicates (a) at least a second target data stream and (b) at least a first non-real-time category of isolated read channels, storing metadata indicating that a second isolated read channel of the first non-real-time category has been established and associated with the second target data stream;

in response to a first read request directed to the first target data stream using the first isolated read channel, providing, from a first storage configuration, contents of a data record of the first target data stream; and

in response to a second read request directed to the second target data stream using the second isolated read channel, providing, from a second storage configuration, contents of a data record of the second target data stream.

7. The method as recited in claim 6 , further comprising performing, by the one or more computing devices:

in response to a third programmatic channel establishment request, wherein the third programmatic channel establishment request indicates (a) at least the first target data stream and (b) at least the first non-real-time category of isolated read channels, storing metadata indicating that a third isolated read channel of the first non-real-time category has been established and associated with the first target data stream; and

in response to a third read request directed to the first target data stream using the third isolated read channel, providing contents of a data record of the first target data stream.

8. The method as recited in claim 6 , further comprising performing, by the one or more computing devices:

providing, via a programmatic interface, an indication of a first storage device type associated with a particular category of isolated read channels; and

in response to a read request indicating an isolated read channel of the particular category, obtaining contents of one or more data records from a storage device of the first storage device type.

9. The method as recited in claim 6 , further comprising performing, by the one or more computing devices:

in response to a third programmatic channel establishment request, wherein the third programmatic channel establishment request indicates a particular category of isolated read channels optimized to enable lagging readers to catch up with writes of a data stream, storing metadata indicating that a third isolated read channel of the particular category has been established; and

in response to a third read request using the third isolated read channel, obtaining contents of a plurality of data records from a storage configuration in which the plurality of data records is stored contiguously.

10. The method as recited in claim 6 , wherein the plurality of data records is part of a third target data stream, wherein at least some data records of the second data stream are stored non-contiguously in a first storage repository, and wherein the plurality of data records is stored contiguously in a second storage repository.

11. The method as recited in claim 6 , further comprising performing, by the one or more computing devices:

obtaining an indication, via a programmatic interface, of one or more properties of a first client-customized category of isolated read channels;

storing an indication of the first client-customized category;

in response to a third programmatic channel establishment request, wherein the third programmatic channel establishment request indicates the first client-customized category of isolated read channels, storing metadata indicating that a third isolated read channel of the first client-customized category has been established; and

in response to a third read request using the third isolated read channel, providing contents of a plurality of data records.

12. The method as recited in claim 6 , further comprising performing, by the one or more computing devices:

obtaining an indication, via a programmatic interface, of one or more targeted properties of isolated read channels, including at least a first performance-related property;

identifying, based at least in part on the one or more targeted properties, a particular isolated read channel category from among a plurality of supported isolated read channel categories of a stream management service; and

providing an indication of the particular isolated read channel category via a programmatic interface.

13. The method as recited in claim 6 , further comprising performing, by the one or more computing devices:

obtaining an indication, via a programmatic interface, of a requested change of a category of an isolated read channel which has been established and associated with the first target data stream; and

initiating one or more re-configuration operations in accordance with the requested change of the category.

14. The method as recited in claim 6 , further comprising performing, by the one or more computing devices:

providing, via a programmatic interface, (a) a first set of performance metrics associated with the first real-time category of isolated read channels associated with the first target data stream and (b) a second set of performance metrics associated with the first non-real-time category of isolated read channels associated with the first target data stream.

15. The method as recited in claim 6 , further comprising performing, by the one or more computing devices:

in response to a third programmatic channel establishment request, wherein the third programmatic channel establishment request indicates a first tenancy-based category of isolated read channels for which resources configured in a particular tenancy mode are to be used, wherein the particular tenancy mode is selected from a set comprising a single-tenant mode and a multi-tenant mode, storing metadata indicating that a third isolated read channel of the first tenancy-based category has been established; and

in response to a third read request using the third isolated read channel, obtaining contents of a plurality of data records from a storage configuration corresponding to the particular tenancy mode.

16. One or more non-transitory computer-accessible storage media storing program instructions that when executed on or across one or more processors cause the one or more processors to:

in response to a first programmatic channel establishment request, wherein the first programmatic channel establishment request indicates (a) at least a first target data stream and (b) at least a first real-time category of isolated read channels, store metadata indicating that a first isolated read channel of the first real-time category has been established and associated with the first target data stream;

in response to a second programmatic channel establishment request, wherein the second programmatic channel establishment request indicates (a) at least a second target data stream and (b) at least a first non-real-time category of isolated read channels, store metadata indicating that a second isolated read channel of the first non-real-time category has been established and associated with the second target data stream;

in response to a first read request directed to the first target data stream using the first isolated read channel, provide contents of a data record of the first target data stream; and

in response to a second read request directed to the second target data stream using the second isolated read channel, providing contents of a data record of the second target data stream.

17. The one or more non-transitory computer-accessible storage media as recited in claim 16 , wherein the instructions when executed on or across the one or more processors cause the one or more processors to:

in response to a third programmatic channel establishment request, wherein the third programmatic channel establishment request indicates a particular category of isolated read channels optimized to enable lagging readers to catch up with writes of a data stream, store metadata indicating that a third isolated read channel of the particular category has been established; and

in response to a third read request using the third isolated read channel, obtain contents of a plurality of data records from a storage configuration in which the plurality of data records is stored contiguously.

18. The one or more non-transitory computer-accessible storage media as recited in claim 16 , wherein the instructions when executed on or across the one or more processors cause the one or more processors to:

in response to a third programmatic channel establishment request, wherein the third programmatic channel establishment request indicates (a) at least the first target data stream and (b) at least the first non-real-time category of isolated read channels, store metadata indicating that a third isolated read channel of the first non-real-time category has been established and associated with the first target data stream; and

in response to a third read request directed to the first target data stream using the third isolated read channel, provide contents of a data record of the first target data stream.

19. The one or more non-transitory computer-accessible storage media as recited in claim 16 , wherein the instructions when executed on or across the one or more processors cause the one or more processors to:

provide, via a programmatic interface, an indication of a first storage device type associated with a particular category of isolated read channels; and

in response to a read request indicating an isolated read channel of the particular category, obtain contents of one or more data records from a storage device of the first storage device type.

20. The one or more non-transitory computer-accessible storage media as recited in claim 16 , wherein the instructions when executed on or across the one or more processors cause the one or more processors to:

obtain an indication, via a programmatic interface, of a requested change of a category of an isolated read channel which has been established and associated with the first target data stream; and

initiate one or more re-configuration operations in accordance with the requested change of the category.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Dec 20, 2018
From: MERCIER, BENJAMIN WARREN; CHAKRAVORTY, SAYANTAN; AVCULAR, YASEMIN; PAUCARD, CHARLIE
To: AMAZON TECHNOLOGIES, INC.
Reel/Frame 047836/0517 →
Continuity (1)
Provisional Application 62698815 · Jul 16, 2018
Cited By (4)
US 12,407,730 US 12,413,624 US 12,445,493 US 12,452,306