IP Library Granted Patent US 9,772,787
Granted Patent B2
US 9,772,787 · App. 14/231,116 · Granted Sep 26, 2017

File storage using variable stripe sizes

Inventors: Matti Juhani Oikarinen (Wilmington, MA); Matteo Frigo (Acton, MA); Pradeep Vincent (Kenmore, WA)
Assignee: Amazon Technologies, Inc.
G06F3/0631G06F3/061G06F3/064G06F3/0604G06F3/0614G06F3/0689G06F11/00
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 9,772,787
App. No.
14/231,116
Granted
Sep 26, 2017
Kind
B2
Abstract

A write request directed to a storage object is received at a distributed file storage service. Based on a variable stripe size selection policy, a size of a particular stripe of storage space to be allocated for the storage object is determined, which differs from the size of another stripe allocated earlier for the same storage object. Allocation of storage for the particular stripe at a particular storage device is requested, and if the allocation succeeds, the contents of the storage device are modified in accordance with the write request.

Claims (43)

1. A system, comprising:

one or more computing devices configured to:

receive, at a multi-tenant storage service configured to distribute file contents across a plurality of storage devices, a write request directed to a file, wherein the write request indicates (a) a write offset within the file and (b) a write data payload;

determine, based at least in part on the write offset within the file and the write data payload, that storage space is to be allocated to respond to the write request;

select, based at least in part on the write offset within the file, a size of a particular stripe of storage space to be allocated for the file;

identify, based at least in part on the size of the particular stripe to be allocated for the file, a storage subsystem node of the multi-tenant storage service at which at least one replica of the particular stripe to be allocated for the file is to be stored, wherein at least one replica of a different stripe of the file is stored at a different storage subsystem node, and wherein the size of the particular stripe to be allocated for the file differs from a size of the different stripe of the file;

allocate storage at the particular storage subsystem node for the particular stripe; and

modify contents of the particular stripe in accordance with the write request.

2. The system as recited in claim 1 , wherein the different stripe of the file starts at a smaller offset within the file than the write offset indicated in the write request, and wherein the size of the different stripe of the file is smaller than the size of the particular stripe of the file.

3. The system as recited in claim 1 , wherein the size of the particular stripe is selected based at least in part on an analysis of metrics collected from a file system to which the file belongs.

4. The system as recited in claim 1 , wherein the size of the particular stripe is determined based at least in part on a hint provided by a client.

5. The system as recited in claim 1 , wherein the size of the particular stripe is determined based at least in part on a name of the file.

6. A method, comprising:

performing, by one or more computing devices:

receiving, at a distributed storage service, a write request directed to a file;

determining, based at least in part on a variable stripe size selection policy, a size of a particular stripe of storage space to be allocated for the file to accommodate the write request;

allocating storage for the particular stripe of the file at a particular storage device, wherein storage for a different stripe of the file is allocated at a different storage device, and wherein the different stripe of the file has a different size than the particular stripe of the file; and

modifying contents of the particular storage device in accordance with the write request.

7. The method as recited in claim 6 , wherein in accordance with the variable stripe size selection policy, the size of the particular stripe is determined based at least in part on a write offset indicated in the write request.

8. The method as recited in claim 7 , wherein the different stripe of the file starts at a smaller offset within the file than the write offset indicated in the write request, and wherein the size of the different stripe of the file is smaller than the size of the particular stripe of the file.

9. The method as recited in claim 6 , wherein in accordance with the variable stripe size selection policy, the size of the particular stripe is determined based at least in part on an analysis of metrics collected from a file system to which the file belongs.

10. The method as recited in claim 6 , wherein in accordance with the variable stripe size selection policy, the size of the particular stripe is determined based at least in part on a hint provided by a client.

11. The method as recited in claim 6 , wherein in accordance with the variable stripe size selection policy, the size of the particular stripe is determined based at least in part on a name of the file.

12. The method as recited in claim 6 , further comprising:

allocating, in response to a triggering condition, storage space at a target storage device to store contents of the particular stripe and a different stripe of the file; and

combining contents of the particular stripe and the different stripe at the target storage device.

13. The method as recited in claim 12 , wherein the triggering condition comprises a determination that the file has exceeded a threshold size.

14. The method as recited in claim 6 , wherein the write request is formatted in accordance with one of: (a) a version of NFS (Network File System), or (b) a version of SMB (Server Message Block).

15. The method as recited in claim 6 , wherein the distributed storage service comprises a plurality of storage extents including a particular extent at the particular storage device, wherein each extent of the plurality of extents supports allocation of pages of one or more sizes, further comprising:

selecting, from among the plurality of storage devices, the particular storage device to allocate the particular stripe, based at least in part on a determination that the particular extent supports an allocation of a page of a particular size.

16. The method as recited in claim 6 , wherein the file does not comprise any stripes prior to said receiving the write request, and wherein in accordance with the variable stripe size selection policy, the size of the particular stripe is determined based at least in part on a size of a write payload of the write request.

17. A non-transitory computer-accessible storage medium storing program instructions that when executed on one or more processors:

receive, at a distributed storage service, a write request directed to a storage object, wherein the write request indicates a write offset within the storage object;

determine, based at least in part on the write offset within the storage object indicated in the write request, a size of a particular stripe of storage space to be allocated for the storage object;

request an allocation of storage for the particular stripe of storage space for the storage object at a particular storage device, wherein the storage object includes at least some storage space previously allocated for the storage object; and

initiate a modification of contents of the particular storage device in accordance with the write request.

18. The non-transitory computer-accessible storage medium as recited in claim 17 , wherein the size of the particular stripe to be allocated for the storage object differs from a size of another stripe of the storage object.

19. The non-transitory computer-accessible storage medium as recited in claim 17 , wherein the instructions when executed on the one or more processors:

determine the size of the particular stripe based at least in part on an analysis of metrics collected from a file system to which the storage object belongs.

20. The non-transitory computer-accessible storage medium as recited in claim 17 , wherein the storage object comprises one of: a file, or a metadata structure.

21. The non-transitory computer-accessible storage medium as recited in claim 17 , wherein the instructions when executed on the one or more processors:

allocate, in response to a triggering condition, storage space at a target storage device to store contents of the particular stripe and a different stripe of the file; and

combine contents of the particular stripe and the different stripe at the target storage device.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded May 29, 2014
From: OIKARINEN, MATTI JUHANI; FRIGO, MATTEO; VINCENT, PRADEEP
To: AMAZON TECHNOLOGIES, INC.
Reel/Frame 032990/0918 →
Continuity (1)
Related Publication 20150277802A1 · Oct 1, 2015