IP Library Granted Patent US 10,437,675
Granted Patent B2
US 10,437,675 · App. 15/667,097 · Granted Oct 8, 2019

Distributed erasure coded virtual file system

Inventors: Maor Ben Dayan (Tel Aviv, IL); Omri Palmon (Tel Aviv, IL); Liran Zvibel (Tel Aviv, IL)
Assignee: Weka.IO Ltd.
G06F11/1088G06F3/064G06F3/067G06F3/0619G06F3/0664G06F11/1076G06F11/1415G06F16/182G06F16/188G06F2201/80
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,437,675
App. No.
15/667,097
Granted
Oct 8, 2019
Kind
B2
Abstract

A plurality of computing devices are communicatively coupled to each other via a network, and each of the plurality of computing devices comprises one or more of a plurality of storage devices. A plurality of failure resilient address spaces are distributed across the plurality of storage devices such that each of the plurality of failure resilient address spaces spans a plurality of the storage devices. Each one of the plurality of failure resilient address spaces is organized into a plurality of stripes. Each one or more stripes of the plurality of stripes is part of a respective one of a plurality of forward error correction (FEC) protection domains. Each of the plurality of stripes may comprise a plurality of storage blocks. Each block of a particular one of the plurality of stripes may reside on a different one of the plurality of storage devices.

Claims (35)

1. A system comprising:

a computing device comprising a plurality of storage devices, wherein:

a plurality of address spaces are distributed across the plurality of storage devices such that each of the plurality of address spaces spans two or more of the plurality of storage devices;

each one of the plurality of address spaces is organized into a plurality of stripes;

the computing device is operable to select which of the plurality of stripes to use for a next commit to the particular one of the plurality of address spaces;

each one or more stripes of the plurality of stripes is part of a respective one of a plurality of forward error correction (FEC) protection domains;

the plurality of FEC protection domains comprises a plurality of first storage blocks and a second storage block;

stored in one or more of the plurality of first storage blocks is a data portion of a plurality of data portions; and

stored in the second storage block is a protection portion calculated based on contents of the plurality of first storage blocks.

2. The system of claim 1 , wherein the selection of which of the plurality of stripes to use for the next commit is based on how many storage blocks are in each of the plurality of stripes.

3. The system of claim 1 , wherein the selection of which of the plurality of stripes to use for the next commit is based on a bitmap stored on the plurality of storage devices.

4. The system of claim 1 , wherein the selection of which of the plurality of stripes to use for the next commit is based on how many storage blocks currently storing data are in each of the plurality of stripes.

5. The system of claim 1 , wherein the selection of which of the plurality of stripes to use for the next commit is based on read and write overhead for committing to each of the plurality of stripes.

6. The system of claim 1 , wherein each of the address spaces spans a plurality of failure domains, such that upon failure of any particular one of the plurality of failure domains, a workload for reconstructing the lost data is distributed among each of the others of the plurality of failure domains.

7. The system of claim 1 , wherein, in an instance when multiple of the plurality of stripes have one or more failed blocks, the computing device is operable to prioritize reconstruction of the multiple of the plurality of stripes in descending order of number of failed blocks in each of the multiple of the plurality of stripes.

8. The system of claim 1 , wherein the plurality of computing devices are operable to perform the reconstruction of a stripe of the plurality of stripes that has two or more failed blocks at a higher rate than a rate at which the computing device performs the reconstruction of a stripe of the plurality of stripes which has only one failed block.

9. The system of claim 1 , wherein the computing device is operable to:

perform successive commit operations to a selected one of the plurality of stripes until the selected one of the plurality of stripes no longer meets a determined criterion; and

upon the selected one of the plurality of stripes no longer meeting the determined criterion, select a different one of the plurality of stripes.

10. The system of claim 9 , wherein the criterion is based on how many blocks are available for new data to be written to.

11. A method comprising:

distributing a plurality of address spaces across a plurality of storage devices such that each of the plurality of address spaces spans two or more of the plurality of storage devices;

organizing each of the plurality of address spaces is into a plurality of stripes, wherein each stripe of the plurality of stripes is part of a forward error correction (FEC) protection domain that comprises a plurality of first storage blocks and a second storage block;

during a commit, storing a data portion in one or more of the plurality of first storage blocks of a particular stripe of the plurality of stripes;

during the commit, storing a protection portion in the second storage block of the particular stripe, wherein the protection portion is based on contents of the plurality of first storage blocks of the particular stripe;

performing successive commit operations to a selected one of the plurality of stripes until the selected one of the plurality of stripes no longer meets a determined criterion; and

upon the selected one of the plurality of stripes no longer meeting the determined criterion, selecting a different one of the plurality of stripes.

12. The method of claim 11 , wherein the particular stripe to use for the commit is based on how many storage blocks are in each of the plurality of stripes.

13. The method of claim 11 , wherein the particular stripe to use for the commit is based on a bitmap stored on the plurality of storage devices.

14. The method of claim 11 , wherein the particular stripe to use for the commit is based on how many storage blocks currently storing data are in each of the plurality of stripes.

15. The method of claim 11 , wherein the particular stripe to use for the commit is based on read and write overhead for committing to each of the plurality of stripes.

16. The method of claim 11 , wherein each of the address spaces spans a plurality of failure domains, and wherein upon failure of a particular failure domain of the plurality of failure domains, the method comprises distributing a workload among each of the plurality of failure domains that did not fail, and reconstructing lost data of the particular failure domain according to the distributed workload.

17. The method of claim 11 , wherein, in an instance when multiple of the plurality of stripes have one or more failed blocks, the method comprises prioritize reconstructing the multiple of the plurality of stripes in descending order of number of failed blocks in each of the multiple of the plurality of stripes.

18. The method of claim 11 , wherein the method comprises reconstructing a first stripe of the plurality of stripes that has two or more failed blocks at a higher rate than a second stripe of the plurality of stripes which has only one failed block.

19. The method of claim 11 , wherein the criterion is based on how many blocks are available for new data to be written to.

Assignments (4)
RELEASE OF SECURITY INTEREST Recorded Jun 20, 2024
From: BANK LEUMI LE-ISRAEL B.M.
To: WEKAIO LTD.
Reel/Frame 067783/0962 →
CORRECTIVE ASSIGNMENT TO CORRECT THE ASSIGNEE NAME PREVIOUSLY RECORDED AT REEL: 051875 FRAME: 0773. ASSIGNOR(S) HEREBY CONFIRMS THE ASSIGNMENT. Recorded Oct 12, 2020
From: BEN DAYAN, MAOR; PALMON, OMRI; ZVIBEL, LIRAN
To: WEKA.IO LTD.
Reel/Frame 054049/0574 →
SECURITY INTEREST Recorded Mar 29, 2020
From: WEKAIO LTD.
To: BANK LEUMI LE-ISRAEL B.M.
Reel/Frame 052253/0860 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Feb 20, 2020
From: BEN DAYAN, MAOR; PALMON, OMRI; ZVIBEL, LIRAN
To: ILLINOIS TOOL WORKS INC.
Reel/Frame 051875/0773 →
Continuity (3)
Continuation 15243519 · Aug 22, 2016
Continuation 14833053 · Aug 22, 2015
Related Publication 20170329673A1 · Nov 16, 2017
Cited By (1)
US 12,292,797