IP Library Granted Patent US 10,594,792
Granted Patent B1
US 10,594,792 · App. 15/791,388 · Granted Mar 17, 2020

Scale-out erasure coding

View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,594,792
App. No.
15/791,388
Granted
Mar 17, 2020
Kind
B1
Abstract

Overhead associated with data re-protection during scaling out and/or scaling up of a cloud storage system can be reduced. During initial data protection (e.g., prior to a change in the cluster size), data fragments and coding fragments can be stored as a set of adjacent data sub-fragments and a set of adjacent coding sub-fragments, respectively. In response to determining that the cluster has expanded, a portion of the set of adjacent data sub-fragments can be moved to the new nodes that have been added to the cluster. Further, the set of adjacent coding sub-fragments can be combined to generate a new coding fragment that can be stored within the new cluster. Accordingly, the amount of system resources utilized during re-protection can be significantly reduced.

Claims (41)

1. A system, comprising:

a processor; and

a memory that stores executable instructions that, when executed by the processor, facilitate performance of operations, comprising:

storing, across first nodes of a network-based storage system, data fragments associated with a chunk of data, wherein the storing comprises storing the data fragments as respective groups of adjacent data sub-fragments;

storing, within a second node of the network-based storage system, a first coding fragment associated with the data fragments, wherein the storing comprises storing the first coding fragment as a group of adjacent coding sub-fragments; and

in response to determining that a set of additional nodes, of the network-based storage system other than the first nodes or the second node, have been added to the network-based storage system,

moving a portion of the respective groups of adjacent data sub-fragments to the set of additional nodes, and

combining the group of adjacent coding sub-fragments to generate a second coding fragment different than the first coding fragment.

2. The system of claim 1 , wherein the combining comprises adding the group of adjacent coding sub-fragments together to generate the second coding fragment.

3. The system of claim 1 , wherein the combining comprises performing an exclusive disjunction operation with respect to at least two adjacent coding sub-fragments of the group of adjacent coding sub-fragments.

4. The system of claim 1 , wherein the operations further comprise:

storing, within the second node, the second coding fragment; and

deleting the first coding fragment.

5. The system of claim 1 , wherein the first coding fragment is determined based on defined coding matrix data.

6. The system of claim 1 , wherein the data fragments and the first coding fragment are determined based on an erasure coding process and the operations further comprise:

in response to the determining, updating the erasure coding process resulting in an updated erasure coding process.

7. The system of claim 6 , wherein the updating comprises updating the erasure coding process based on an analysis that compares a capacity benefit of the updated erasure coding process relative to a cost associated with data re-protection via the updated erasure coding process.

8. The system of claim 6 , wherein the updating comprises updating the erasure coding process to increase a defined measure of capacity use efficiency for the network-based storage system.

9. The system of claim 6 , wherein the updating comprises updating the erasure coding process to satisfy a defined tolerance criterion associated with a number of node failures determined to have been endured by the network-based storage system.

10. The system of claim 1 , wherein the network-based storage system comprises an object storage system.

11. A method, comprising:

based on an erasure coding scheme, determining, by a system comprising a processor, data fragments and a coding fragment associated with a chunk of data;

storing, within different nodes of an object storage system, the data fragments as respective groups of adjacent data sub-fragments and the coding fragment as a group of adjacent coding sub-fragments; and

in response to determining that additional nodes have been added to the object storage system:

moving a portion of the respective groups of adjacent data sub-fragments to the additional nodes, and

combining the group of adjacent coding sub-fragments to generate an updated coding fragment.

12. The method of claim 11 , wherein the combining adding the group of adjacent coding sub-fragments.

13. The method of claim 11 , wherein the combining comprises performing an exclusive disjunction operation to generate the updated coding fragment.

14. The method of claim 11 , wherein the storing comprises storing the data fragments and the coding fragment within an elastic cloud system.

15. The method of claim 11 , wherein the erasure coding scheme is a first erasure coding scheme and the method further comprises:

in response to the determining that the set of additional nodes have been added to the object storage system, selecting a second erasure coding scheme to protect the chunk of data.

16. The method of claim 11 , wherein the determining the coding fragment comprises determining the coding fragment based on a defined coding matrix associated with the erasure coding scheme.

17. A non-transitory computer-readable medium comprising instructions that, in response to execution, cause a system comprising a processor to perform operations, comprising:

based on an erasure coding, determining, data fragments and a coding fragment associated with a chunk of data;

storing, within different nodes of an object storage system, the data fragments as respective groups of adjacent data sub-fragments and the coding fragment as a group of adjacent coding sub-fragments; and

in response to determining that additional nodes have been added to the object storage system,

facilitating a transfer of a portion of the respective groups of adjacent data sub-fragments to the additional nodes, and

updating the coding fragment based on a result of combining the group of adjacent coding sub-fragments.

18. The non-transitory computer-readable medium of claim 17 , wherein the updating comprises adding the group of adjacent coding sub-fragments.

19. The non-transitory computer-readable medium of claim 17 , wherein the updating comprises generating an updated coding fragment based on performing an exclusive OR operation on the group of adjacent coding sub-fragments.

20. The non-transitory computer-readable medium of claim 17 , wherein the object storage system comprises an elastic cloud system.

Assignments (8)
RELEASE OF SECURITY INTEREST IN PATENTS PREVIOUSLY RECORDED AT REEL/FRAME (053546/0001) Recorded Jun 23, 2022
From: THE BANK OF NEW YORK MELLON TRUST COMPANY, N.A., AS NOTES COLLATERAL AGENT
To: DELL MARKETING L.P. (ON BEHALF OF ITSELF AND AS SUCCESSOR-IN-INTEREST TO CREDANT TECHNOLOGIES, INC.); DELL INTERNATIONAL L.L.C.; DELL PRODUCTS L.P.; DELL USA L.P.; EMC CORPORATION; DELL MARKETING CORPORATION (SUCCESSOR-IN-INTEREST TO FORCE10 NETWORKS, INC. AND WYSE TECHNOLOGY L.L.C.); EMC IP HOLDING COMPANY LLC
Reel/Frame 071642/0001 →
RELEASE OF SECURITY INTEREST IN PATENTS PREVIOUSLY RECORDED AT REEL/FRAME (044535/0109) Recorded May 20, 2022
From: THE BANK OF NEW YORK MELLON TRUST COMPANY, N.A., AS NOTES COLLATERAL AGENT
To: DELL PRODUCTS L.P.; EMC CORPORATION; EMC IP HOLDING COMPANY LLC; DELL MARKETING CORPORATION (SUCCESSOR-IN-INTEREST TO WYSE TECHNOLOGY L.L.C.)
Reel/Frame 060753/0414 →
RELEASE OF SECURITY INTEREST AT REEL 044535 FRAME 0001 Recorded Nov 2, 2021
From: CREDIT SUISSE AG, CAYMAN ISLANDS BRANCH
To: DELL PRODUCTS L.P.; EMC CORPORATION; EMC IP HOLDING COMPANY LLC; WYSE TECHNOLOGY L.L.C.
Reel/Frame 058298/0475 →
SECURITY AGREEMENT Recorded Apr 22, 2020
From: CREDANT TECHNOLOGIES INC.; DELL INTERNATIONAL L.L.C.; DELL MARKETING L.P.; DELL PRODUCTS L.P.; DELL USA L.P.; EMC CORPORATION; FORCE10 NETWORKS, INC.; WYSE TECHNOLOGY L.L.C.; EMC IP HOLDING COMPANY LLC
To: THE BANK OF NEW YORK MELLON TRUST COMPANY, N.A.
Reel/Frame 053546/0001 →
SECURITY AGREEMENT Recorded Mar 21, 2019
From: CREDANT TECHNOLOGIES, INC.; DELL INTERNATIONAL L.L.C.; DELL MARKETING L.P.; DELL PRODUCTS L.P.; DELL USA L.P.; EMC CORPORATION; FORCE10 NETWORKS, INC.; WYSE TECHNOLOGY L.L.C.; EMC IP HOLDING COMPANY LLC
To: THE BANK OF NEW YORK MELLON TRUST COMPANY, N.A.
Reel/Frame 049452/0223 →
PATENT SECURITY AGREEMENT (NOTES) Recorded Nov 29, 2017
From: DELL PRODUCTS L.P.; EMC CORPORATION; EMC IP HOLDING COMPANY LLC; WYSE TECHNOLOGY L.L.C.
To: THE BANK OF NEW YORK MELLON TRUST COMPANY, N.A., AS COLLATERAL AGENT
Reel/Frame 044535/0109 →
PATENT SECURITY AGREEMENT (CREDIT) Recorded Nov 29, 2017
From: DELL PRODUCTS L.P.; EMC CORPORATION; EMC IP HOLDING COMPANY LLC; WYSE TECHNOLOGY L.L.C.
To: CREDIT SUISSE AG, CAYMAN ISLANDS BRANCH, AS COLLATERAL AGENT
Reel/Frame 044535/0001 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Oct 23, 2017
From: DANILOV, MIKHAIL; KURILOV, ANDREY
To: EMC IP HOLDING COMPANY LLC
Reel/Frame 043928/0357 →