IP Library Granted Patent US 10,817,374
Granted Patent B2
US 10,817,374 · App. 15/952,179 · Granted Oct 27, 2020

Meta chunks

Inventors: Mikhail Danilov (Saint Petersburg, RU); Kirill Gusakov (Saint Petersburg, RU)
Assignee: EMC IP HOLDING COMPANY LLC
G06F11/1076G06F3/067G06F3/0619G06F3/0644G06F3/0652H03M13/154
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,817,374
App. No.
15/952,179
Granted
Oct 27, 2020
Kind
B2
Abstract

Data protection with meta chunks increases capacity use efficiency without verification and data copying. In one aspect, a meta chunk is a data protection unit, which combines two or more source chunks that are determined to have a reduced sets of data fragments. The meta chunk can be encoded to generate a set of coding fragments, which can be stored and utilized to recover data fragments of any of the two or more source chunks. Further, the source chunks can be linked to the meta chunk. Furthermore, the sets of coding fragments, that were previously generated by individually encoding each source chunk, can be deleted.

Claims (42)

1. A system, comprising:

a processor; and

a memory that stores executable instructions that, when executed by the processor, facilitate performance of operations, comprising:

determining source chunks stored within a storage system, wherein the source chunks are determined to have a collective number of data fragments that is less than a number of data fragments that a chunk is configured to be able to store;

based on combining the source chunks, generating a meta chunk; and

encoding the meta chunk to generate coding fragments that are employable to recover at least a portion of the source chunks,

wherein the encoding comprises erasure coding the meta chunk.

2. The system of claim 1 , wherein the operations further comprise:

linking the source chunks to the meta chunk.

3. The system of claim 2 , wherein the linking comprises updating metadata associated with a source chunk of the source chunks to comprise reference data indicative of the meta chunk.

4. The system of claim 1 , wherein the coding fragments are meta chunk coding fragments and the operations further comprise:

subsequent to the encoding, deleting source chunk coding fragments that have been generated based on individually encoding the source chunks.

5. The system of claim 1 , wherein the collective number of data fragments is a first collective number of data fragments, wherein the meta chunk is a first meta chunk, the source chunks are first source chunks, the first source chunks comprise a second meta chunk that has been generated based on a combination of second source chunks, and wherein the second source chunks are determined to have a second collective number of data fragments that is less than the number of data fragments.

6. The system of claim 1 , wherein the generating the meta chunk comprises selecting ones of the source chunks such that the collective number of data fragments of the selected ones of the source chunks is closest to the number of data fragments.

7. The system of claim 1 , wherein the erasure coding the meta chunk comprises generating, for k fragments of the meta chunk, m redundant coding fragments, k and m being positive integers and m being less than k.

8. The system of claim 7 , further comprising:

performing recovery of an unavailable data fragment by utilizing mapping information that maps the source chunks to the meta chunk.

9. The system of claim 1 , wherein physical capacity is not allocated for the meta chunk.

10. The system of claim 8 , wherein the performing the recovery comprises employing a decoding matrix that corresponds to a coding matrix that was utilized in the erasure coding.

11. The system of claim 1 , wherein the generating the meta chunk comprises:

creating a layout within the meta chunk that maps data fragments of the source chunks to the meta chunk independently of modifying a storage location within the source chunks of the data fragments.

12. The system of claim 1 , wherein the generating the meta chunk comprises:

maintaining locations of the data chunks within the source chunks, the meta chunk referring to the locations of the data chunks.

13. A method, comprising:

selecting, by a system comprising a processor, source chunks from chunks of an object storage system, wherein the source chunks are determined to combine to have fewer data fragments than a first number of data fragments that are able to be stored in a chunk;

combining, by the system, the source chunks to generate a meta chunk; and

based on erasure coding the meta chunk, determining, by the system, coding fragments for the source chunks at a meta chunk level, wherein the coding fragments are to be employed to recover at least a portion of the source chunks during a failure condition.

14. The method of claim 13 , further comprising:

storing, by the system, reference data that links the source chunks to the meta chunk.

15. The method of claim 14 , further comprising:

based on the reference data, determining, by the system, the meta chunk linked to at least the portion of the source chunks.

16. The method of claim 15 , further comprising:

recovering, by the system, at least the portion of the source chunks based on performing a decoding operation at the meta chunk level.

17. The method of claim 13 , wherein the coding fragments are first coding fragments, and the method further comprises:

subsequent to the determining, deleting, by the system, source second fragments that have been generated based on individually erasure coding the source chunks.

18. A non-transitory computer-readable storage medium comprising instructions that, in response to execution, cause a server device comprising a processor to perform operations, comprising:

combining source chunks stored within an object storage system to generate a meta chunk, wherein the source chunks are determined not to have more than a defined number of data fragments equal to a number of data fragments configured to be stored in a chunk; and

based on erasure coding the meta chunk, determining coding fragments for the source chunks at a meta chunk level, wherein the coding fragments are to be employed to recover at least a portion of the source chunks during a failure condition.

19. The non-transitory computer-readable storage medium of claim 18 , wherein the operations further comprise:

determining reference data that links the source chunks to the meta chunk.

20. The non-transitory computer-readable storage medium of claim 19 , wherein the operations further comprise:

updating metadata associated with the source chunks the reference data.

Assignments (8)
RELEASE OF SECURITY INTEREST IN PATENTS PREVIOUSLY RECORDED AT REEL/FRAME (053546/0001) Recorded Jun 23, 2022
From: THE BANK OF NEW YORK MELLON TRUST COMPANY, N.A., AS NOTES COLLATERAL AGENT
To: DELL MARKETING L.P. (ON BEHALF OF ITSELF AND AS SUCCESSOR-IN-INTEREST TO CREDANT TECHNOLOGIES, INC.); DELL INTERNATIONAL L.L.C.; DELL PRODUCTS L.P.; DELL USA L.P.; EMC CORPORATION; DELL MARKETING CORPORATION (SUCCESSOR-IN-INTEREST TO FORCE10 NETWORKS, INC. AND WYSE TECHNOLOGY L.L.C.); EMC IP HOLDING COMPANY LLC
Reel/Frame 071642/0001 →
RELEASE OF SECURITY INTEREST IN PATENTS PREVIOUSLY RECORDED AT REEL/FRAME (046366/0014) Recorded May 20, 2022
From: THE BANK OF NEW YORK MELLON TRUST COMPANY, N.A., AS NOTES COLLATERAL AGENT
To: DELL PRODUCTS L.P.; EMC CORPORATION; EMC IP HOLDING COMPANY LLC
Reel/Frame 060450/0306 →
RELEASE OF SECURITY INTEREST AT REEL 046286 FRAME 0653 Recorded Nov 2, 2021
From: CREDIT SUISSE AG, CAYMAN ISLANDS BRANCH
To: DELL PRODUCTS L.P.; EMC CORPORATION; EMC IP HOLDING COMPANY LLC
Reel/Frame 058298/0093 →
SECURITY AGREEMENT Recorded Apr 22, 2020
From: CREDANT TECHNOLOGIES INC.; DELL INTERNATIONAL L.L.C.; DELL MARKETING L.P.; DELL PRODUCTS L.P.; DELL USA L.P.; EMC CORPORATION; FORCE10 NETWORKS, INC.; WYSE TECHNOLOGY L.L.C.; EMC IP HOLDING COMPANY LLC
To: THE BANK OF NEW YORK MELLON TRUST COMPANY, N.A.
Reel/Frame 053546/0001 →
SECURITY AGREEMENT Recorded Mar 21, 2019
From: CREDANT TECHNOLOGIES, INC.; DELL INTERNATIONAL L.L.C.; DELL MARKETING L.P.; DELL PRODUCTS L.P.; DELL USA L.P.; EMC CORPORATION; FORCE10 NETWORKS, INC.; WYSE TECHNOLOGY L.L.C.; EMC IP HOLDING COMPANY LLC
To: THE BANK OF NEW YORK MELLON TRUST COMPANY, N.A.
Reel/Frame 049452/0223 →
PATENT SECURITY AGREEMENT (CREDIT) Recorded Jun 1, 2018
From: DELL PRODUCTS L.P.; EMC CORPORATION; EMC IP HOLDING COMPANY LLC
To: CREDIT SUISSE AG, CAYMAN ISLANDS BRANCH, AS COLLATERAL AGENT
Reel/Frame 046286/0653 →
PATENT SECURITY AGREEMENT (NOTES) Recorded Jun 1, 2018
From: DELL PRODUCTS L.P.; EMC CORPORATION; EMC IP HOLDING COMPANY LLC
To: THE BANK OF NEW YORK MELLON TRUST COMPANY, N.A., AS COLLATERAL AGENT
Reel/Frame 046366/0014 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Apr 12, 2018
From: DANILOV, MIKHAIL; GUSAKOV, KIRILL
To: EMC IP HOLDING COMPANY LLC
Reel/Frame 045528/0470 →
Continuity (1)
Related Publication 20190317859A1 · Oct 17, 2019