Granted Patent B2

US 11,836,388 · App. 17/236,398 · Granted Dec 5, 2023

Inventors: Sandeep Nirmale (Santa Clara, CA); Ramprasad Chinthekindi (San Jose, CA); Gobikrishnan Sundharraj (Santa Clara, CA); Rahul Goyal (Indore, IN)

Assignee: EMC IP HOLDING COMPANY LLC

G06F3/0661G06F3/067G06F3/0608G06F3/0641G06F3/0652H03M7/30

Abstract

Claims (49)

1. A computer-implemented method, comprising:

receiving a plurality of data segments and a plurality of metadata segments associated with the data segments to be stored in a storage system, wherein the data segments are deduplicated segments and each of the metadata segments includes a fingerprint for the corresponding data segment;

for each of the metadata segments,

determining that a metadata segment contains one or more markers inserted by a client device of the storage system,

examining the metadata segment to determine whether the metadata segment satisfies a predetermined condition including a presence of an inline marker suitable for compression, wherein each inline marker has a size equal to or less than two hundred bytes,

in response to determining that the metadata segment satisfies the predetermined condition,

compressing the metadata segment using a predetermined compression algorithm, and

storing the compressed metadata segment in the storage system, and

otherwise storing the metadata segment in the storage system without compression; and

storing the data segments in the storage system.

2. The method of claim 1 , wherein the metadata segment satisfies the predetermined condition if a size of the metadata segment exceeds a predetermined size threshold.

3. The method of claim 1 , wherein the metadata segment satisfies the predetermined condition if a system parameter of the storage system has been set to a predetermined configuration state.

4. The method of claim 1 , wherein the metadata segment satisfies the predetermined condition if a number of markers contained in the metadata segment exceeds a predetermined marker threshold.

5. The method of claim 4 , wherein the metadata segment satisfies the predetermined condition if a compression ratio for a metadata segment having a similar number of markers is above a predetermined compression threshold based on a histogram of prior metadata segment compressions.

6. The method of claim 1 , wherein the metadata segment satisfies the predetermined condition if heuristics of prior compression of metadata indicates that a prior compression ratio is above a predetermined compression threshold.

7. The method of claim 1 , wherein the metadata segment satisfies the predetermined condition if available processing resources of the storage system for data compression are above a predetermined resource threshold.

8. The method of claim 1 , wherein the metadata segment satisfies the predetermined condition if an available storage space of the storage system is above predetermined storage threshold.

9. The method of claim 1 , wherein the data segments and the metadata segments are received from the client device as a part of a backup operation from the client device to the storage system.

10. The method of claim 1 , wherein the data segments and the metadata segments are identified by a garbage collection process periodically performed on existing data segments and metadata segments previously stored in the storage system.

11. A non-transitory machine-readable medium having instructions stored therein, which when executed by a processor, cause the processor to perform operations, the operations comprising:

receiving a plurality of data segments and a plurality of metadata segments associated with the data segments to be stored in a storage system, wherein the data segments are deduplicated segments and each of the metadata segments includes a fingerprint for the corresponding data segment;

for each of the metadata segments,

determining that a metadata segment contains one or markers inserted by a client device of the storage system,

examining the metadata segment to determine whether the metadata segment satisfies a predetermined condition including a presence of an inline marker suitable for compression, wherein the inline marker has a size equal to or less than two hundred bytes,

in response to determining that the metadata segment satisfies the predetermined condition,

compressing the metadata segment using a predetermined compression algorithm, and

storing the compressed metadata segment in the storage system, and

otherwise storing the metadata segment in the storage system without compression; and

storing the data segments in the storage system.

12. The machine-readable medium of claim 11 , wherein the metadata segment satisfies the predetermined condition if a size of the metadata segment exceeds a predetermined size threshold.

13. The machine-readable medium of claim 11 , wherein the metadata segment satisfies the predetermined condition if a system parameter of the storage system has been set to a predetermined configuration state.

14. The machine-readable medium of claim 11 , wherein the metadata segment satisfies the predetermined condition if a number of markers contained in the metadata segment exceeds a predetermined marker threshold.

15. The machine-readable medium of claim 14 , wherein the metadata segment satisfies the predetermined condition if a compression ratio for a metadata segment having a similar number of markers is above a predetermined compression threshold based on a histogram of prior metadata segment compressions.

16. The machine-readable medium of claim 11 , wherein the metadata segment satisfies the predetermined condition if heuristics of prior compression of metadata indicates that a prior compression ratio is above a predetermined compression threshold.

17. The machine-readable medium of claim 11 , wherein the metadata segment satisfies the predetermined condition if available processing resources of the storage system for data compression are above a predetermined resource threshold.

18. The machine-readable medium of claim 11 , wherein the metadata segment satisfies the predetermined condition if an available storage space of the storage system is above predetermined storage threshold.

19. The machine-readable medium of claim 11 , wherein the data segments and the metadata segments are received from the client device as a part of a backup operation from the client device to the storage system.

20. A data processing system, comprising:

a processor; and

a memory coupled to the processor and storing instructions, which when executed by the processor, cause the processor to perform operations, the operations comprising:

receiving a plurality of data segments and a plurality of metadata segments associated with the data segments to be stored in a storage system, wherein the data segments are deduplicated segments and each of the metadata segments includes a fingerprint for the corresponding data segment;

for each of the metadata segments,

determining that a metadata segment contains one or markers inserted by a client device of the storage system,

examining the metadata segment to determine whether the metadata segment satisfies a predetermined condition,

in response to determining that the metadata segment satisfies the predetermined condition including a presence of an inline marker suitable for compression, wherein the inline marker has a size equal to or less than two hundred bytes,

compressing the metadata segment using a predetermined compression algorithm, and

storing the compressed metadata segment in the storage system, and

otherwise storing the metadata segment in the storage system without compression; and

storing the data segments in the storage system.

Assignments (10)

Continuity (1)