Granted Patent B2

US 12,436,919 · App. 18/925,675 · Granted Oct 7, 2025

Inventor: Sweetesh Singh (New Delhi, IN)

Assignee: Cohesity, Inc.

G06F16/1748G06F16/1744

Abstract

Claims (140)

1. A method comprising:

selecting, by processing circuitry of a data platform, a plurality of data chunks from a data store for creating a chunkfile;

in response to selecting the plurality of data chunks from the data store for creating the chunkfile, determining, by the processing circuitry of the data platform, an entropy value for each of the plurality of data chunks selected to obtain a corresponding plurality of entropy values by, at least in part, calculating, via the processing circuitry, the entropy value based on a determined frequency of each of a plurality of symbols representative of each of the plurality of data chunks selected;

reorganizing, by the processing circuitry and based on the corresponding plurality of entropy values, the plurality of data chunks to obtain a reorganized plurality of data chunks;

compressing, by the processing circuitry, the reorganized plurality of data chunks to obtain a compressed chunkfile; and

storing, by the processing circuitry, the compressed chunkfile superseding the plurality of data chunks.

2. The method of claim 1 , further comprising:

deduplicating, by a chunkfile manager, a collection of data chunks stored by a storage system to create a deduplicated collection of data chunks; and

selecting, by the processing circuitry of the data platform, the plurality of data chunks from the deduplicated collection of data chunks.

3. The method of claim 1 , further comprising:

calculating, by the processing circuitry of the data platform, the entropy value for each data chunk within a collection of data chunks stored by a storage system;

comparing, by a chunkfile manager, each data chunk within the collection of data chunks with an entropy value threshold; and

in response to comparing each data chunk within the collection of data chunks with the entropy value threshold, selecting, by the processing circuitry of the data platform, the plurality of data chunks from the collection of data chunks based on the entropy value for each data chunk selected as satisfying the entropy value threshold.

4. The method of claim 1 , further comprising:

encrypting, by processing circuitry, the compressed chunkfile as a single file to obtain an encrypted compressed chunkfile.

5. The method of claim 1 , further comprising:

reorganizing, by the processing circuitry, pointers referencing each of the plurality of data chunks stored by a storage system into one of a descending order or an ascending order, according to the entropy value for each of the plurality of data chunks; and

responsive to reorganizing the pointers referencing each of the plurality of data chunks into the ascending order or the descending order, updating, by the processing circuitry of the data platform, the pointers referencing each of the plurality of data chunks within the storage system into the ascending order or the descending order.

6. The method of claim 5 :

wherein each of the pointers referencing each of the plurality of data chunks stored by the storage system are stored as nodes within a linked list; and

wherein the method further comprises sequentially organizing the nodes within the linked list corresponding to the pointers referencing each of the plurality of data chunks stored by the storage system into one of the descending order or the ascending order according to establish a new order for the plurality of data chunks.

7. The method of claim 1 , further comprising:

selecting a compression algorithm from a plurality of compression algorithms based on properties of the plurality of data chunks; and

compressing the reorganized plurality of data chunks to obtain the compressed chunkfile using the compression algorithm selected.

8. The method of claim 1 , wherein:

determining the entropy value for each of the plurality of data chunks selected includes calculating, via the processing circuitry, the entropy value according to a following formula:

H

=

-

1

*

sum

(

p

i

*

log

⁢

(

p

i

)

)

;

wherein H represents the entropy value as calculated by the processing circuitry;

wherein i represents an index for each of the plurality of symbols representative of each of the plurality of data chunks selected;

wherein p i is the frequency for each of the plurality of symbols i; and

wherein the entropy value H is in bits per symbol when a log base is 2.

9. A data platform comprising:

processing circuitry;

a storage system;

a chunkfile manager;

a compression manager;

non-transitory computer readable media; and

wherein instructions, when executed by the processing circuitry, configure the processing circuitry to:

select, by the processing circuitry of the data platform, a plurality of data chunks from the storage system for creating a chunkfile;

in response to selecting the plurality of data chunks from the storage system for creating the chunkfile, determine, by the processing circuitry of the data platform, an entropy value for each of the plurality of data chunks selected to obtain a corresponding plurality of entropy values by, at least in part, calculating, via the processing circuitry, the entropy value based on a determined frequency of each of a plurality of symbols representative of each of the plurality of data chunks selected;

reorganize, by the processing circuitry and based on the corresponding plurality of entropy values, the plurality of data chunks to obtain a reorganized plurality of data chunks;

compress, by the processing circuitry, the reorganized plurality of data chunks to obtain a compressed chunkfile; and

store, by the processing circuitry to the storage system, the compressed chunkfile superseding the plurality of data chunks.

10. The data platform of claim 9 , wherein the instructions cause the processing circuitry to:

deduplicate, by the chunkfile manager, a collection of data chunks stored by the storage system to create a deduplicated collection of data chunks; and

select, by the processing circuitry of the data platform, the plurality of data chunks from the deduplicated collection of data chunks.

11. The data platform of claim 9 , wherein the instructions cause the processing circuitry to:

calculate, by an entropy calculator of the data platform, the entropy value for each data chunk within a collection of data chunks stored by the storage system;

compare, by the chunkfile manager, each data chunk within the collection of data chunks with an entropy value threshold; and

in response to a comparison of each data chunk within the collection of data chunks with the entropy value threshold, the instructions cause the processing circuitry to select the plurality of data chunks from the collection of data chunks based on the entropy value for each data chunk selected as satisfying the entropy value threshold.

12. The data platform of claim 9 , wherein the instructions cause the processing circuitry to:

reorganize, by the chunkfile manager of the data platform, pointers referencing each of the plurality of data chunks stored by the storage system into one of a descending order or an ascending order, according to the entropy value for each of the plurality of data chunks; and

responsive to a reorganization of the pointers referencing each of the plurality of data chunks into the ascending order or the descending order, the instructions cause the processing circuitry to update the pointers referencing each of the plurality of data chunks within the storage system into the ascending order or the descending order.

13. The data platform of claim 12 :

wherein each of the pointers referencing each of the plurality of data chunks stored by the storage system are stored as nodes within a linked list; and

wherein the instructions cause the processing circuitry to sequentially organize the nodes within the linked list corresponding to the pointers referencing each of the plurality of data chunks stored by the storage system into one of the descending order or the ascending order according to establish a new order for the plurality of data chunks.

14. The data platform of claim 9 , wherein the instructions cause the processing circuitry to:

select a compression algorithm from a plurality of compression algorithms based on properties of the plurality of data chunks; and

compress the reorganized plurality of data chunks to obtain the compressed chunkfile using the compression algorithm selected.

15. The data platform of claim 9 , wherein to determine the entropy value for each of the plurality of data chunks selected includes the instructions further configured to cause the processing circuitry to:

calculate, via the processing circuitry, the entropy value according to a following formula:

H

=

-

1

*

sum

(

p

i

*

log

⁢

(

p

i

)

)

;

wherein H represents the entropy value as calculated by the processing circuitry;

wherein i represents an index for each of the plurality of symbols representative of each of the plurality of data chunks selected;

wherein p i is the frequency for each of the plurality of symbols i; and

wherein the entropy value H is in bits per symbol when a log base is 2.

16. Computer-readable storage media comprising instructions that, when executed, configure processing circuitry to:

select a plurality of data chunks from a data store for creating a chunkfile;

responsive to a selection of the plurality of data chunks from the data store for creating the chunkfile, determine an entropy value for each of the plurality of data chunks selected to obtain a corresponding plurality of entropy values by, at least in part, the processing circuitry configured by the instructions to calculate the entropy value based on a determined frequency of each of a plurality of symbols representative of each of the plurality of data chunks selected;

reorganize, by the processing circuitry and based on the corresponding plurality of entropy values, the plurality of data chunks to obtain a reorganized plurality of data chunks;

compress, by the processing circuitry, the reorganized plurality of data chunks to obtain a compressed chunkfile; and

store, by the processing circuitry to the data store, the compressed chunkfile superseding the plurality of data chunks.

17. The computer-readable storage media of claim 16 , wherein the instructions cause the processing circuitry to:

deduplicate a collection of data chunks stored by a storage system to create a deduplicated collection of data chunks; and

select the plurality of data chunks from the deduplicated collection of data chunks.

18. The computer-readable storage media of claim 16 , wherein the instructions cause the processing circuitry to:

calculate the entropy value for each data chunk within a collection of data chunks;

compare each data chunk within the collection of data chunks with an entropy value threshold; and

in response to a comparison of each data chunk within the collection of data chunks with the entropy value threshold, the instructions cause the processing circuitry to select the plurality of data chunks from the collection of data chunks based on the entropy value for each data chunk selected as satisfying the entropy value threshold.

19. The computer-readable storage media of claim 16 , wherein the instructions cause the processing circuitry to:

reorganize pointers referencing each of the plurality of data chunks stored by a storage system into one of a descending order or an ascending order, according to the entropy value for each of the plurality of data chunks; and

responsive to a reorganization of the pointers referencing each of the plurality of data chunks into the ascending order or the descending order, the instructions cause the processing circuitry to update the pointers referencing each of the plurality of data chunks within the storage system into the ascending order or the descending order.

20. The computer-readable storage media of claim 16 , wherein to determine the entropy value for each of the plurality of data chunks selected includes the instructions, when executed, to further cause the processing circuitry to:

calculate, via the processing circuitry, the entropy value according to a following formula:

H

=

-

1

*

sum

(

p

i

*

log

⁢

(

p

i

)

)

;

wherein H represents the entropy value as calculated by the processing circuitry;

wherein i represents an index for each of the plurality of symbols representative of each of the plurality of data chunks selected;

wherein p i is the frequency for each of the plurality of symbols i; and

wherein the entropy value H is in bits per symbol when a log base is 2.

Assignments (1)

Continuity (2)

References Cited (16)