Granted Patent B2

US 11,755,549 · App. 17/183,736 · Granted Sep 12, 2023

Inventors: Tao Chen (Beijing, CN); Geng Han (Beijing, CN)

Assignee: EMC IP HOLDING COMPANY LLC

G06F16/215H03M7/40

Abstract

Claims (68)

1. A method for data processing, including:

determining, based on sizes of multiple data segments included in data to be processed, a first time required to perform a matching operation for each data segment, wherein the matching operation is used to determine non-duplicate data segments;

determining, based on the size of each data segment and a compression level for the data to be processed, a second time required to perform a compression operation for each data segment, wherein a different compression level corresponds to a different compression ratio, different storage space that compressed data requires, and different time requirements to process;

determining, based on the first time, the second time, and a de-duplication rate for the data to be processed, a target mode for processing the multiple data segments from a first mode and a second mode,

wherein in the first mode, a compression operation is performed only on the non-duplicate data segments in the multiple data segments based on a time required to process the data using the first mode, the first time, the second time, and the de-duplication rate of the data to be processed, and

wherein in the second mode, a compression operation is performed on each of the multiple data segments; and

performing the matching operation and the compression operation based on the target mode.

2. The method according to claim 1 , wherein determining the target mode includes:

determining that the target mode is the first mode if it is determined that the de-duplication rate is greater than a ratio of the first time to the second time; and

determining that the target mode is the second mode if it is determined that the de-duplication rate is less than the ratio of the first time to the second time.

3. The method according to claim 1 , wherein the target mode is the first mode, and the method further includes:

performing the matching operation for the multiple data segments, respectively;

determining, based on results of the matching operations, a set of data segments to be compressed from the multiple data segments; and

performing the compression operation on the set of data segments to be compressed in accordance with the compression level.

4. The method according to claim 1 , wherein the target mode is the second mode, and the method further includes:

performing the matching operation and the compression operation on each of the multiple data segments in accordance with the compression level.

5. The method according to claim 1 , wherein determining the first time includes:

determining an average of sizes of data segments that were processed over a historical time period; and

determining the first time based on the average.

6. The method according to claim 1 , wherein determining the second time includes:

acquiring a compression-level mapping table, wherein the compression-level mapping table includes multiple compression operation times corresponding to multiple sizes of data segments and multiple compression levels; and

determining the second time from the multiple compression operation times based on the compression-level mapping table, the sizes of the data segments, and the compression levels.

7. An electronic device, including:

a processor; and

a memory coupled to the processor, wherein the memory has instructions stored therein that, when executed by the processor, cause the electronic device to perform actions including:

determining, based on sizes of multiple data segments included in data to be processed, a first time required to perform a matching operation for each data segment, wherein the matching operation is used to determine non-duplicate data segments;

determining, based on the size of each data segment and a compression level for the data to be processed, a second time required to perform a compression operation for each data segment, wherein a different compression level corresponds to a different compression ratio, different storage space that compressed data requires, and different time requirements to process, wherein the;

determining, based on the first time, the second time, and a de-duplication rate for the data to be processed, a target mode for processing the multiple data segments from a first mode and a second mode,

wherein in the first mode, a compression operation is performed only on the non-duplicate data segments in the multiple data segments based on a time required to process the data using the first mode, the first time, the second time, and the de-duplication rate of the data to be processed, and

wherein in the second mode, a compression operation is performed on each of the multiple data segments; and

performing the matching operation and the compression operation based on the target mode.

8. The electronic device according to claim 7 , wherein determining the target mode includes:

determining that the target mode is the first mode if it is determined that the de-duplication rate is greater than a ratio of the first time to the second time; and

determining that the target mode is the second mode if it is determined that the de-duplication rate is less than the ratio of the first time to the second time.

9. The electronic device according to claim 7 , wherein the target mode is the first mode, and the actions further comprise:

performing the matching operation for the multiple data segments, respectively;

determining, based on results of the matching operations, a set of data segments to be compressed from the multiple data segments; and

performing the compression operation on the set of data segments to be compressed in accordance with the compression level.

10. The electronic device according to claim 7 , wherein the target mode is the second mode, and the actions further comprise:

performing the matching operation and the compression operation on each of the multiple data segments in accordance with the compression level.

11. The electronic device according to claim 7 , wherein determining the first time includes:

determining an average of sizes of data segments that were processed over a historical time period; and

determining the first time based on the average.

12. The electronic device according to claim 7 , wherein determining the second time includes:

acquiring a compression-level mapping table, wherein the compression-level mapping table includes multiple compression operation times corresponding to multiple sizes of data segments and multiple compression levels; and

determining the second time from the multiple compression operation times based on the compression-level mapping table, the sizes of the data segments, and the compression levels.

13. A computer program product that is tangibly stored on a non-transitory computer-readable medium and includes machine-executable instructions, which when executed by a processor, cause the processor to perform actions, the actions comprising:

determining, based on sizes of multiple data segments included in data to be processed, a first time required to perform a matching operation for each data segment, wherein the matching operation is used to determine non-duplicate data segments;

determining, based on the size of each data segment and a compression level for the data to be processed, a second time required to perform a compression operation for each data segment, wherein a better compression level corresponds to a higher compression ratio, less storage space that compressed data requires, and more time requirements to process;

determining, based on the first time, the second time, and a de-duplication rate for the data to be processed, a target mode for processing the multiple data segments from a first mode and a second mode,

wherein in the first mode, a compression operation is performed only on the non-duplicate data segments in the multiple data segments based on a time required to process the data using the first mode, the first time, the second time, and the de-duplication rate of the data to be processed, and

wherein in the second mode, a compression operation is performed on each of the multiple data segments; and

performing the matching operation and the compression operation based on the target mode.

14. The computer program product according to claim 13 , wherein determining the target mode includes:

determining that the target mode is the first mode if it is determined that the de-duplication rate is greater than a ratio of the first time to the second time; and

determining that the target mode is the second mode if it is determined that the de-duplication rate is less than the ratio of the first time to the second time.

15. The computer program product according to claim 13 , wherein the target mode is the first mode, and the actions further comprise:

performing the matching operation for the multiple data segments, respectively;

determining, based on results of the matching operations, a set of data segments to be compressed from the multiple data segments; and

performing the compression operation on the set of data segments to be compressed in accordance with the compression level.

16. The computer program product according to claim 13 , wherein the target mode is the second mode, and the actions further comprise:

performing the matching operation and the compression operation on each of the multiple data segments in accordance with the compression level.

17. The computer program product according to claim 13 , wherein determining the first time includes:

determining an average of sizes of data segments that were processed over a historical time period; and

determining the first time based on the average.

18. The computer program product according to claim 13 , wherein determining the second time includes:

acquiring a compression-level mapping table, wherein the compression-level mapping table includes multiple compression operation times corresponding to multiple sizes of data segments and multiple compression levels; and

determining the second time from the multiple compression operation times based on the compression-level mapping table, the sizes of the data segments, and the compression levels.

Assignments (10)

Priority Claims (1)

Continuity (1)