Method, electronic device and computer program product for storing data
Techniques for storing data involve estimating a hit ratio of a digest cache associated with a target storage device, the digest cache recording a digest of data that is stored in the target storage device after preprocessing; generating, according to a determination that the hit ratio is lower than a predetermined threshold, a digest for target data to be stored and performing the preprocessing; and storing, according to a determination that the digest of the target data is missing in the digest cache, the preprocessed target data in the target storage device, and recording the digest of the target data in the digest cache. Such techniques can achieve good system performance in both cases of high data repetition and low data repetition.
1. A method for storing data, comprising:
estimating a hit ratio of a digest cache associated with a target storage device, based at least on corresponding hit ratios of the digest cache in a plurality of time windows and corresponding weights of the plurality of time windows, wherein the plurality of time windows include i) a short time window that is a predetermined length of time preceding a current time, and ii) a long time window that extends from a time of a predetermined event until the current time, and wherein the short time window and the long time window have different corresponding weights, the digest cache recording a digest of data that is stored in the target storage device after preprocessing;
generating, in response to a determination that the hit ratio is lower than a predetermined threshold, a digest for target data to be stored and performing the preprocessing, wherein the preprocessing comprises compressing the target data, wherein generating the digest for the target data and compressing the target data are performed in parallel, wherein the generating of the digest for the target data and the compressing of the target data are both performed after reading the target data from a memory to a processor cache of a processor, and wherein generating the digest for the target data and compressing the target data are performed in parallel by hardware supporting parallel computing within the processor; and
storing, according to a determination that the digest of the target data is missing in the digest cache, the preprocessed target data in the target storage device, and recording the digest of the target data in the digest cache.
2. The method of claim 1 , further comprising:
determining whether the digest of the target data is hit in the digest cache; and
updating, based on a result of the determination, the corresponding hit ratios of the digest cache in the plurality of time windows.
3. The method of claim 1 , wherein generating the digest for the target data and performing the preprocessing further comprises:
reading the target data from a memory to the processor cache; and
generating, in the processor cache, the digest for the target data and performing the preprocessing.
4. The method of claim 3 , wherein storing the preprocessed target data in the target storage device comprises:
flushing the preprocessed target data from the processor cache to the target storage device.
5. The method of claim 1 , further comprising:
generating, according to a determination that the hit ratio exceeds the predetermined threshold,
the digest of the target data; and
performing, according to a determination that the digest of the target data is missing in the digest cache, the preprocessing on the target data.
6. The method of claim 5 , wherein generating the digest of the target data comprises:
reading the target data from a memory to the processor cache; and
generating, in the processor cache, the digest of the target data.
7. The method of claim 6 , wherein performing the preprocessing on the target data comprises:
reading, based on a determination that the target data is missing in the processor cache, the target data from the memory to the processor cache; and
performing, in the processor cache, the preprocessing on the target data.
8. The method of claim 1 , wherein the predetermined event comprises system startup.
9. The method of claim 1 , wherein the predetermined event comprises creation of a logical storage unit.
10. An electronic device, comprising:
at least one processing unit;
at least one memory coupled to the at least one processing unit and storing instructions for execution by the at least one processing unit, wherein when executed by the at least one processing unit, the instructions cause the electronic device to perform actions comprising:
estimating a hit ratio of a digest cache associated with a target storage device, based at least on corresponding hit ratios of the digest cache in a plurality of time windows and corresponding weights of the plurality of time windows, wherein the plurality of time windows include i) a short time window that is a predetermined length of time preceding a current time, and ii) a long time window that extends from a time of a predetermined event until the current time, and wherein the short time window and the long time window have different corresponding weights, the digest cache recording a digest of data that is stored in the target storage device after preprocessing;
generating, in response to a determination that the hit ratio is lower than a predetermined threshold, a digest for target data to be stored and performing the preprocessing, wherein the preprocessing comprises compressing the target data, wherein generating the digest for the target data and compressing the target data are performed in parallel, wherein the generating of the digest for the target data and the compressing of the target data are both performed after reading the target data from a memory to a processor cache of a processor, and wherein generating the digest for the target data and compressing the target data are performed in parallel by hardware supporting parallel computing within the processor; and
storing, according to a determination that the digest of the target data is missing in the digest cache, the preprocessed target data in the target storage device, and recording the digest of the target data in the digest cache.
11. The electronic device of claim 10 , wherein the actions further comprise:
determining whether the digest of the target data is hit in the digest cache; and
updating, based on a result of the determination, the corresponding hit ratios of the digest cache in the plurality of time windows.
12. The electronic device of claim 10 , wherein generating the digest for the target data and performing the preprocessing comprises:
reading the target data from the memory to the processor cache; and
generating, in the processor cache, the digest for the target data and performing the preprocessing.
13. The electronic device of claim 12 , wherein storing the preprocessed target data in the target storage device comprises:
flushing the preprocessed target data from the processor cache to the target storage device.
14. The electronic device of claim 10 , wherein the actions further comprise:
generating, according to a determination that the hit ratio exceeds the predetermined threshold,
the digest of the target data; and
performing, according to a determination that the digest of the target data is missing in the digest cache, the preprocessing on the target data.
15. The electronic device of claim 14 , wherein generating the digest of the target data comprises:
reading the target data from the memory to the processor cache; and
generating, in the processor cache, the digest of the target data.
16. The electronic device of claim 14 , wherein performing the preprocessing on the target data comprises:
reading, based on a determination that the target data is missing in the processor cache, the target data from the memory to the processor cache; and
performing, in the processor cache, the preprocessing on the target data.
17. A computer program product having a non-transitory computer readable medium which stores a set of instructions to store data; the set of instructions, when carried out by computerized circuitry, causing the computerized circuitry to perform a method of:
estimating a hit ratio of a digest cache associated with a target storage device, based at least on corresponding hit ratios of the digest cache in a plurality of time windows and corresponding weights of the plurality of time windows, wherein the plurality of time windows include i) a short time window that is a predetermined length of time preceding a current time, and ii) a long time window that extends from a time of a predetermined event until the current time, and wherein the short time window and the long time window have different corresponding weights, the digest cache recording a digest of data that is stored in the target storage device after preprocessing;
generating, in response to a determination that the hit ratio is lower than a predetermined threshold, a digest for target data to be stored and performing the preprocessing, wherein the preprocessing comprises compressing the target data, wherein generating the digest for the target data and compressing the target data are performed in parallel, wherein the generating of the digest for the target data and the compressing of the target data are both performed after reading the target data from a memory to a processor cache of a processor, and wherein generating the digest for the target data and compressing the target data are performed in parallel by hardware supporting parallel computing within the processor; and
storing, according to a determination that the digest of the target data is missing in the digest cache, the preprocessed target data in the target storage device, and recording the digest of the target data in the digest cache.