Concurrent data entropy and digest computation operating on same data for CPU cache efficiency
View Patent ↗Techniques for CPU cache efficiency may include performing concurrent processing, such as for first and second data operations, in a synchronized manner that prevents loading the same data chunk into the CPU cache more than once. Processing may include synchronizing the first and second data operations with respect to a first data chunk to ensure that both complete prior to proceeding with performing such processing on a second data chunk. The first and second data operations for a chunk may be deduplication digest computation and entropy computation performed inline as part of the data path. If the chunk cannot be deduplicated, the entropy value may be used with an adaptive entropy threshold to determine whether to compress the chunk inline. The entropy value may be determined using less than all bytes in the chunk. The chunk's entropy value may be determined based on a data set entropy value.
1. A method of performing data processing comprising:
performing, using a first processor, first processing that computes a data deduplication digest for a first data chunk;
performing, using a second processor, second processing that computes an entropy value for the first data chunk; and
synchronizing the first processing performed using the first processor and the second processing performed using the second processor with respect to the first data chunk, wherein said synchronizing ensures that both the first processing performed using the first processor and the second processing performed using the second processor have completed prior to proceeding with performing any of the first processing and the second processing on a second data chunk.
2. The method of claim 1 , wherein the first processing and the second processing for the first data chunk are performed in parallel.
3. The method of claim 2 , wherein one of the first processing and the second processing completes processing for the first data chunk prior to another one of the first processing and the second processing, and wherein said synchronizing does not allow the one of the first and second processing to proceed with processing the second data chunk until the another one of the first processing and the second processing has completed its processing for the first data chunk.
4. The method of claim 2 , wherein the first data processing and the second data processing are performed inline as part of I/O path processing.
5. The method of claim 2 , further comprising:
determining, using the data deduplication digest for a first data chunk, whether the first chunk can be deduplicated whereby the first chunk is a duplicate of an existing chunk currently associated with the data deduplication digest; and
responsive to determining that the first chunk can be deduplicated, deduplicating the first data chunk.
6. The method of claim 5 , further comprising:
responsive to determining that the first chunk cannot be deduplicated, performing processing including:
determining whether the entropy value for the first data chunk is greater than an entropy threshold;
responsive to determining that the entropy value for the first data chunk is not greater than the entropy threshold, determining not to compress the first data chunk inline as part of the I/O path; and
responsive to determining that the entropy value for the first data chunk is equal to or less than the entropy threshold, compressing the first data chunk inline as part of the I/O path.
7. The method of claim 6 , wherein the first data chunk is data of a write operation received at a data storage system and the method further comprises:
determining a current workload of the data storage system; and
modifying the entropy threshold in accordance with the current workload of the data storage system.
8. The method of claim 7 , further comprising:
determining whether the current workload is above a high workload threshold; and
responsive to determining the current workload is above the high workload threshold, decreasing the entropy threshold in accordance with the current workload.
9. The method of claim 7 , further comprising:
determining whether the current workload is less than a low workload threshold; and
responsive to determining the current workload is less than the low workload threshold, increasing the entropy threshold in accordance with the current workload.
10. The method of claim 1 , wherein the entropy value for the first data chunk is determined using less than all data of the first data chunk.
11. The method of claim 10 , wherein the entropy value for the first data chunk is determined using selected bytes of the first data chunk.
12. The method of claim 11 , wherein the first data chunk is included in a data set having a data set entropy value and the entropy value for the first data chunk is determined as the data set entropy value.
13. The method of claim 12 , wherein the data set includes a first plurality of data chunks and the method further comprising;
selecting a second plurality of data chunks of the data set, the second plurality being less than the first plurality of data chunks;
determining a plurality of entropy values for the second plurality of data chunks;
determining whether the plurality of entropy values are similar to one another; and
responsive to determining the plurality of entropy values are similar to one another, determining the data set entropy value in accordance with plurality of entropy values for the second number of data chunks of the data set.
14. A system comprising:
one or more processors; and
a memory comprising code stored thereon that, when executed, performs a method of performing data processing comprising:
performing first processing that computes a data deduplication digest for a first data chunk;
performing second processing that computes an entropy value for the first data chunk; and
synchronizing the first processing and the second processing with respect to the first data chunk, wherein said synchronizing ensures that both the first processing and the second processing have completed prior to proceeding with performing any of the first processing and the second processing on a second data chunk.
15. A computer readable medium comprising code stored thereon that, when executed, performs a method of performing data processing comprising:
performing first processing that computes a data deduplication digest for a first data chunk;
performing second processing that computes an entropy value for the first data chunk; and
synchronizing the first processing and the second processing with respect to the first data chunk, wherein said synchronizing ensures that both the first processing and the second processing have completed prior to proceeding with performing any of the first processing and the second processing on a second data chunk.
16. The computer readable medium of claim 15 , wherein the first processing and the second processing for the first data chunk are performed in parallel.
17. The computer readable medium of claim 16 , wherein one of the first processing and the second processing completes processing for the first data chunk prior to another one of the first processing and the second processing, and wherein said synchronizing does not allow the one of the first and second processing to proceed with processing the second data chunk until the another one of the first processing and the second processing has completed its processing for the first data chunk.
18. The computer readable medium of claim 16 , wherein the first data processing and the second data processing are performed inline as part of I/O path processing.
19. The computer readable medium of claim 16 , wherein the method further comprises:
determining, using the data deduplication digest for a first data chunk, whether the first chunk can be deduplicated whereby the first chunk is a duplicate of an existing chunk currently associated with the data deduplication digest; and
responsive to determining that the first chunk can be deduplicated, deduplicating the first data chunk.
20. The computer readable medium of claim 19 , wherein the method further comprises:
responsive to determining that the first chunk cannot be deduplicated, performing processing including:
determining whether the entropy value for the first data chunk is greater than an entropy threshold;
responsive to determining that the entropy value for the first data chunk is not greater than the entropy threshold, determining not to compress the first data chunk inline as part of the I/O path; and
responsive to determining that the entropy value for the first data chunk is equal to or less than the entropy threshold, compressing the first data chunk inline as part of the I/O path.