Selective deduplication based on data storage device controller status and media characteristics
View Patent ↗Systems and methods are disclosed for determining whether data to be written to a memory should be deduplicated. In some implementations, a method is provided. The method includes determining whether data to be written to a memory should be deduplicated based, at least in part, on status information of a controller and media characteristics of the memory, wherein the status information of the controller indicates a level of resources available for a deduplication operation. In response to determining that the data should be deduplicated, determining whether the data is duplicative based on the type of memory the data is being written to.
1. An apparatus comprising:
a memory; and
a controller configured to:
determine whether data to be written to the memory should be deduplicated based, at least in part, on status information of the controller and media characteristics of the memory, wherein the status information of the controller indicates a level of resources available for a deduplication operation; and
in response to determining that the data should be deduplicated, determine whether the data is duplicative.
2. The apparatus of claim 1 , wherein to determine whether the data is duplicative, the controller is further configured to:
calculate a fingerprint for the data based, at least in part, on a type of the memory;
compare the fingerprint to values in a metadata table; and
further determine whether the data is duplicative responsive to the fingerprint matching a value in the metadata table.
3. The apparatus of claim 2 , wherein the controller is further configured to write the data to the memory responsive to the fingerprint not matching any values in the metadata table.
4. The apparatus of claim 2 , wherein the memory has an access time for a unit of data that is less than the time required to calculate a fingerprint for the unit of data, and wherein to calculate the fingerprint for the data, the controller is further configured to calculate a weak fingerprint for the data.
5. The apparatus of claim 4 , wherein to further determine whether the data is duplicative, the controller is further configured to:
compare the data to stored data from the memory;
responsive to determining that the data matches any of the stored data, update a logical to physical table with the weak fingerprint for the data; and
responsive to determining that the data does not match any of the stored data, write the data to the memory.
6. The apparatus of claim 2 , wherein the memory has an access time for a unit of data that is greater than the time required to calculate a fingerprint for the unit of data, and wherein to calculate the fingerprint for the data, the controller is further configured to:
calculate a strong fingerprint when a workload deduplication ratio is above or equal to a threshold; and
calculate a weak fingerprint when the workload deduplication ratio is below the threshold.
7. The apparatus of claim 6 , wherein the workload deduplication ratio is below the threshold and wherein to further determine whether the data is duplicative, the controller is further configured to:
calculate a second strong fingerprint;
compare the second strong fingerprint to the values in the metadata table;
update a logical to physical table based on the strong fingerprint if the second strong fingerprint matches any of the values in the metadata table; and
write the data to the memory if the second strong, fingerprint does not match any of the values in the metadata table.
8. The apparatus of claim 1 , wherein the status information of the controller comprises CPU utilization, temperature data, bandwidth, number of input/output operations per second, latency, buffer utilization, and pre-fetch status of the controller.
9. The apparatus of claim 1 , wherein the media characteristics of the memory comprise a latency and a residual bit error rate (RBER) of the memory.
10. The apparatus of claim 1 , wherein the controller is further configured to: in response to determining; that the data should not be deduplicated, write the data to the memory.
11. The apparatus of claim 1 , wherein the controller is further configured to determine whether the data should be deduplicated based, further in part, on a workload status of the data to be written.
12. The apparatus of claim 1 , wherein the controller is further configured to utilize a machine learning algorithm to determine whether the data to be written to the memory should be deduplicated.
13. An apparatus comprising:
a memory including a first memory having an access time for a unit of data that is greater than the time required to calculate a fingerprint for the unit of data, and a second memory having an access time for a unit of data that is less than the time required to calculate a fingerprint for the unit of data; and
a controller configured to:
determine whether data to be written to the memory should be deduplicated based, at least in part, on status information of the controller and media characteristics of the memory, wherein the status information of the controller indicates a level of resources available for a deduplication operation; and
in response to determining that the data should be deduplicated, determine whether the data is duplicative based on whether the data to be written to the memory is to be written to the first memory or the second memory.
14. The apparatus of claim 13 , wherein to determine whether the data is duplicative, the controller is further configured to:
calculate a fingerprint for the data based, at least in part, on a type of the memory;
compare the fingerprint to values in a metadata table; and
further determine whether the data is duplicative responsive to the fingerprint matching a value in the metadata table.
15. The apparatus of claim 13 , wherein the status information of the controller comprises CPU utilization, temperature data, bandwidth, number of input/output operations per second, latency, buffer utilization, and pre-fetch status of the controller.
16. The apparatus of claim 13 , wherein the media characteristics of the memory comprise a latency and a residual bit error rate (RBER) of the memory.
17. A method comprising:
determining whether data to be written to a memory should be deduplicated based, at least in part, on status information of a controller and media characteristics of the memory, wherein the status information of the controller indicates a level of resources available for a reduplication operation; and
in response to determining that the data should be deduplicated, determining whether the data is duplicative.
18. The method of claim 17 , wherein determining whether the data is duplicative comprises:
calculating a fingerprint for the data based, at least in part, on a type of the memory;
comparing the fingerprint to values in a metadata table; and
further determining whether the data is duplicative responsive to the fingerprint matching a value in the metadata table.
19. The method of claim 18 , wherein the status information of the controller comprises CPU utilization, temperature data, bandwidth, number of input/output operations per second, latency, utilization, and pre-fetch status of the controller.
20. The method of claim 18 , wherein the media characteristics of the memory comprise a latency and a residual bit error rate (RBER) of the memory.