Intelligent deduplication in storage system based on application IO tagging
Host computers running applications that store data on a block-based storage system such as a SAN provide hints that differentiate IO data based on which application generated the IO. The hints may include tags that are associated with IO commands sent to the block-based storage system. Each host application is associated with a unique identifier that is placed in the tag. Application name-to-identifier mappings may be sent from the hosts to the block-based storage system. Per-identifier/application deduplication statistics are maintained by the block-based storage system and shared with other block-based storage system. Deduplication is disabled or de-emphasized for IO data generated by applications with statistically low deduplication ratios.
1. A method comprising:
in a block-based storage system:
receiving an input-output (IO) command to write data to a logical device that is maintained by the block-based storage system;
receiving a hint that identifies an application that generated the IO command; and
selectively performing deduplication of the data based on the application identified by the hint.
2. The method of claim 1 wherein receiving the hint comprises receiving a unique application identifier in a tag.
3. The method of claim 2 comprising updating deduplication statistics associated with the unique application identifier.
4. The method of claim 3 comprising updating deduplication enablement settings based on the deduplication statistics.
5. The method of claim 4 comprising sending the deduplication statistics to another block-based storage system.
6. The method of claim 4 comprising updating deduplication enablement settings based on deduplication statistics received from another block-based storage system.
7. The method of claim 2 comprising:
in a host:
obtaining the unique application identifier in response to generation of the IO command by the application, wherein the application runs on the host.
8. The method of claim 7 comprising, in the host, sending the unique application identifier to the block-based storage system in a tag associated with the IO command.
9. The method of claim 2 comprising:
in a host:
creating the unique application identifier in response to generation of the IO command by the application, wherein the application runs on the host; and
providing a mapping between the application and the unique application identifier to the block-based storage system.
10. An apparatus comprising:
a block-based storage system comprising:
a plurality of compute nodes;
an array of drives managed by the compute nodes; and
computer program code comprising:
instructions that receive an input-output (IO) command to write data to a logical device that is maintained by the block-based storage system;
instructions that receive a hint that identifies an application that generated the IO command; and
instructions that selectively perform deduplication of the data based on the application identified by the hint.
11. The apparatus of claim 10 wherein the instructions that receive the hint comprise instructions that receive a unique application identifier in a tag.
12. The apparatus of claim 11 comprising instructions that update deduplication statistics associated with the unique application identifier.
13. The apparatus of claim 12 comprising instructions that update deduplication enablement settings based on the deduplication statistics.
14. The apparatus of claim 13 comprising instructions that send the deduplication statistics to another block-based storage system.
15. The apparatus of claim 13 comprising instructions that update deduplication enablement settings based on deduplication statistics received from another block-based storage system.
16. The apparatus of claim 11 comprising:
a host computer comprising processor resources, memory resources, and computer program code comprising instructions that obtain the unique application identifier in response to generation of the IO command by the application, wherein the application runs on the host.
17. The apparatus of claim 16 comprising, in the host, instructions that send the unique application identifier to the block-based storage system in a tag associated with the IO command.
18. The apparatus of claim 11 comprising:
a host computer comprising processor resources, memory resources, and computer program code comprising instructions that create the unique application identifier in response to generation of the IO command by the application, wherein the application runs on the host and provide a mapping between the application and the unique application identifier to the block-based storage system.