File system inline fine grained tiering
View Patent ↗A technique for managing storage tiering in a data storage apparatus operates at a fine level of granularity in response to write requests as blocks are allocated for writing. As write requests arrive or are otherwise processed by the data storage apparatus, the data storage apparatus identifies a quality of service to be provided for satisfying each write request and allocates a set of blocks from storage devices of storage tiers that provide the identified quality of service. The data storage apparatus may then store the information specified in the write request in the newly allocated blocks.
1. A method of storing data in different storage tiers of a data storage apparatus, the method comprising:
receiving, via an IO stack of a storage processor of the data storage apparatus, a request to write a set of information in the data storage apparatus;
obtaining, in response to receiving the request via the IO stack, a data element designating a quality of service to be provided for storing the set of information; and
allocating a set of blocks from one or more storage devices of a storage tier that provides the quality of service designated in the data element read in response to receiving the request,
wherein the quality of service designated in the data element is a first quality of service, and wherein the method further comprises:
receiving, via the IO stack of the storage processor of the data storage apparatus, a second request to write a second set of information in the data storage apparatus;
obtaining, in response to receiving the second request via the IO stack, a second data element designating a second quality of service to be provided for storing the second set of information; and
allocating a second set of blocks from one or more storage devices of a second storage tier that provide the second quality of service designated in the second data element read in response to receiving the second request,
wherein the set of information includes file data to be written to a file of a file system, and
wherein the second set of information includes metadata pertaining to the file of the file system,
wherein the request to write the set of information is directed to a file of an upper-deck file system of the data storage apparatus,
wherein the method further comprises mapping the upper-deck file system to a file of an internal, lower-deck file system of the data storage apparatus, the file of the lower-deck file system providing a complete representation of the upper-deck file system and including both data and metadata of the upper-deck file system,
wherein obtaining the data element designating the quality of service to be provided for storing the set of information is performed by a manager of the upper-deck file system,
wherein the method further comprises conveying the data element, via the IO stack, to a manager of the lower-deck file system, the manager of the lower-deck file system thereby receiving the data element via the IO stack, and
wherein allocating the set of blocks is preformed in response to the manager of the lower-deck file system reading the data element received from the manager of the upper-deck file system and applying the data element to select the storage tier that provides the quality of service designated in the data element.
2. The method of claim 1 , wherein obtaining the data element includes:
accessing a policy that specifies different qualities of service for different types of information and/or circumstances;
identifying, a quality of service specified by the policy for the set of information and/or current circumstances; and
generating the data element to designate the quality of service specified by the policy.
3. The method of claim 2 , wherein the set of information includes data to be written to a file of the upper-deck file system, wherein the policy specifies a quality of service for storing file data, and wherein identifying the quality of service specified by the policy includes identifying the quality of service for storing file data from the policy.
4. The method of claim 2 , wherein the set of information includes data to be written to a particular range of a file of the upper-deck file system, wherein the policy specifies a quality of service for file data of the particular range, and wherein identifying the quality of service specified by the policy includes identifying the quality of service specified for the particular range of file data from the policy.
5. The method of claim 2 , wherein the set of information includes metadata pertaining to a file of the upper-deck file system, wherein the policy specifies a quality of service for file metadata, and wherein identifying the quality of service specified by the policy includes identifying the quality of service specified for file metadata from the policy.
6. The method of claim 5 , wherein the set of information pertains to an inode of the file of the upper-deck file system, wherein the policy specifies a quality of service for inodes, and wherein identifying the quality of service specified by the policy includes identifying the quality of service specified for inodes from the policy.
7. The method of claim 5 , wherein the set of information pertains to at least one indirect block of the file of the upper-deck file system, wherein the policy specifies a quality of service for indirect blocks, and wherein identifying the quality of service specified by the policy includes identifying the quality of service specified for indirect blocks from the policy.
8. The method of claim 2 , wherein the set of information includes data of a virtual hard disk (VHD) stored in a file of the upper-deck file system, wherein the policy specifies a quality of service for VHD data, and wherein identifying the quality of service specified by the policy includes identifying the quality of service specified for VHD data from the policy.
9. The method of claim 2 , wherein the set of information includes data of a virtual memory disk (VMDK) stored in a file of the upper-deck file system, wherein the policy specifies a quality of service for VMDK data, and wherein identifying the quality of service specified by the policy includes identifying the quality of service specified for VMDK data from the policy.
10. The method of claim 1 , wherein the request to write the set of information is directed to a file of a file system of the data storage apparatus, and wherein the method further comprises, after reading the data element designating the quality of service to be used for storing the set of information, checking a data structure of the file system to determine whether any free blocks are available to provide the quality of service designated in the data element read from the request.
11. The method of claim 1 , wherein the request to write the set of information is directed to a file of a file system of the data storage apparatus, and wherein obtaining the data element designating the quality of service includes accessing metadata for the file stored in the file system to receive a designation of quality of service from the metadata.
12. The method of claim 11 , wherein obtaining the data element designating the quality of service includes accessing an inode for the file stored in the file system to receive a designation of quality of service from the inode.
13. The method of claim 11 , wherein obtaining the data element designating the quality of service includes accessing an indirect block for the file stored in the file system to receive a designation of quality of service for a particular range of the file from the indirect block.
14. The method of claim 1 , wherein obtaining the data element designating the quality of service to be provided for storing the set of information includes, by the manager of the upper-deck file system:
sending a message to a tiering policy requesting a quality of service to be provided for the set of information;
in response to the tiering policy receiving the message, receiving the data element from the tiering policy; and
applying the data element to the request as a tag,
wherein the request and the applied tag propagate through the IO stack to the manager of the lower-deck file system.
15. The method of claim 1 , wherein obtaining the data element designating the quality of service to be provided for storing the set of information includes, by the manager of the upper-deck file system:
checking metadata of the upper-deck file system to identify a quality of service to be provided for the set of information;
in response to checking the metadata, receiving the data element; and
applying the data element to the request as a tag,
wherein the request and the applied tag propagate through the IO stack to the manager of the lower-deck file system.
16. A data storage apparatus, comprising:
a storage processor; and
multiple storage devices, coupled to the storage processor, for storing data of at least one host,
wherein the storage processor includes a set of processing units coupled to memory, the memory storing executable instructions, which when executed by the set of processing units cause the set of processing units to perform a method of storing data in different storage tiers of the data storage apparatus, wherein the method comprises:
receiving, via an IO stack of the storage processor of the data storage apparatus, a request to write a set of information in the data storage apparatus;
obtaining, in response to receiving the request via the IO stack, a data element designating a quality of service to be provided for storing the set of information; and
allocating a set of blocks from one or more storage devices of a storage tier that provides the quality of service designated in the data element read in response to receiving the request,
wherein the request to write the set of information is directed to a file of an upper-deck file system of the data storage apparatus, the upper-deck file system mapped to a file in a lower-deck file system of the data storage apparatus, a manager of the upper-deck file system obtaining the data element designating the quality of service, the IO stack conveying the data element to a manager of the lower-deck file system, and wherein allocating the set of blocks is performed in response to the manager of the lower-deck file system applying the data element to select the storage tier that provides the quality of service designated in the data element.
17. The data storage apparatus of claim 16 , wherein the file of the lower-deck file system provides a complete representation of the upper-deck file system and includes both data and metadata of the upper-deck file system.
18. The data storage apparatus of claim 16 , wherein the quality of service designated in the data element is a first quality of service, and wherein the method further comprises:
receiving, via the IO stack of the storage processor of the data storage apparatus, a second request to write a second set of information in the data storage apparatus;
obtaining, in response to receiving the second request via the IO stack, a second data element designating a second quality of service to be provided for storing the second set of information; and
allocating a second set of blocks from one or more storage devices of a second storage tier that provide the second quality of service designated in the second data element read in response to receiving the second request,
wherein the set of information includes file data to be written to a file of a file system, and
wherein the second set of information includes metadata pertaining to the file of the file system.
19. A non-transitory computer readable medium including instructions which, when executed by a set of processing units of a storage processor of a data storage apparatus, cause the set of processing units to perform a method of storing data in different storage tiers of the data storage apparatus, wherein the method comprises:
receiving, via an IO stack of the storage processor of the data storage apparatus, a request to write a set of information in the data storage apparatus;
obtaining, in response to receiving the request via the IO stack, a data element designating a quality of service to be provided for storing the set of information; and
allocating a set of blocks from one or more storage devices of a storage tier that provides the quality of service designated in the data element read in response to receiving the request,
wherein the request to write the set of information is directed to a file of an upper-deck file system of the data storage apparatus, the upper-deck file system mapped to a file in a lower-deck file system of the data storage apparatus, a manager of the upper-deck file system obtaining the data element designating the quality of service, the IO stack conveying the data element to a manager of the lower-deck file system, and wherein allocating the set of blocks is performed in response to the manager of the lower-deck file system applying the data element to select the storage tier that provides the quality of service designated in the data element.
20. The non-transitory computer readable medium of claim 19 , wherein obtaining the data element includes:
accessing a policy that specifies different qualities of service for different types of information and/or circumstances;
identifying, a quality of service specified by the policy for the set of information and/or current circumstances; and
generating the data element to designate the quality of service specified by the policy.
21. The non-transitory computer readable medium of claim 19 , wherein the quality of service designated in the data element is a first quality of service, and wherein the method further comprises:
receiving, via the IO stack of the storage processor of the data storage apparatus, a second request to write a second set of information in the data storage apparatus;
obtaining, in response to receiving the second request via the IO stack, a second data element designating a second quality of service to be provided for storing the second set of information; and
allocating a second set of blocks from one or more storage devices of a second storage tier that provide the second quality of service designated in the second data element read in response to receiving the second request,
wherein the set of information includes file data to be written to a file of a file system, and
wherein the second set of information includes metadata pertaining to the file of the file system.