Per-service storage of attributes
A method performed by a data storage apparatus of maintaining filesystem object attributes in connection with a plurality of drivers is provided. Each driver of the plurality is configured to operate on a respective set of filesystem objects. The method includes (a) for each driver of the plurality, creating a respective driver-specific storage volume; (b) storing, within each driver-specific storage volume, attributes specific to respective filesystem objects operated on by the driver of that driver-specific storage volume; and (c) while operating a particular driver of the plurality on a particular filesystem object, using values of the stored attributes specific to that particular filesystem object operated on by that particular driver. An apparatus, system, and computer program product for performing a similar method are also provided.
1. A method performed by a data storage apparatus of maintaining filesystem object attributes in connection with a plurality of drivers, each driver of the plurality being configured to operate on a respective set of filesystem objects, the method comprising:
for each driver of the plurality, creating a respective driver-specific storage volume, each driver of the plurality of drivers being a software programming construct constructed and arranged to perform volume-based storage activities and providing a respective layer of operational processing, each driver-specific storage volume created for a different driver of the plurality of drivers in a one-to-one relationship;
storing, within each driver-specific storage volume, an indicator of each of the set of filesystem objects with which the respective driver is configured to operate and, for each indicated filesystem object, attributes specific to the respective filesystem object operated on by the driver of that driver-specific storage volume; and
while operating a particular driver of the plurality on a particular filesystem object, using values of the stored attributes specific to that particular filesystem object operated on by that particular driver,
wherein storing, within each driver-specific storage volume, attributes specific to respective filesystem objects operated on by the driver of that driver-specific storage volume includes providing first and second B+ trees, the first B+ tree mapping identifiers of user volumes to respective leaves, the leaves corresponding to respective user volumes and including pointers to respective attribute realms, the second B+ tree mapping the respective attribute realms to respective attribute values.
2. The method of claim 1 wherein:
each respective set of filesystem objects includes a user-accessible storage volume; and
creating the respective driver-specific storage volume for each driver of the plurality includes creating the driver-specific storage volumes as hidden storage volumes that are not accessible to users.
3. The method of claim 1 wherein storing attributes specific to respective filesystem objects within each driver-specific storage volume includes:
storing, in a primary data structure within that driver-specific storage volume, a first set of pointers that respectively point to secondary data structures, each pointer of the first set corresponding to a respective filesystem object operated on by the driver of that driver-specific storage volume; and
storing the secondary data structures pointed to by the first set of pointers within that driver-specific storage volume, each of the secondary data structures corresponding to a respective filesystem object operated on by the driver of that driver-specific storage volume, each secondary data structure storing the attributes specific to its corresponding filesystem object that are operated on by the driver of that driver-specific storage volume.
4. The method of claim 3 wherein the primary data structure is a B+ tree with keys based on filesystem objects and pointers to secondary data structures serving as values in key-value pairs.
5. The method of claim 4 wherein the B+ tree includes a plurality of nodes, each node occupying a single filesystem block.
6. The method of claim 4 wherein the keys based on filesystem objects are unique identifiers of filesystem objects that have been processed with non-cryptographic hashing.
7. The method of claim 4 wherein each secondary data structure is an embedded B+ tree with keys based on attributes of filesystem objects and values of those attributes serving as values in key-value pairs.
8. The method of claim 7 wherein the embedded B+ trees include a plurality of nodes, each node occupying a single filesystem block.
9. The method of claim 7 wherein the keys based on attributes of filesystem objects are locally-unique identifiers of attributes that have been processed with non-cryptographic hashing.
10. The method of claim 1 wherein the plurality of drivers are layered storage service drivers configured to send and receive storage operations according to a same interface shared with the other layered storage service drivers.
11. The method of claim 1 wherein:
a first driver of the plurality is a copy driver configured to copy data from a first filesystem object to a second filesystem object; and
a second driver of the plurality is a collator driver configured to:
receive a plurality of storage commands from at least one other driver of the plurality;
combine the plurality of storage commands into a combined storage command; and
issue the combined storage command to yet another driver of the plurality.
12. An apparatus comprising processing circuitry coupled to memory configured to store filesystem object attributes in connection with a plurality of drivers, each driver of the plurality being configured to operate on a respective set of filesystem objects, by:
for each driver of the plurality, creating a respective driver-specific storage volume, each driver of the plurality of drivers being a software programming construct constructed and arranged to perform volume-based storage activities and providing a respective layer of operational processing, each driver-specific storage volume created for a different driver of the plurality of drivers in a one-to-one relationship;
storing, within each driver-specific storage volume, an indicator of each of the set of filesystem objects with which the respective driver is configured to operate and, for each indicated filesystem object, attributes specific to the respective filesystem object operated on by the driver of that driver-specific storage volume; and
while operating a particular driver of the plurality on a particular filesystem object, using values of the stored attributes specific to that particular filesystem object operated on by that particular driver,
wherein storing, within each driver-specific storage volume, attributes specific to respective filesystem objects operated on by the driver of that driver-specific storage volume includes providing first and second B+ trees, the first B+ tree mapping identifiers of user volumes to respective leaves, the leaves corresponding to respective user volumes and including pointers to respective attribute realms, the second B+ tree mapping the respective attribute realms to respective attribute values.
13. A computer program product comprising a non-transitory storage medium storing instructions, which, when performed by computing circuitry of a data storage apparatus, cause the data storage apparatus to store filesystem object attributes in connection with a plurality of drivers, each driver of the plurality being configured to operate on a respective set of filesystem objects, by:
for each driver of the plurality, creating a respective driver-specific storage volume, each driver of the plurality of drivers being a software programming construct constructed and arranged to perform volume-based storage activities and providing a respective layer of operational processing, each driver-specific storage volume created for a different driver of the plurality of drivers in a one-to-one relationship;
storing, within each driver-specific storage volume, an indicator of each of the set of filesystem objects with which the respective driver is configured to operate and, for each indicated filesystem object, attributes specific to the respective filesystem object operated on by the driver of that driver-specific storage volume; and
while operating a particular driver of the plurality on a particular filesystem object, using values of the stored attributes specific to that particular filesystem object operated on by that particular driver,
wherein storing, within each driver-specific storage volume, attributes specific to respective filesystem objects operated on by the driver of that driver-specific storage volume includes providing first and second B+ trees, the first B+ tree mapping identifiers of user volumes to respective leaves, the leaves corresponding to respective user volumes and including pointers to respective attribute realms, the second B+ tree mapping the respective attribute realms to respective attribute values.
14. The computer program product of claim 13 wherein:
each respective set of filesystem objects includes a user-accessible storage volume; and
creating the respective driver-specific storage volume for each driver of the plurality includes creating the driver-specific storage volumes as hidden storage volumes that are not accessible to users.
15. The computer program product of claim 13 wherein storing attributes specific to respective filesystem objects within each driver-specific storage volume includes:
storing, in a primary data structure within that driver-specific storage volume, a first set of pointers that respectively point to secondary data structures, each pointer of the first set corresponding to a respective filesystem object operated on by the driver of that driver-specific storage volume; and
storing the secondary data structures pointed to by the first set of pointers within that driver-specific storage volume, each of the secondary data structures corresponding to a respective filesystem object operated on by the driver of that driver-specific storage volume, each secondary data structure storing the attributes specific to its corresponding filesystem object that are operated on by the driver of that driver-specific storage volume.
16. The computer program product of claim 15 wherein the primary data structure is a B+ tree with keys based on filesystem objects and pointers to secondary data structures serving as values in key-value pairs.
17. The computer program product of claim 16 wherein the B+ tree includes a plurality of nodes, each node occupying a single filesystem block.
18. The computer program product of claim 16 wherein the keys based on filesystem objects are unique identifiers of filesystem objects that have been processed with non-cryptographic hashing.
19. The computer program product of claim 16 wherein each secondary data structure is an embedded B+ tree with keys based on attributes of filesystem objects and values of those attributes serving as values in key-value pairs.
20. The computer program product of claim 19 wherein the embedded B+ trees include a plurality of nodes, each node occupying a single filesystem block.