Using persistent memory technology as a host-side storage tier for clustered/distributed file systems, managed by storage appliance
View Patent ↗Embodiments are described for a multi-node file system, such as a clustered or distributed file system, with a file system buffer cache and an additional host-side tier non-volatile storage cache such as 3DXP storage. Cache coherency can be maintained by one of three models: (i) host-side tier management, (ii) file system management, or (iii) storage array management. performing a storage tier-specific file system action in a file system that comprises a namespace that spans multiple tiers of storage.
1. A computer-implemented method of managing cache coherency in a multi-node file system, each node having a file system buffer cache with cache coherency managed by an operating system of the node and a host-side cache with cache coherency managed by a storage device that is coupled to the nodes in the multi-node file system, the method comprising:
receiving, by the storage device, an indication of a read request for a copy of a file system object (FSO) at a second node of the multi-node file system, the read request received from an application at a file system buffer cache of the second node and propagated to a host-side cache of the second node;
receiving, by the storage device, an indication that a host-side cache of a first node within the multi-node file system contains a most recent copy of the FSO within a plurality of nodes of the multi-node file system;
instructing, by the storage device, the host-side cache of the first node to write the FSO to the storage device; and
receiving, by the storage device, the FSO and storing the FSO in the storage device,
wherein, for each node in the multi-node file system, the file system buffer cache is implemented in volatile random access memory (RAM), the host-side cache is implemented in high-speed non-volatile RAM, and the host-side cache is distinct from the file system buffer cache.
2. The method of claim 1 , wherein the coherency of a host-side cache of each node in the multi-node file system is managed by the storage device, including the storage device monitoring the host-side cache of each node in the multi-node file system, to determine whether a host-side cache read or write operation has been performed, wherein the storage device comprises a storage appliance.
3. The method of claim 1 , further comprising:
sending, by the storage device, an invalidate FSO message to the plurality of nodes of the multi-node file system, other than the first node, wherein the invalidate FSO message instructs each of the plurality of nodes, other than the first node, to invalidate a copy of the FSO held in the host-side cache of the node, if a copy of the FSO is present in the host-side cache.
4. The method of claim 1 , further comprising:
sending, by the storage device, the copy of the FSO received from the host-side cache of the first node, to the host-side cache of the second node.
5. The method of claim 4 , further comprising:
in response to the storage device receiving a read request for the FSO from a third node, before receiving an indication of invalidation of the FSO:
instructing the host-side cache of the first node or the second node to move a copy of the FSO to the host-side cache of the third node.
6. A non-transitory computer-readable medium programmed with executable instructions that, when executed by a processing system having at least one hardware processor, perform operations for managing cache coherency in a multi-node file system, each node having a file system buffer cache with cache coherency managed by an operating system of the node and a host-side cache with cache coherency managed by a storage device that is coupled to the nodes in the multi-node file system, the operations comprising:
receiving, by the storage device, an indication of a read request for a copy of a file system object (FSO) at the second node of a multi-node file system, the read request received from an application at a file system buffer cache of the second node and propagated to a host-side cache of the second node;
receiving, by the storage device, an indication that a host-side cache of a first node within the multi-node file system contains a most recent copy of the FSO within a plurality of nodes of the multi-node file system;
instructing, by the storage device, the host-side cache of the first node to write the FSO to the storage device; and
receiving, by the storage device, the FSO and storing the FSO in the storage device,
wherein, for each node in the multi-node file system, the file system buffer cache is implemented in volatile random access memory (RAM), the host-side cache is implemented in high-speed non-volatile RAM, and the host-side cache is distinct from the file system buffer cache.
7. The medium of claim 6 , wherein the coherency of a host-side cache of each node in the multi-node file system is managed by the storage device, including the storage device monitoring the host-side cache of each node in the multi-node file system, to determine whether a host-side cache read or write operation has been performed, wherein the storage device comprises a storage appliance.
8. The medium of claim 6 , the operations further comprising:
sending, by the storage device, an invalidate FSO message to the plurality of nodes of the multi-node file system, other than the first node, wherein the invalidate FSO message instructs each of the plurality of nodes, other than the first node, to invalidate a copy of the FSO held in the host-side cache of the node, if a copy of the FSO is present in the host-side cache.
9. The medium of claim 6 , the operations further comprising:
sending, by the storage device, the copy of the FSO received from the host-side cache of the first node, to the host-side cache of the second node.
10. The medium of claim 9 , the operations further comprising:
in response to the storage device receiving a read request for the FSO from a third node, before receiving an indication of invalidation of the FSO:
instructing the host-side cache of the first node or the second node to move a copy of the FSO to the host-side cache of the third node.
11. A system comprising:
a processing system having at least one hardware processor, the processing system coupled to a memory programmed with executable instructions that, when executed by the processing system, perform operations for managing cache coherency in a multi-node file system, each node having a file system buffer cache with cache coherency managed by an operating system of the node and a host-side cache with cache coherency managed by a storage device that is coupled to the nodes in the multi-node file system, the operations comprising:
receiving, by the storage device, an indication of a read request for a copy of a file system object (FSO) at the second node of a multi-node file system, the read request received from an application at a file system buffer cache of the second node and propagated to a host-side cache of the second node;
receiving, by the storage device, an indication that a host-side cache of a first node within the multi-node file system contains a most recent copy of the FSO within a plurality of nodes of the multi-node file system;
instructing, by the storage device, the host-side cache of the first node to write the FSO to the storage device; and
receiving, by the storage device, the FSO and storing the FSO in the storage device,
wherein, for each node in the multi-node file system, the file system buffer cache is implemented in volatile random access memory (RAM), the host-side cache is implemented in high-speed non-volatile RAM, and the host-side cache is distinct from the file system buffer cache.
12. The system of claim 11 , wherein the coherency of a host-side cache of each node in the multi-node file system is managed by the storage device, including the storage device monitoring the host-side cache of each node in the multi-node file system, to determine whether a host-side cache read or write operation has been performed, wherein the storage device comprises a storage appliance.
13. The system of claim 11 , the operations further comprising:
sending, by the storage device, an invalidate FSO message to the plurality of nodes of the multi-node file system, other than the first node, wherein the invalidate FSO message instructs each of the plurality of nodes, other than the first node, to invalidate a copy of the FSO held in the host-side cache of the node, if a copy of the FSO is present in the host-side cache.
14. The system of claim 11 , the operations further comprising:
sending, by the storage device, the copy of the FSO received from the host-side cache of the first node, to the host-side cache of the second node.
15. The system of claim 14 , the operations further comprising:
in response to the storage device receiving a read request for the FSO from a third node, before receiving an indication of invalidation of the FSO:
instructing the host-side cache of the first node or the second node to move a copy of the FSO to the host-side cache of the third node.