Using persistent memory technology as a host-side storage tier for clustered/distributed file systems, managed by cluster file system
Embodiments are described for a multi-node file system, such as a clustered or distributed file system, with a file system buffer cache and an additional host-side tier non-volatile storage cache such as 3DXP storage. Cache coherency can be maintained by one of three models: (i) host-side tier management, (ii) file system management, or (iii) storage array management. performing a storage tier-specific file system action in a file system that comprises a namespace that spans multiple tiers of storage.
1. A computer-implemented method for managing cache coherency of a multi-node file system, each node having a file system buffer cache and a host-side cache that is distinct from the file system buffer cache, the method comprising:
in response to receiving, by a file system buffer cache of a first node in the multi-node file system, a request to read a file system object (FSO) from an application, the multi-node file system sharing a storage device that is distinct from the file system buffer cache and the host-side cache of each node:
sending a message to a plurality of other nodes in the multi-node file system, indicating an intent to read the FSO, the message causing a second node of the plurality of other nodes in the multi-node file system with a most recent copy of the FSO stored in a file system buffer cache of the second node to flush the most recent copy of the FSO from the file system buffer cache of the second node to a host-side cache of the second node;
in response to the host-side cache of the second node storing the most recent copy of the FSO from the file system buffer cache of the second node, causing the second node to flush the most recent copy of the FSO from the host-side cache of the second node to the storage device;
reading, by the first node, the most recent copy of the FSO from the storage device and storing the most recent copy of the FSO in a host-side cache of the first node; and
providing the most recent copy of the FSO to fulfill the request, from the host-side cache of the first node by:
reading, by the first node, the most recent copy of the FSO stored in the host-side cache of the first node and storing the most recent copy of the FSO in the file system buffer cache of the first node, and
subsequently, sending the most recent copy of the FSO stored in the file system buffer cache of the first node to the application.
2. The method of claim 1 , wherein the host-side cache comprises a non-volatile storage.
3. The method of claim 1 , further comprising:
notifying each of the nodes in the multi-node file system when the most recent copy of the FSO has been flushed to the storage device.
4. The method of claim 1 , further comprising determining that the second node in the multi-node file system has the most recent copy of the FSO wherein the message is sent to the second node in response to determining that the second node has the most recent copy of the FSO.
5. The method of claim 1 , wherein the reading of the most recent copy of the FSO from the storage device and the providing of the most recent copy of the FSO is performed once the most recent copy of the FSO has been flushed to the storage device.
6. The method of claim 1 , wherein cache coherency logic in each node manages both the file system buffer cache of the node and the host-side cache of the node.
7. A non-transitory computer-readable medium programmed with executable instructions that, when executed by a processing system having at least one hardware processor, perform operations for managing cache coherency of a multi-node file system, each node having a file system buffer cache and a host-side cache that is distinct from the file system buffer cache, comprising:
in response to receiving, by a file system buffer cache of a first node in the multi-node file system, a request to read a file system object (FSO) from an application, the multi-node file system sharing a storage device that is distinct from the file system buffer cache and the host-side cache of each node:
sending a message to a plurality of other nodes in the multi-node file system, indicating an intent to read the FSO, the message causing a second node of the plurality of other nodes in the multi-node file system with a most recent copy of the FSO stored in the file system buffer cache of the second node to flush the most recent copy of the FSO from the file system buffer cache of the second node to a host-side cache of the second node;
in response to the host-side cache of the second node storing the most recent copy of the FSO from the file system buffer cache of the second node, causing the second node to flush the most recent copy of the FSO from the host-side cache of the second node to the storage device;
reading, by the first node, the most recent copy of the FSO from the storage device and storing the most recent copy of the FSO in a host-side cache of the first node; and
providing the most recent copy of the FSO to fulfill the request, from the host-side cache of the first node by:
reading, by the first node, the most recent copy of the FSO stored in the host-side cache of the first node and storing the most recent copy of the FSO in the file system buffer cache of the first node, and
subsequently, sending the most recent copy of the FSO stored in the file system buffer cache of the first node to the application.
8. The medium of claim 7 , wherein the host-side cache comprises a non-volatile storage.
9. The medium of claim 7 , further comprising:
notifying each of the nodes in the multi-node file system when the most recent copy of the FSO has been flushed to the storage device.
10. The medium of claim 7 , further comprising determining that the second node of the plurality of other nodes in the multi-node file system has the most recent copy of the FSO, wherein the message is sent to the second node in response to determining that the second node has the most recent copy of the FSO.
11. The medium of claim 7 , wherein the reading of the most recent copy of the FSO from the storage device and the providing of the most recent copy of the FSO is performed once the most recent copy of the FSO has been flushed to the storage device.
12. The medium of claim 7 , wherein cache coherency logic in each node manages both the file system buffer cache of the node and the host-side cache of the node.
13. A system comprising:
a processing system having at least one hardware processor, the processing system coupled to a memory programmed with executable instructions that, when executed by the processing system, perform operations for managing cache coherency of a multi-node file system, each node having a file system buffer cache and a host-side cache that is distinct from the file system buffer cache comprising:
in response to receiving, by a file system buffer cache of a first node in the multi-node file system, a request to read a file system object (FSO) from an application, the multi-node file system sharing a storage device that is distinct from the file system buffer cache and the host-side cache of each node:
sending a message to a plurality of other nodes in the multi-node file system, indicating an intent to read the FSO, the message causing a second node of the plurality of other nodes in the multi-node file system with a most recent copy of the FSO stored in a file system buffer cache of the second node to flush the most recent copy of the FSO from the file system buffer cache of the second node to a host-side cache of the second node;
in response to the host-side cache of the second node storing the most recent copy of the FSO from the file system buffer cache of the second node, causing the second node to flush the most recent copy of the FSO from the host-side cache of the second node to the storage device;
reading, by the first node, the most recent copy of the FSO from the storage device and storing the most recent copy of the FSO in a host-side cache of the first node; and
providing the most recent copy of the FSO to fulfill the request, from the host-side cache of the first node by:
reading, by the first node, the most recent copy of the FSO stored in the host-side cache of the first node and storing the most recent copy of the FSO in the file system buffer cache of the first node, and
subsequently, sending the most recent copy of the FSO stored in the file system buffer cache of the first node to the application.
14. The system of claim 13 , wherein the host-side cache comprises a non-volatile storage.
15. The system of claim 13 , further comprising:
notifying each of the nodes in the multi-node file system when the most recent copy of the FSO has been flushed to the storage device.
16. The system of claim 13 , further comprising determining that the second node in the multi-node file system has the most recent copy of the FSO, wherein the message is sent to the second node in response to determining that the second node has the most recent copy of the FSO.
17. The system of claim 13 , wherein the reading of the most recent copy of the FSO from the storage device and the providing of the most recent copy of the FSO is performed once the most recent copy of the FSO has been flushed to the storage device.
18. The system of claim 13 , wherein cache coherency logic in each node manages both the file system buffer cache of the node and the host-side cache of the node.