Using persistent memory technology as a host-side storage tier for clustered/distributed file systems, managed by host-side tier
Embodiments are described for a multi-node file system, such as a clustered or distributed file system, with a file system buffer cache and an additional host-side tier non-volatile storage cache such as 3DXP storage. Cache coherency can be maintained by one of three models: (i) host-side tier management, (ii) file system management, or (iii) storage array management. performing a storage tier-specific file system action in a file system that comprises a namespace that spans multiple tiers of storage.
1. A computer-implemented method for managing cache coherency of a multi-node file system sharing a storage system, each node having a file system buffer cache and a host-side cache, comprising:
receiving, by a host-side cache on a first node of the multi-node file system, a copy of a file system object (FSO) from a file system buffer cache on the first node;
storing the copy of the FSO in the host-side cache on the first node;
sending, by the first node, a message to a second and a third node to invalidate a copy of the FSO in a host-side cache of each of the second and third nodes in response to the storing of the copy of the FSO in the host-side cache on the first node from the buffer cache on the first node; and
sending a copy of the FSO from the host-side cache of the first node to at least the host-side cache of the second node independently of the shared storage system;
wherein each of the first, second, and third nodes comprise host-side cache coherency logic, and the coherency logic of the host-side cache of the first, second, and third nodes communicate with each other to implement coherency logic for the combined first, second, and third node host-side caches.
2. The method of claim 1 , wherein the host-side cache comprises a non-volatile storage, distinct from the file system buffer cache.
3. The method of claim 1 , wherein cache coherency logic of the host-side cache of the first node is independent of cache coherency logic of the file system buffer cache of the first node.
4. The method of claim 1 , wherein the sending of the copy of the FSO from the host-side cache of the first node to the host-side cache of the second node is in response to a request from the second node.
5. The method of claim 1 , wherein the copy of the FSO received from the file system buffer cache of the first node was received in response to a read request of an outdated copy of the FSO.
6. The method of claim 1 , further comprising flushing the received copy of the FSO from the host-side cache of the first node to the storage system in response to an event, wherein the event comprises one of:
a receipt of a flush host-side cache message,
the FSO has not been read for a predetermined period of time,
storage available in the host-side cache of the first node is below a threshold minimum value, or
receipt of a request to flush pending writes has been received from the storage system.
7. A non-transitory computer-readable medium programmed with executable instructions that, when executed by a processing system having at least one hardware processor, perform operations for managing cache coherency of a multi-node file system sharing a storage system, each node having a file system buffer cache and a host-side cache, comprising:
receiving, by a host-side cache on a first node of the multi-node file system, a copy of a file system object (FSO) from a file system buffer cache on the first node;
storing the copy of the FSO in the host-side cache on the first node;
sending, by the first node, a message to a second and a third node to invalidate a copy of the FSO in a host-side cache of each of the second and third nodes in response to the storing of the copy of the FSO in the host-side cache on the first node from the buffer cache on the first node; and
sending a copy of the FSO from the host-side cache of the first node to at least the host-side cache of the second node independently of the shared storage system;
wherein each of the first, second, and third nodes comprise host-side cache coherency logic, and the coherency logic of the host-side cache of the first, second, and third nodes communicate with each other to implement coherency logic for the combined first, second, and third node host-side caches.
8. The medium of claim 7 , wherein the host-side cache comprises a non-volatile storage, distinct from the file system buffer cache.
9. The medium of claim 7 , wherein cache coherency logic of the host-side cache of the first node is independent of cache coherency logic of the file system buffer cache of the first node.
10. The medium of claim 7 , wherein the sending of the copy of the FSO from the host-side cache of the first node to the host-side cache of the second node is in response to a request from the second node.
11. The medium of claim 7 , wherein the copy of FSO received from the file system buffer cache of the first node was received in response to a read request of an outdated copy of the FSO.
12. The medium of claim 7 , further comprising flushing the received copy of the FSO from the host-side cache of the first node to the storage system in response to an event, wherein the event comprises one of:
a receipt of a flush host-side cache message,
the FSO has not been read for a predetermined period of time,
storage available in the host-side cache of the first node is below a threshold minimum value, or
receipt of a request to flush pending writes has been received from the storage system.
13. A system comprising:
a processing system having at least one hardware processor, the processing system coupled to a memory programmed with executable instructions that, when executed by the processing system, perform operations for managing cache coherency of a multi-node file system sharing a storage system, each node having a file system buffer cache and a host-side cache, comprising:
receiving, by a host-side cache on a first node of the multi-node file system, a copy of a file system object (FSO) from a file system buffer cache on the first node;
storing the copy of the FSO in the host-side cache on the first node;
sending, by the first node, a message to a second and a third node to invalidate a copy of the FSO in a host-side cache of each of the second and third nodes in response to the storing of the copy of the FSO in the host-side cache on the first node from the buffer cache on the first node; and
sending a copy of the FSO from the host-side cache of the first node to at least the host-side cache of the second node independently of the shared storage system;
wherein each of the first, second, and third nodes comprise host-side cache coherency logic, and the coherency logic of the host-side cache of the first, second, and third nodes communicate with each other to implement coherency logic for the combined first, second, and third node host-side caches.
14. The system of claim 13 , wherein the host-side cache comprises a non-volatile storage, distinct from the file system buffer cache.
15. The system of claim 13 , wherein cache coherency logic of the host-side cache of the first node is independent of cache coherency logic of the file system buffer cache of the first node.
16. The system of claim 13 , wherein the sending of the copy of the FSO from the host-side cache of the first node to the host-side cache of the second node is in response to a request from the second node.
17. The system of claim 13 , wherein the copy of the FSO received from the file system buffer cache of the first node was received in response to a read request of an outdated copy of the FSO.
18. The system of claim 13 , further comprising flushing the received copy of the FSO from the host-side cache of the first node to the storage system in response to an event, wherein the event comprises one of:
a receipt of a flush host-side cache message,
the FSO has not been read for a predetermined period of time,
storage available in the host-side cache of the first node is below a threshold minimum value, or
receipt of a request to flush pending writes has been received from the storage system.