Write performance in fault-tolerant clustered storage systems
Embodiments of the invention relate to supporting transaction data committed to a stable storage. Committed data in the cluster is stored in the persistent cache layer and replicated and stored in the cache layer of one or more secondary nodes. One copy is designated as a master copy and all other copies are designated as replica, with an exclusive write lock assigned to the master and a shared write lock extended to the replica. An acknowledgement of receiving the data is communicated following confirmation that the data has been replicated to each node designated to receive the replica. Managers and a director are provided to support management of the master copy and the replicas within the file system, including invalidation of replicas, fault tolerance associated with failure of a node holding a master copy, recovery from a failed node, recovered of the file system from a power failure, and transferring master and replica copies within the file system.
1. A method comprising:
integrating a stable memory layer with a page cache layer in a file system to temporarily hold committed data in distributed non-volatile memory of nodes in a cluster;
in response to receiving a synchronous write transaction in the file system, placing data associated with the received write transaction in the page cache layer and replicating the received data within the page cache layer of one or more remote nodes in the cluster;
distinguishing between a master copy of the received data and a replica of the received data, including applying existing cache policies to the master copy of the received data; and
in response to flushing the master copy to persistent storage, invalidating each replica on the one or more remote nodes.
2. The method of claim 1 , further comprising the master copy holding an exclusive cluster-wide write lock and the replica holding a shared cluster-wide write lock.
3. The method of claim 2 , further comprising maintaining fault tolerance in response to a failure of a node holding the master copy, including electing a remote node to acquire the master copy by acquiring the exclusive cluster-wide write lock for the data.
4. The method of claim 3 , further comprising recovering content on the recovered failed node in response to the recovery of the failed node, including reading data from non-volatile memory of a surviving node, wherein the surviving node is selected from the group consisting of: an available node and the master node.
5. The method of claim 2 , further comprising transferring designation of the master copy to a requestor node, including revoking the exclusive write lock from the master copy, and transferring the exclusive write lock to the requestor node.
6. The method of claim 1 , further comprising in response to recovery of the file system from a power failure, recovering data from non-volatile memory content on each node, and identifying master and replica copies from a characteristic of data byte-range and validating master and replica copies by re-acquiring cluster-wide write locks.
7. A method comprising:
integrating a stable memory layer with a page cache layer in a file system to temporarily hold data committed to a stable storage in non-volatile memory of nodes in a cluster;
receiving a synchronous write transaction in the file system, including placing data associated with the transaction in the page cache layer of the file system and replicating received data within the page cache layer;
maintaining a master copy and a replica of the received data, the master copy having distinguishing characteristics from the replica; and
applying existing cache policies to the master copy of the received data; and
invalidating the replica following flushing of the master copy to persistent storage.
8. The method of claim 7 , further comprising the master copy holding an exclusive cluster-wide write lock and the replica holding a shared cluster-wide write lock.
9. The method of claim 7 , further comprising recovering data from non-volatile memory content on each node, identifying master and replica copies from a characteristic of data byte-range, and validating master and replica copies by re-acquiring cluster-wide write locks, responsive to recovery from a power failure.