Block write cache replication model
Caching write input/output (I/O) operations in a replica-based storage system. A write I/O operation is received from a consumer, and a corresponding replica list is identified. A first replica set is selected from the replica list for caching the I/O operation, and a first log corresponding to the I/O operation is added to a primary ring buffer of the first replica set. When the first log cannot be replicated to a secondary ring buffer of the first replica set, a second replica set is selected from the replica list for caching the I/O operation. A second log corresponding to the I/O operation is added to a primary ring buffer of the second replica set. Once the second log has been replicated to a secondary ring buffer of the second replica set, the I/O operation is acknowledged to the consumer, and the second log is de-staged to a backing store.
1 . A method implemented in a computer system that includes a processor system, comprising:
receiving a write input/output (I/O) operation from a consumer;
identifying a replica list associated with the consumer, the replica list specifying a first replica set and a second replica set;
selecting the first replica set for caching the write I/O operation;
adding a first log corresponding to the write I/O operation to a primary ring buffer of the first replica set, the primary ring buffer of the first replica set being stored in the computer system;
determining that the first log cannot be replicated to a secondary ring buffer of the first replica set, the secondary ring buffer of the first replica set being stored in a first secondary computer system;
selecting the second replica set for caching the write I/O operation, based on determining that the first log cannot be replicated to the secondary ring buffer of the first replica set;
adding a second log corresponding to the write I/O operation to a primary ring buffer of the second replica set, the primary ring buffer of the second replica set being stored in the computer system;
determining that the second log has been replicated to a secondary ring buffer of the second replica set, the secondary ring buffer of the second replica set being in a second secondary computer system; and
based on determining that the second log has been replicated to the secondary ring buffer of the second replica set,
acknowledging the write I/O operation to the consumer; and
de-staging the second log to a backing store.
2 . The method of claim 1 , wherein the consumer is a virtual machine (VM) or a container executing in the computer system.
3 . The method of claim 2 , wherein de-staging the second log to the backing store comprises de-staging the second log to a virtual disk corresponding to the VM or the container.
4 . The method of claim 1 , wherein,
the primary ring buffer of the first replica set is stored in a first persistent memory in the computer system;
the primary ring buffer of the second replica set is stored in the first persistent memory in the computer system;
the secondary ring buffer of the first replica set is stored in a second persistent memory in the first secondary computer system; and
the secondary ring buffer of the second replica set is stored in a third persistent memory in the second secondary computer system.
5 . The method of claim 1 , wherein determining that the first log cannot be replicated to the secondary ring buffer of the first replica set comprises:
determining that the first log cannot be replicated to all secondary ring buffers of the first replica set.
6 . The method of claim 1 , wherein determining that the second log has been replicated to the secondary ring buffer of the second replica set comprises:
determining that the second log has been replicated to all secondary ring buffers of the second replica set.
7 . The method of claim 1 , wherein the replica list is associated with a plurality of consumers.
8 . The method of claim 7 , wherein a remote management service associates the plurality of consumers with the replica list.
9 . The method of claim 1 , wherein,
the write I/O operation is a first write I/O operation,
the replica list also specifies a third replica set, and
the method further comprises:
receiving a second write I/O operation from the consumer;
selecting the second replica set for caching the write I/O operation;
determining that the primary ring buffer of the second replica set is full;
selecting the third replica set for caching the write I/O operation, based on determining that the primary ring buffer of the second replica set is full;
adding a third log corresponding to the second write I/O operation to a primary ring buffer of the third replica set, the primary ring buffer of the third replica set being stored in the computer system;
determining that the third log has been replicated to all secondary ring buffers of the third replica set; and
based on determining that the third log has been replicated to all secondary ring buffers of the third replica set,
acknowledging the second write I/O operation to the consumer; and
de-staging the third log to the backing store.
10 . A method implemented in a computer system that includes a processor system, comprising:
receiving an election as a de-stage primary host for a replica set, the replica set comprising a primary ring buffer and one or more secondary ring buffers stored across a plurality of hosts; and
based on receiving the election as the de-stage primary host for the replica set,
identifying a ring buffer for the replica set that is stored in the computer system, the ring buffer comprising a plurality of logs replicated from the primary ring buffer at a different host of the plurality of hosts, each log corresponding to a different cached write input/output (I/O) request; and
de-staging the plurality of logs from the ring buffer to a backing store.
11 . The method of claim 10 , wherein receiving the election as the de-stage primary host for the replica set comprises receiving the election from a management service.
12 . The method of claim 10 , wherein receiving the election as the de-stage primary host for the replica set comprises receiving the election from one or more secondary hosts.
13 . The method of claim 10 , wherein the ring buffer is stored in a persistent memory in the computer system.
14 . The method of claim 10 , wherein,
receiving the election as the de-stage primary host for the replica set comprises receiving an election as a de-stage primary host for a plurality of replica sets, and
the method further comprises:
based on receiving the election as the de-stage primary host for the plurality of replica sets,
identifying a plurality of ring buffers stored in the computer system, each ring buffer corresponding to one of the plurality of replica sets and comprising a corresponding plurality of logs replicated from a corresponding primary ring buffer at a different host of the plurality of hosts, each log corresponding to a different cached write I/O request; and
de-staging the corresponding plurality of logs from each of the plurality of ring buffers to the backing store.
15 . The method of claim 10 , wherein the election as the de-stage primary host for the replica set is received after a failure of another host of the plurality of hosts to de-stage logs as a prior de-stage primary host.
16 . The method of claim 10 , wherein the method further comprises:
sending a notification to a management service after de-staging the plurality of logs from the ring buffer to the backing store.
17 . The method of claim 10 , wherein the method further comprises:
sending a notification to one or more of the plurality of hosts, after de-staging the plurality of logs from the ring buffer to the backing store.
18 . A computer system, comprising:
a processor system; and
a computer storage medium that stores computer-executable instructions that are executable by the processor system to at least:
receive a write input/output (I/O) operation from a consumer;
select a first replica set for caching the write I/O operation from a replica list;
add a first log corresponding to the write I/O operation to a primary ring buffer of the first replica set;
determine that the first log cannot be replicated to a secondary ring buffer of the first replica set;
select a second replica set for caching the write I/O operation from the replica list;
add a second log corresponding to the write I/O operation to a primary ring buffer of the second replica set;
determine that the second log has been replicated to a secondary ring buffer of the second replica set; and
based on determining that the second log has been replicated to the secondary ring buffer of the second replica set,
acknowledge the write I/O operation to the consumer; and
de-stage the second log to a backing store.
19 . The computer system of claim 18 , wherein,
the consumer is a virtual machine (VM) or a container executing in the computer system; and
de-staging the second log to the backing store comprises de-staging the second log to a virtual disk corresponding to the VM or the container.
20 . The computer system of claim 18 , wherein,
the primary ring buffer of the first replica set is stored in a first persistent memory in the computer system;
the primary ring buffer of the second replica set is stored in the first persistent memory in the computer system;
the secondary ring buffer of the first replica set is stored in a second persistent memory in a first secondary computer system; and
the secondary ring buffer of the second replica set is stored in a third persistent memory in a second secondary computer system.