System and method for parallel journaling in a storage cluster
A method, computer program product, and computing system for receiving, via a first node of a plurality of nodes, data for storage in a storage system. The data may be written, via the first node, to one or more data pages of a log buffer of the first node. One or more page descriptors associated with the one or more data pages may be generated via the first node. The one or more page descriptors may be sent to a second node concurrently with the writing of the data to the one or more data pages of the log buffer of the first node.
1. A computer-implemented method, executed on a computing device, comprising:
receiving, via a first node of a plurality of nodes, data for storage in a storage system;
writing, via the first node, the data to one or more data pages of a log buffer of the first node;
generating, via the first node, one or more page descriptors associated with the one or more data pages;
sending the one or more page descriptors to a second node concurrently with the writing of the data to the one or more data pages of the log buffer of the first node;
defining a last confirmed page descriptor associated with a last data page of the one or more data pages written to the log buffer of the first node;
sending the last confirmed page descriptor associated with the last data page written to the log buffer of the first node, to the second node;
determining that the first node has failed;
determining, via the second node, the last confirmed page descriptor associated with the last data page written to the log buffer of the first node; and
verifying, on the first node, one or more subsequent page descriptors based upon, at least in part, the last confirmed page descriptor associated with the last data page written to the log buffer of the first node.
2. The computer-implemented method of claim 1 , further comprising:
writing, via the second node, the one or more page descriptors to a page descriptor buffer of the second node.
3. The computer-implemented method of claim 2 , further comprising:
in response to writing the one or more page descriptors to the page descriptor buffer of the second node, sending an acknowledgement signal to the first node indicating that the one or more page descriptors are written to the page descriptor buffer of the second node.
4. The computer-implemented method of claim 3 , further comprising:
sending an acknowledgement signal to a computing device that sent the data to the storage system in response to:
determining that the data is written to the one or more data pages of the log buffer of the first node, and
receiving the acknowledgement signal from the second node indicating that the one or more page descriptors are written to the page descriptor buffer.
5. The computer-implemented method of claim 1 , further comprising:
defining one or more ranges of confirmed page descriptors associated with a plurality of data pages written to the log buffer of the first node; and
sending the one or more ranges of confirmed page descriptors associated with a plurality of data pages written to the log buffer of the first node, to the second node.
6. A computer program product residing on a non-transitory computer readable medium having a plurality of instructions stored thereon which, when executed by a processor, cause the processor to perform operations comprising:
receiving, via a first node of a plurality of nodes, data for storage in the storage system;
writing, via the first node, the data to one or more data pages of a log buffer of the first node;
generating, via the first node, one or more page descriptors associated with the one or more data pages;
sending the one or more page descriptors to a second node concurrently with the writing of the data to the one or more data pages of the log buffer of the first node;
defining one or more ranges of confirmed page descriptors associated with a plurality of data pages written to the log buffer of the first node; and
sending, within the one or more page descriptors, the one or more ranges of confirmed page descriptors associated with a plurality of data pages written to the log buffer of the first node, to the second node.
7. The computer program product of claim 6 , wherein the operations further comprise:
writing, via the second node, the one or more page descriptors to a page descriptor buffer of the second node.
8. The computer program product of claim 7 , wherein the operations further comprise:
in response to writing the one or more page descriptors to the page descriptor buffer of the second node, sending an acknowledgement signal to the first node indicating that the one or more page descriptors are written to the page descriptor buffer of the second node.
9. The computer program product of claim 8 , wherein the operations further comprise:
sending an acknowledgement signal to a computing device that sent the data to the storage system in response to:
determining that the data is written to the one or more data pages of the log buffer of the first node, and
receiving the acknowledgement signal from the second node indicating that the one or more page descriptors are written to the page descriptor buffer.
10. The computer program product of claim 6 , wherein the operations further comprise:
defining a last confirmed page descriptor associated with a last data page of the one or more data pages written to the log buffer of the first node; and
sending the last confirmed page descriptor associated with the last data page written to the log buffer of the first node, to the second node.
11. The computer program product of claim 10 , wherein the operations further comprise:
determining that the first node has failed;
determining the last confirmed page descriptor associated with the last data page written to the log buffer of the first node; and
verifying, on the first node, one or more subsequent page descriptors based upon, at least in part, the last confirmed page descriptor associated with the last data page written to the log buffer of the first node.
12. A computing system comprising:
a memory; and
a processor configured to receive, via a first node of a plurality of nodes, data for storage in the storage system, wherein the processor is further configured to write, via the first node, the data to one or more data pages of a log buffer of the first node, wherein the processor is further configured to generate, via the first node, one or more page descriptors associated with the one or more data pages, wherein the processor is further configured to send the one or more page descriptors to a second node concurrently with the writing of the data to the one or more data pages of the log buffer of the first node, wherein the processor is further configured to define one or more ranges of confirmed page descriptors associated with a plurality of data pages written to the log buffer of the first node, and wherein the processor is further configured to send, within the one or more page descriptors, the one or more ranges of confirmed page descriptors associated with a plurality of data pages written to the log buffer of the first node, to the second node.
13. The computing system of claim 12 , wherein the processor is further configured to:
write, via the second node, the one or more page descriptors to a page descriptor buffer of the second node.
14. The computing system of claim 13 , wherein the processor is further configured to:
in response to writing the one or more page descriptors to the page descriptor buffer of the second node, send an acknowledgement signal to the first node indicating that the one or more page descriptors are written to the page descriptor buffer of the second node.
15. The computing system of claim 14 , wherein the processor is further configured to:
send an acknowledgement signal to a computing device that sent the data to the storage system in response to:
determining that the data is written to the one or more data pages of the log buffer of the first node, and
receiving the acknowledgement signal from the second node indicating that the one or more page descriptors are written to the page descriptor buffer.
16. The computing system of claim 12 , wherein the processor is further configured to:
define a last confirmed page descriptor associated with a last data page of the one or more data pages written to the log buffer of the first node; and
send the last confirmed page descriptor associated with the last data page written to the log buffer of the first node, to the second node.
17. The computing system of claim 16 , wherein the processor is further configured to:
determine that the first node has failed;
determine, via the second node, the last confirmed page descriptor associated with the last data page written to the log buffer of the first node; and
verify, on the first node, one or more subsequent page descriptors based upon, at least in part, the last confirmed page descriptor associated with the last data page written to the log buffer of the first node.