System and method for dual node parallel flush
A method, computer program product, and computer system for identifying a first node that has written a first page of a plurality of pages to be flushed. A second node that has written a second page of the plurality of pages to be flushed may be identified. It may be determined whether the first page of the plurality of pages is to be flushed by one of the first node and the second node and whether the second page of the plurality of pages is to be flushed by one of the first node and the second node based upon, at least in part, one or more factors. The first node may allocate the first page of the plurality of pages and the second page of the plurality of pages to be flushed in parallel by one of the first node and the second node based upon, at least in part, the one or more factors.
1. A computer-implemented method comprising:
identifying a first node that has written a first page of a plurality of pages to be flushed;
identifying a second node that has written a second page of the plurality of pages to be flushed;
determining whether the first page of the plurality of pages is to be flushed by one of the first node and the second node and whether the second page of the plurality of pages is to be flushed by one of the first node and the second node based upon, at least in part, one or more factors, wherein the one or more factors includes considering tail movement of a log, wherein the tail movement occurs due to one or more of the first page and the second page being organized into a bucket and the use of lowest sequence number (LSN) tree; and
allocating, by the first node, the first page of the plurality of pages and the second page of the plurality of pages to be flushed in parallel by one of the first node and the second node based upon, at least in part, the one or more factors.
2. The computer-implemented method of claim 1 further comprising sending, by the first node to the second node, a page ID and a log offset of the plurality of pages to be flushed.
3. The computer-implemented method of claim 1 further comprising receiving, by the first node from the second node, the page ID and the log offset of the plurality of pages that are flushed.
4. The computer-implemented method of claim 1 wherein at least one factor of the one or more factors includes the first node being identified as having written the first page of the plurality of pages to be flushed and the second node being identified as having written the second page of the plurality of pages to be flushed.
5. The computer-implemented method of claim 4 wherein at least one factor of the one or more factors limits disk reads necessary to service a parallel flush request to have the first page of the plurality of pages and the second page of the plurality of pages flushed in parallel.
6. The computer-implemented method of claim 1 wherein at least one factor of the one or more factors includes load balancing of the first node and the second node.
7. The computer-implemented method of claim 1 wherein at least one factor of the one or more factors includes whether one of the first node and the second node are unavailable during a flush request to flush at least one of the first page and the second page.
8. A computer program product residing on a non-transitory computer readable storage medium having a plurality of instructions stored thereon which, when executed across one or more processors, causes at least a portion of the one or more processors to perform operations comprising:
identifying a first node that has written a first page of a plurality of pages to be flushed;
identifying a second node that has written a second page of the plurality of pages to be flushed;
determining whether the first page of the plurality of pages is to be flushed by one of the first node and the second node and whether the second page of the plurality of pages is to be flushed by one of the first node and the second node based upon, at least in part, one or more factors, wherein the one or more factors includes considering tail movement of a log, wherein the tail movement occurs due to one or more of the first page and the second page being organized into a bucket and the use of lowest sequence number (LSN) tree; and
allocating, by the first node, the first page of the plurality of pages and the second page of the plurality of pages to be flushed in parallel by one of the first node and the second node based upon, at least in part, the one or more factors.
9. The computer program product of claim 8 wherein the operations further comprise sending, by the first node to the second node, a page ID and a log offset of the plurality of pages to be flushed.
10. The computer program product of claim 8 wherein the operations further comprise receiving, by the first node from the second node, the page ID and the log offset of the plurality of pages that are flushed.
11. The computer program product of claim 8 wherein at least one factor of the one or more factors includes the first node being identified as having written the first page of the plurality of pages to be flushed and the second node being identified as having written the second page of the plurality of pages to be flushed.
12. The computer program product of claim 11 wherein at least one factor of the one or more factors limits disk reads necessary to service a parallel flush request to have the first page of the plurality of pages and the second page of the plurality of pages flushed in parallel.
13. The computer program product of claim 8 wherein at least one factor of the one or more factors includes load balancing of the first node and the second node.
14. The computer program product of claim 8 wherein at least one factor of the one or more factors includes whether one of the first node and the second node are unavailable during a flush request to flush at least one of the first page and the second page.
15. A computing system including one or more processors and one or more memories configured to perform operations comprising:
identifying a first node that has written a first page of a plurality of pages to be flushed;
identifying a second node that has written a second page of the plurality of pages to be flushed;
determining whether the first page of the plurality of pages is to be flushed by one of the first node and the second node and whether the second page of the plurality of pages is to be flushed by one of the first node and the second node based upon, at least in part, one or more factors, wherein at least one factor of the one or more factors includes the first node being identified as having written the first page of the plurality of pages to be flushed and the second node being identified as having written the second page of the plurality of pages to be flushed, wherein at least one factor of the one or more factors limits disk reads necessary to service a parallel flush request to have the first page of the plurality of pages and the second page of the plurality of pages flushed in parallel; and
allocating, by the first node, the first page of the plurality of pages and the second page of the plurality of pages to be flushed in parallel by one of the first node and the second node based upon, at least in part, the one or more factors.
16. The computing system of claim 15 wherein the operations further comprise sending, by the first node to the second node, a page ID and a log offset of the plurality of pages to be flushed and receiving, by the first node from the second node, the page ID and the log offset of the plurality of pages that are flushed.
17. The computing system of claim 15 wherein at least one factor of the one or more factors includes load balancing of the first node and the second node.
18. The computing system of claim 15 wherein at least one factor of the one or more factors includes whether one of the first node and the second node are unavailable during a flush request to flush at least one of the first page and the second page.