System and method for parallel flushing with bucketized data
A method, computer program product, and computer system for organizing a plurality of log records into a plurality of buckets, wherein each bucket is associated with a range of a plurality of ranges within a backing store. A bucket of the plurality of buckets from which a portion of the log records of the plurality of log records are to be flushed may be selected. The portion of the log records may be organized into parallel flush jobs. The portion of the log records may be flushed to the backing store in parallel.
1. A computer-implemented method comprising:
organizing a plurality of log records into a plurality of buckets, wherein each bucket is associated with a range of a plurality of ranges within a backing store;
selecting a bucket of the plurality of buckets from which a portion of the log records of the plurality of log records are to be flushed in parallel with one or more buckets of the plurality of buckets, wherein the bucket selected is the bucket with a lowest log sequence number (LSN) from a tree, wherein the log records organized in the selected bucket are processed in a chronological LSN order, and wherein the selected bucket is configured to be flushed in one parallel request;
organizing the portion of the log records into parallel flush jobs; and
flushing the portion of the log records to the backing store in parallel.
2. The computer-implemented method of claim 1 wherein the portion of the log records include destination logical block address (LBA) of the backing store associated with the range, and wherein the portion of the log records include LSNs.
3. The computer-implemented method of claim 1 wherein selecting the bucket of the plurality of buckets from which the portion of the log records of the plurality of log records are to be flushed includes selecting the bucket with the lowest LSN from the tree.
4. The computer-implemented method of claim 3 wherein flushing the portion of the log records to the backing store in parallel includes invalidating the portion of the log records in memory.
5. The computer-implemented method of claim 4 wherein flushing the portion of the log records to the backing store in parallel further includes removing the portion of the log records from the bucket.
6. The computer-implemented method of claim 5 wherein flushing the portion of the log records to the backing store in parallel further includes removing the portion of the log records from the tree.
7. The computer-implemented method of claim 6 further comprising rebalancing the tree.
8. A computer program product residing on a non-transitory computer readable storage medium having a plurality of instructions stored thereon which, when executed across one or more processors, causes at least a portion of the one or more processors to perform operations comprising:
organizing a plurality of log records into a plurality of buckets, wherein each bucket is associated with a range of a plurality of ranges within a backing store;
selecting a bucket of the plurality of buckets from which a portion of the log records of the plurality of log records are to be flushed in parallel with one or more buckets of the plurality of buckets, wherein the bucket selected is the bucket with a lowest log sequence number (LSN) from a tree, wherein the log records organized in the selected bucket are processed in a chronological LSN order, and wherein the selected bucket is configured to be flushed in one parallel request;
organizing the portion of the log records into parallel flush jobs; and
flushing the portion of the log records to the backing store in parallel.
9. The computer program product of claim 8 wherein the portion of the log records include destination logical block address (LBA) of the backing store associated with the range, and wherein the portion of the log records include LSNs.
10. The computer program product of claim 8 wherein selecting the bucket of the plurality of buckets from which the portion of the log records of the plurality of log records are to be flushed includes selecting the bucket with the lowest LSN from the tree.
11. The computer program product of claim 10 wherein flushing the portion of the log records to the backing store in parallel includes invalidating the portion of the log records in memory.
12. The computer program product of claim 11 wherein flushing the portion of the log records to the backing store in parallel further includes removing the portion of the log records from the bucket.
13. The computer program product of claim 12 wherein flushing the portion of the log records to the backing store in parallel further includes removing the portion of the log records from the tree.
14. The computer program product of claim 13 wherein the operations further comprise rebalancing the tree.
15. A computing system including one or more processors and one or more memories configured to perform operations comprising:
organizing a plurality of log records into a plurality of buckets, wherein each bucket is associated with a range of a plurality of ranges within a backing store;
selecting a bucket of the plurality of buckets from which a portion of the log records of the plurality of log records are to be flushed in parallel with one or more buckets of the plurality of buckets, wherein the bucket selected is the bucket with a lowest log sequence number (LSN) from a tree, wherein the log records organized in the selected bucket are processed in a chronological LSN order, and wherein the selected bucket is configured to be flushed in one parallel request;
organizing the portion of the log records into parallel flush jobs; and
flushing the portion of the log records to the backing store in parallel.
16. The computing system of claim 15 wherein the portion of the log records include destination logical block address (LBA) of the backing store associated with the range, and wherein the portion of the log records include LSNs.
17. The computing system of claim 15 wherein selecting the bucket of the plurality of buckets from which the portion of the log records of the plurality of log records are to be flushed includes selecting the bucket with the lowest LSN from the tree.
18. The computing system of claim 17 wherein flushing the portion of the log records to the backing store in parallel includes invalidating the portion of the log records in memory and removing the portion of the log records from the bucket.
19. The computing system of claim 18 wherein flushing the portion of the log records to the backing store in parallel further includes removing the portion of the log records from the tree.
20. The computing system of claim 19 wherein the operations further comprise rebalancing the tree.