Log destaging from fixed-size log portion
The bifurcation of a log into a fixed-size log portion and a growable log portion. Log records are first written to the fixed-size log portion. At some point, the older log records within the fixed-size log portion are destaged to the growable log portion. Destaging may involve copying the log records from the fixed-size log portion and tacking the log record onto the end of the growable log portion. This destaging might occur one sub-portion (e.g., one virtual log file) at a time. The more recent log records are within the fixed-size log portion. Accordingly, new writes to the log occur to the fixed-size log portion. Furthermore, a large portion of the reads are of the more recent log records within the fixed-size log portion. Thus, optimizing performance within the fixed-size log portion results in significant improvement in the performance of the overall log.
1. A computing system comprising:
one or more processors; and
one or more computer-readable storage media having thereon computer-execution instructions that are structured such that, when executed by the one or more processors, the computing system is caused to perform steps for maintaining a log, the steps comprising:
writing log records to a fixed-size log portion where a tail of the log is located; and
destaging log records from the fixed-size log portion to a growable log portion, wherein the fixed sized log portion includes a plurality of sub-portions, each sub-portion containing a corresponding plurality of blocks, wherein there is an odd number of the plurality of sub-portions within the fixed-size log portion, wherein destaging log records from the fixed-size log portion to a growable log portion comprises copying one sub-portion at a time to the growable log, and wherein the sub-portions are destaged one at a time to the growable log in advance of that sub-portion being overwritten by a subsequent sequential write of blocks.
2. The computing system in accordance with claim 1 , wherein a location within the fixed-size log portion of a block having a log record is predetermined based on a log sequence number of the log record.
3. The computing system in accordance with claim 2 , wherein a location of the log record of the block having the log record is also predetermined based on the log sequence number of the log record.
4. The computing system in accordance with claim 1 , wherein writing log records to the fixed-size log portion comprises writing blocks sequentially to the fixed-size log portion, and returning to the beginning of the fixed-size log portion when the end of the fixed-size log portion is encountered.
5. The computing system in accordance with claim 1 , wherein after each sub-portion is destaged from the fixed-size log portion to the growable log portion, that destaged sub-portion is replaced by initializing a new sub-portion within the fixed-size log portion, a sub-portion identifier for each successively initialized sub-portion being incremented each time a new sub-portion is initialized within the fixed-size log portion, wherein after initializing the sub-portion with a new sub-portion identifier, each block of the new sub-portion is written with log marking data that has one of two values depending on whether the sub-portion identifier of the new sub-portion is odd or even.
6. The computing system in accordance with claim 1 , wherein after a sub-portion is destaged from the fixed-size log portion to the growable log portion, a new sub-portion is initialized in the fixed-size log portion at the same location as the destaged sub-portion, the initialization include assignment of a new sub-portion identifier, wherein after initializing the sub-portion with the new sub-portion identifier, when writing a block of data to that respective block after the sub-portion was initialized, the log marking data is updated to represent that the respective block has been written to since the time that the respective block was initialized.
7. The computing system in accordance with claim 4 , each of the plurality of sub-portions being of equal size.
8. The computing system in accordance with claim 4 , each of the plurality of sub-portions being a virtual log file.
9. A method for maintaining a log, the method comprising:
writing log records to a fixed-size log portion where a tail of the log is located; and
destaging log records from the fixed-size log portion to a growable log portion, wherein the fixed sized log portion includes a plurality of sub-portions, each sub-portion containing a corresponding plurality of blocks, wherein there is an odd number of the plurality of sub-portions within the fixed-size log portion, wherein destaging log records from the fixed-size log portion to a growable log portion comprises copying one sub-portion at a time to the growable log, and wherein the sub-portions are destaged one at a time to the growable log in advance of that sub-portion being overwritten by a subsequent sequential write of blocks.
10. The method in accordance with claim 9 , wherein a location within the fixed-size log portion of a block having a log record is predetermined based on a log sequence number of the log record.
11. The method in accordance with claim 10 , wherein a location of the log record of the block having the log record is also predetermined based on the log sequence number of the log record.
12. The method in accordance with claim 9 , wherein writing log records to the fixed-size log portion comprises writing blocks sequentially to the fixed-size log portion, and returning to the beginning of the fixed-size log portion when the end of the fixed-size log portion is encountered.
13. The method in accordance with claim 9 , wherein after each sub-portion is destaged from the fixed-size log portion to the growable log portion, that destaged sub-portion is replaced by initializing a new sub-portion within the fixed-size log portion, a sub-portion identifier for each successively initialized sub-portion being incremented each time a new sub-portion is initialized within the fixed-size log portion, wherein after initializing the sub-portion with a new sub-portion identifier, each block of the new sub-portion is written with log marking data that has one of two values depending on whether the sub-portion identifier of the new sub-portion is odd or even.
14. The method in accordance with claim 9 , wherein after a sub-portion is destaged from the fixed-size log portion to the growable log portion, a new sub-portion is initialized in the fixed-size log portion at the same location that the destaged sub-portion, the initialization include assignment of a new sub-portion identifier, wherein after initializing the sub-portion with the new sub-portion identifier, when writing a block of data after the sub-portion was initialized, the log marking data is updated to represent that the respective block has been written to since the time that the respective block was initialized.
15. A computer program product comprising one or more computer-readable storage media having thereon computer-executable instructions that are structured such that, when executed by one or more processors of a computing system, cause the computing system to perform steps for maintaining a log, the steps comprising:
writing log records to a fixed-size log portion where a tail of the log is located; and
destaging log records from the fixed-size log portion to a growable log portion, wherein the fixed sized log portion includes a plurality of sub-portions, each sub-portion containing a corresponding plurality of blocks, wherein there is an odd number of the plurality of sub-portions within the fixed-size log portion, wherein destaging log records from the fixed-size log portion to a growable log portion comprises copying one sub-portion at a time to the growable log, and wherein the sub-portions are destaged one at a time to the growable log in advance of that sub-portion being overwritten by a subsequent sequential write of blocks.