Transactional file system
A transactional file system wherein multiple file system operations may be performed as a transaction. An application specifies that file system-related operations are to be handled as a transaction, and the application is given a file handle associated with a transaction context. For file system requests associated with a transaction context, a file system component manages operations consistent with transactional behavior. Logging and recovery are also facilitated by logging page data separate from the main log with a unique signature that enables the log to determine whether a page was fully flushed to disk prior to a system crash.
1. In a transactional file system, a method of recording page changes in a transactional log, comprising:
receiving a request, from a transaction, to update a file;
copying a page into memory, the page including the file;
prior to applying the update to the file within the page in memory:
adding an undo record to a transactional log in persistent storage to reflect the update, the undo record including a cycle count that represents a current status of the page;
copying the page to a data stream in persistent storage, the data stream being separate from the transactional log, the page including a plurality of sectors;
replacing a portion of each of the plurality of sectors within the data stream with the cycle count that is included in the undo record;
wherein each portion of the plurality of sectors that is replaced by the cycle count is also written to the undo record corresponding to that cycle count;
wherein maintaining the cycle count within the copied page in the data stream and within the undo record enables the data stream and the transactional log to be written to persistent storage in an arbitrary order while still enabling the data stream and transactional log to be used to recover from a system failure during the transaction.
2. The method of claim 1 , wherein the portion comprises the last byte of each sector.
3. The method of claim 1 , further comprising:
after a system failure, locating the undo record in the transactional log thus indicating that the transactional log was written to persistent storage prior to the system failure; and
determining whether the data stream containing the copied page was written to persistent storage prior to the system failure by comparing the cycle count in the undo record with cycle count stored in the portion of each sector, and:
if the cycle count in the undo record matches the cycle count in the portion of each sector, determining that the data stream was written to persistent storage prior to the system failure, and restoring each portion of the plurality of sectors that was overwritten by the cycle count with the corresponding portion of the plurality of sectors that was written to the undo record; whereas
if the cycle count in the undo record does not match the cycle count in the portion of each sector, determining that the data stream was not written to persistent storage prior to the system failure.
4. The method of claim 1 , further comprising:
creating a verify bitmap to maintain an indication of the state of each page in the data stream;
upon the pointer wrapping around, verifying that the current cycle count does not match the cycle count of any page in the data stream; and
updating a bit in the verify bitmap for each page for which the current cycle count differs from the cycle count stored with the page.
5. The method of claim 4 , wherein updating the bit in the verify bitmap for each page comprises:
upon the pointer wrapping around, resetting the bitmap so that the bit for each page is set to a first value; and
upon verifying that the current cycle count does not match the cycle count of a page in the data stream, changing the bit corresponding to the page to a second value.
6. The method of claim 1 , wherein the undo record includes an offset indicating the location of the page in the data stream.
7. The method of claim 1 , wherein the undo record includes a log sequence number for the update.
8. One or more memory storage devices storing computer executable instructions which when executed perform a method of recording page changes in a transactional log, comprising:
receiving a request, from a transaction, to update a file;
copying a page into memory, the page including the file;
prior to applying the update to the file within the page in memory:
adding an undo record to a transactional log in persistent storage to reflect the update, the undo record including a cycle count that represents a current status of the page;
copying the page to a data stream in persistent storage, the data stream being separate from the transactional log, the page including a plurality of sectors;
replacing a portion of each of the plurality of sectors within the data stream with the cycle count that is included in the undo record;
wherein each portion of the plurality of sectors that is replaced by the cycle count is also written to the undo record corresponding to that cycle count;
wherein maintaining the cycle count within the copied page in the data stream and within the undo record enables the data stream and the transactional log to be written to persistent storage in an arbitrary order while still enabling the data stream and transactional log to be used to recover from a system failure during the transaction.
9. The one or more memory storage devices of claim 8 , wherein the page comprises a plurality of sectors and wherein including the cycle count with the copied page comprises:
copying a portion of each sector into the undo record; and
storing the cycle count in place of the copied portion of each sector.
10. The one or more memory storage devices of claim 9 , wherein the portion comprises the last byte of each sector.
11. The one or more memory storage devices of claim 9 , further comprising:
after a system failure, locating the undo record in the transactional log thus indicating that the transactional log was written to persistent storage prior to the system failure; and
determining whether the data stream containing the copied page was written to persistent storage prior to the system failure by comparing the cycle count in the undo record with cycle count stored in the portion of each sector, and:
if the cycle count in the undo record matches the cycle count in the portion of each sector, determining that the data stream was written to persistent storage prior to the system failure, and restoring each portion of the plurality of sectors that was overwritten by the cycle count with the corresponding portion of the plurality of sectors that was written to the undo record; whereas
if the cycle count in the undo record does not match the cycle count in the portion of each sector, determining that the data stream was not written to persistent storage prior to the system failure.
12. The one or more memory storage devices of claim 8 , further comprising:
creating a verify bitmap to maintain an indication of the state of each page in the data stream;
upon the pointer wrapping around, verifying that the current cycle count does not match the cycle count of any page in the data stream; and
updating a bit in the verify bitmap for each page for which the current cycle count differs from the cycle count stored with the page.
13. The one or more memory storage devices of claim 12 , wherein updating the bit in the verify bitmap for each page comprises:
upon the pointer wrapping around, resetting the bitmap so that the bit for each page is set to a first value; and
upon verifying that the current cycle count does not match the cycle count of a page in the data stream, changing the bit corresponding to the page to a second value.
14. The one or more memory storage devices of claim 8 , wherein the undo record includes an offset indicating the location of the page in the data stream.
15. The one or more memory storage devices of claim 8 , wherein the undo record includes a log sequence number for the update.
16. A computer system comprising:
one or more processors;
memory; and
persistent storage;
wherein the system memory stores computer executable instructions which when executed by the one or more processors perform a method of recording page changes in a transactional log, comprising:
receiving a request, from a transaction, to update a file;
copying a page into memory, the page including the file;
prior to applying the update to the file within the page in memory:
adding an undo record to a transactional log in persistent storage to reflect the update, the undo record including a cycle count that represents a current status of the page;
copying the page to a data stream in persistent storage, the data stream being separate from the transactional log, the page including a plurality of sectors;
replacing a portion of each of the plurality of sectors within the data stream with the cycle count that is included in the undo record;
wherein each portion of the plurality of sectors that is replaced by the cycle count is also written to the undo record corresponding to that cycle count;
wherein maintaining the cycle count within the copied page in the data stream and within the undo record enables the data stream and the transactional log to be written to persistent storage in an arbitrary order while still enabling the data stream and transactional log to be used to recover from a system failure during the transaction;
after a system failure, locating the undo record in the transactional log thus indicating that the transactional log was written to persistent storage prior to the system failure; and
determining whether the data stream containing the copied page was written to persistent storage prior to the system failure by comparing the cycle count in the undo record with cycle count stored in the portion of each sector, and:
if the cycle count in the undo record matches the cycle count in the portion of each sector, determining that the data stream was written to persistent storage prior to the system failure, and restoring each portion of the plurality of sectors that was overwritten by the cycle count with the corresponding portion of the plurality of sectors that was written to the undo record; whereas
if the cycle count in the undo record does not match the cycle count in the portion of each sector, determining that the data stream was not written to persistent storage prior to the system failure.
17. The system of claim 16 , wherein the page comprises a plurality of sectors and wherein including the cycle count with the copied page comprises:
copying a portion of each sector into the undo record; and
storing the cycle count in place of the copied portion of each sector.
18. The system of claim 16 , further comprising:
creating a verify bitmap to maintain an indication of the state of each page in the data stream;
upon the pointer wrapping around, verifying that the current cycle count does not match the cycle count of any page in the data stream; and
updating a bit in the verify bitmap for each page for which the current cycle count differs from the cycle count stored with the page.
19. The system of claim 18 , wherein updating the bit in the verify bitmap for each page comprises:
upon the pointer wrapping around, resetting the bitmap so that the bit for each page is set to a first value; and
upon verifying that the current cycle count does not match the cycle count of a page in the data stream, changing the bit corresponding to the page to a second value.
20. The method of claim 1 , wherein the cycle count also represents a count of each time a pointer to the data stream wraps around to the start of the data stream.