Techniques for reducing size of log records
Techniques for processing I/O operations include: receiving a write I/O operation that writes first data to a target location, wherein the target location is represented as a logical device and offset within a logical address space of the logical device; storing a log record for the write I/O operation in a log file; and performing first processing of the log record. The log record includes log data, comprising the first data, and a log descriptor. The log descriptor includes a target logical address for the target location in a file system logical address space. The log descriptor includes a first value denoting the binary logarithm of an extent size of the first logical device. The first processing includes flushing the log record from the log file to store the first data of the log record on an extent of physical storage provisioned for the logical device.
1. A method of processing I/O operations comprising:
receiving a write I/O operation that writes first data to a target location, wherein the target location is represented as a first logical device and a first offset within a first logical address space of the first logical device;
storing a log record for the write I/O operation in a log file, wherein the log record includes log data and a log descriptor, wherein the log data includes the first data, wherein the log descriptor includes a target logical address for the target location in a file system logical address space, wherein the target logical address is determined by mapping the target location to a corresponding logical address in the file system logical address space, wherein the log descriptor includes a first value denoting the binary logarithm of an extent size of the first logical device; and
performing first processing of the log record, wherein said first processing includes flushing the log record from the log file to store the first write data of the log record on a first extent of physical storage provisioned for the first logical device.
2. The method of claim 1 , wherein the extent size denotes a size of the first extent of physical storage allocated for storing content of the first logical device, and wherein the first extent is mapped to a subrange of the file system logical address space.
3. The method of claim 2 , wherein the first logical device is implemented as a file in a file system having the file system logical address space.
4. The method of claim 3 , wherein said first processing further comprises:
determining the extent size of the first logical device using the first value of the log record; and
determining, in accordance with the first value and the extent size, a base address in the file system logical address space for the first logical device, wherein the base address denotes a starting logical address in the file system logical address space for the first extent of the first logical device.
5. The method of claim 4 , wherein said first processing further comprises:
determining the first offset using the base address for the first logical device and using the target logical address of the log record for the write I/O operation, wherein the base address is determined by said first processing.
6. The method of claim 5 , further comprising:
storing at least one of the base address, the first offset and the extent size as first metadata associated with the first logical device.
7. The method of claim 6 , further comprising:
performing recovery processing using the at least one of the base address, the first offset and the extent size stored as the first metadata associated with the first logical device, wherein the recovery processing includes recovering second metadata of the file system.
8. The method of claim 7 , wherein the second metadata is included in an index node (inode) of the file system, wherein the inode is uniquely associated with the file used to implement the first logical device.
9. The method of claim 8 , wherein prior to performing said receiving, said storing and said first processing, the method includes performing other processing comprising:
creating a file system object in the file system for the first logical device, wherein said creating the file system object includes:
creating the inode and mapping the inode into the file system logical address space; and
allocating the first extent and mapping the first extent into the file system logical address space.
10. The method of claim 5 , further comprising:
performing verification processing or consistency checking in connection with the first logical device, wherein said verification processing or consistency checking uses at least one of the base address, the first offset and the extent size as first metadata associated with the first logical device.
11. The method of claim 1 , wherein said first processing includes committing a first transaction of a plurality of write I/O operations including the write I/O operation.
12. The method of claim 11 , wherein said committing the first transaction includes writing a plurality of log records, including the log record, to the log file, and wherein the method includes:
responsive to committing the first transaction whereby the plurality of log records have been written to the log file, sending a response to a client that requested the first transaction has completed.
13. The method of claim 2 , wherein the file system logical address space is a range of logical addresses from a starting address to an ending address, wherein a binary logarithm of the ending address is J, and wherein the first value is stored in a field of the log record for the write I/O operation and the first field has a size determined in accordance with J.
14. The method of claim 2 , wherein the extent size is a power of 2.
15. The method of claim 14 , wherein the extent size is greater than a specified minimum extent size, and wherein the minimum extent size is a power of 2.
16. The method of claim 15 , wherein a base address in the file system logical address space for the first logical device denotes a starting logical address in the file system logical address space for the first logical device, and wherein the base address is a power of 2 and wherein the base address is an integer multiple of the extent size of the first logical device.
17. A system comprising:
one or more processors; and
a memory comprising code stored thereon that, when executed, performs a method of processing I/O operations comprising:
receiving a write I/O operation that writes first data to a target location, wherein the target location is represented as a first logical device and a first offset within a first logical address space of the first logical device;
storing a log record for the write I/O operation in a log file, wherein the log record includes log data and a log descriptor, wherein the log data includes the first data, wherein the log descriptor includes a target logical address for the target location in a file system logical address space, wherein the target logical address is determined by mapping the target location to a corresponding logical address in the file system logical address space, wherein the log descriptor includes a first value denoting the binary logarithm of an extent size of the first logical device; and
performing first processing of the log record, wherein said first processing includes flushing the log record from the log file to store the first write data of the log record on a first extent of physical storage provisioned for the first logical device.
18. A non-transitory computer readable medium comprising code stored thereon that, when executed, performs a method of processing I/O operations comprising:
receiving a write I/O operation that writes first data to a target location, wherein the target location is represented as a first logical device and a first offset within a first logical address space of the first logical device;
storing a log record for the write I/O operation in a log file, wherein the log record includes log data and a log descriptor, wherein the log data includes the first data, wherein the log descriptor includes a target logical address for the target location in a file system logical address space, wherein the target logical address is determined by mapping the target location to a corresponding logical address in the file system logical address space, wherein the log descriptor includes a first value denoting the binary logarithm of an extent size of the first logical device; and
performing first processing of the log record, wherein said first processing includes flushing the log record from the log file to store the first write data of the log record on a first extent of physical storage provisioned for the first logical device.
19. The non-transitory computer readable medium of claim 18 , wherein the extent size denotes a size of the first extent of physical storage allocated for storing content of the first logical device, and wherein the first extent is mapped to a subrange of the file system logical address space.
20. The non-transitory computer readable medium of claim 19 , wherein the first logical device is implemented as a file in a file system having the file system logical address space.