Tracking access pattern of inodes and pre-fetching inodes
Disclosed herein are methods, systems, and processes to track access patterns of inodes, and to issue read-ahead instructions to pre-fetch inodes into memory. A location of a unit of metadata in a metadata storage area is determined. Another location in the metadata storage area that corresponds to a current metadata read operation is determined. Whether a metadata read-ahead operation can be performed is determined using the location of the unit of metadata and the another location. In response to a determination that the metadata read-ahead operation can be performed, the metadata-ahead operation is issued.
1. A method comprising:
reading a first unit of metadata from a metadata storage area in response to an application issuing a first request to read metadata;
determining an end location of the first unit of metadata in the metadata storage area, wherein the metadata storage area is in a non-volatile storage unit;
determining a start location for a second unit of metadata in the metadata storage area in response to the application issuing a second request to read metadata;
comparing the end location with the start location; and
generating a metadata read-ahead operation to read a third unit of metadata from the metadata storage area before the application issues a request to read the third unit of metadata if the comparing indicates that the end and start locations are immediately adjacent to each other in the non-volatile storage unit.
2. The method of claim 1 , wherein
the metadata storage area comprises a plurality of inodes;
the first unit of metadata comprises one or more of the plurality of inodes; and
the first unit of metadata is part of a structural file, and the structural file is stored in the non-volatile storage unit.
3. The method of claim 2 , further comprising:
creating an in-core inode, wherein the in-core inode corresponds to the structural file; and
storing the end location of the first unit of metadata in the in-core inode.
4. The method of claim 2 , further comprising:
intercepting a command to read one or more on-disk inodes, wherein
the one or more on-disk inodes are part of the plurality of inodes,
the intercepting is performed in response to an input/output (I/O) operation, and
the I/O operation results in the command;
analyzing an issued metadata read-ahead value in the metadata read-ahead operation,
wherein
the analyzing comprises comparing the issued metadata read-ahead value and a chunk total in the command;
waiting for the I/O operation to complete, if the analyzing indicates the I/O operation should complete; and
issuing an asynchronous metadata read-ahead instruction, if the analyzing indicates that an asynchronous metadata read-ahead instruction can be issued.
5. The method of claim 4 , further comprising:
determining whether the I/O operation is complete;
generating a queue if the I/O operation is complete, wherein
the queue comprises one or more remaining chunks of metadata of the chunk total, and
the one or more remaining chunks of metadata data are not included in the asynchronous metadata read-ahead instruction; and
updating the chunk total in the metadata read-ahead operation, if the I/O operation is incomplete.
6. The method of claim 1 , further comprising:
generating the metadata read-ahead operation to read the third unit of metadata from the metadata storage area;
determining an end location of the second unit of metadata;
storing the end location of the second unit of metadata;
determining an end location of the third unit of metadata; and
in response to generating the metadata read-ahead operation, replacing the end location of the second unit of metadata in memory with the end location of the third unit of metadata.
7. The method of claim 6 , further comprising:
reading a fourth unit of metadata in response to the application issuing a fourth request to read metadata;
determining a start location for a fourth unit of metadata in the metadata storage area in response to the application issuing fourth request to read metadata;
comparing the start location for the fourth unit of metadata with the end location of the third unit of metadata; and
replacing the end location of the third unit of metadata with an end of location of the fourth unit of metadata, wherein
the end of location of the third unit of metadata is replaced without generating another metadata read-ahead operation.
8. A non-transitory computer readable storage medium comprising program instructions executable to:
read a first unit of metadata in a metadata storage area in response to an application issuing a first request to read metadata;
determine an end location of the first unit of metadata in the metadata storage area, wherein the metadata storage area is in a non-volatile storage unit;
determine a start location for a second unit of metadata in the metadata storage area in response to the application issuing a second request to read metadata;
compare the end location with the start location; and
generate a metadata read-ahead operation to read a third unit of metadata from the metadata storage area before the application issues a request to read the third unit of metadata if the program instructions executable to compare indicate that the end and start locations are immediately adjacent to each other in the non-volatile storage unit.
9. The non-transitory computer readable storage medium of claim 8 , wherein
the metadata storage area comprises a plurality of inodes,
the first unit of metadata comprises one or more of the plurality of inodes; and
the first unit of metadata is part of a structural file, and the structural file is stored in the non-volatile storage unit.
10. The non-transitory computer readable storage medium of claim 9 , further comprising further program instructions executable to:
create an in-core inode, wherein the in-core inode corresponds to the structural file; and
store the end location of the first unit of metadata in the in-core inode.
11. The non-transitory computer readable storage medium of claim 9 , further comprising further program instructions executable to:
intercept a command to read one or more on-disk inodes; wherein
the one or more on-disk inodes are part of the plurality of inodes,
the further program instructions executable to intercept are executed in response to an input/output (I/O) operation, and
the I/O operation results in the command;
analyze an issued metadata read-ahead value in the metadata read-ahead operation,
wherein
the issued metadata read-ahead value is analyzed by comparing the issued metadata read-ahead value and a chunk total in the command;
wait for the I/O operation to complete, if the analyzing indicates the I/O operation should complete;
issue an asynchronous metadata read-ahead instruction, if the analyzing indicates that an asynchronous metadata read-ahead instruction can be issued;
determine whether the I/O operation is complete;
generate a queue if the I/O operation is complete, wherein
the queue comprises one or more remaining chunks of metadata of the chunk total, and
the one or more remaining chunks of metadata data are not included in the asynchronous metadata read-ahead instruction; and
update the chunk total in the metadata read-ahead operation, if the I/O operation is incomplete.
12. The non-transitory computer readable storage medium of claim 8 , further comprising further program instructions executable to:
generate the metadata read-ahead operation to read the third unit of metadata from the metadata storage area;
determine an end location of the second unit of metadata;
store the end location of the second unit of metadata;
determine an end location of the third unit of metadata; and
in response to generating the metadata read-ahead operation, replace the end location of the second unit of metadata in memory with the end location of the third unit of metadata.
13. A system comprising:
one or more processors; and
a memory coupled to the one or more processors, wherein the memory stores program instructions executable by the one or more processors to read a first unit of metadata in a metadata storage area in response to an application issuing a first request to read metadata;
determine an end location of the first unit of metadata in the metadata storage area, wherein the metadata storage area is in a non-volatile storage unit;
determine a start location for a second suit of metadata in the metadata storage area in response to the application issuing a second request to read metadata;
compare the end location with the star location; and
generate a metadata read-ahead operation to read a third unit of metadata from the metadata storage area before the application issues a request to read the third unit of metadata, if the program instructions executable by the one or more processors to compare indicate that the end and start locations are immediately adjacent to each other in the non-volatile storage unit.
14. The system of claim 13 , wherein
the metadata storage area comprises a plurality of inodes,
the first unit of metadata comprises one or more of the plurality of inodes; and
the first unit of metadata is part of a structural file, and the structural file is stored in the non-volatile storage unit.
15. The system of claim 14 , further comprising further instructions executable by the one or more processors to:
create an in-core inode, wherein the in-core inode corresponds to the structural file; and
store the end location of the first unit of metadata in the in-core inode.
16. The system of claim 14 , further comprising further comprising further instructions executable by the one or more processors to:
intercept a command to read one or more on-disk inodes, wherein
the one or more on-disk inodes are part of the plurality of inodes,
the further program instructions executable by the one or more processors to intercept are executed in response to an input/output (I/O) operation, and
the I/O operation results in the command;
analyze an issued metadata read-ahead value in the metadata read-ahead operation,
wherein
the issued metadata read-ahead value is analyzed by comparing the issued metadata read-ahead value and a chunk total in the command;
wait for the I/O operation to complete, if the analyzing indicates the I/O operation should complete;
issue an asynchronous metadata read-ahead instruction, if the analyzing indicates that an asynchronous metadata read-ahead instruction can be issued;
determine whether the I/O operation is complete;
generate a queue if the I/O operation is complete, wherein
the queue comprises one or more remaining chunks of metadata of the chunk total, and
the one or more remaining chunks of metadata data are not included in the asynchronous metadata read-ahead instruction; and
update the chunk total in the metadata read-ahead operation, if the I/O operation is incomplete.
17. The system of claim 13 , further comprising further instructions executable by the one or more processors to:
generate the metadata read-ahead operation to read the third unit of metadata from the metadata storage area;
determine an end location of the second unit of metadata;
store the end location of the second unit of metadata;
determine an end location of the third unit of metadata; and
in response to generating the metadata read-ahead operation, replace the end location of the second unit of metadata in memory with the end location of the third unit of metadata.