Method and device for local random readahead of file in distributed file system
Provided are a method and device for local random readahead of a file in a distributed file system. The method includes: creating a readahead structure pointer in an operation handle of a file; in response to reading of the file, accessing and updating information in the readahead structure pointer, and determining whether the reading is local random reading; and performing readahead in a case where the reading is local random reading, the number of instances of continuous reading in the updated information is greater than 1, and a current reading position is greater than or equal to a readahead trigger position in the updated information.
1. A method for local random readahead of a file in a distributed file system, comprising:
creating a readahead structure pointer in an operation handle of a file;
in response to reading of the file, accessing and updating information in the readahead structure pointer, and determining whether the reading is local random reading; and
performing readahead in a case where the reading is local random reading, the number of instances of continuous reading in the updated information is greater than 1, and a current reading position is greater than or equal to a readahead trigger position in the updated information;
wherein the information in the readahead structure pointer comprises: a last reading end position, a last readahead end position, a last readahead size, the number of instances of sequential reading, a data amount of sequential reading, a local random reading window, a maximum readahead size, and the readahead trigger position;
wherein in response to reading of the file, accessing and updating information in the readahead structure pointer comprises:
where a current reading position is the same as the last reading end position, adding 1 to the number of instances of sequential reading, and adding a preset length to the data amount of sequential reading;
wherein in response to reading of the file, determining whether the reading is local random reading comprises:
where the current reading position is different from the last reading end position, determining whether a data amount that differs between a current reading offset and the last reading end position is within one random window, the one random window being the local random reading window; and
in response to where the data amount that differs between the current reading offset and the last reading end position is within the one random window, determining the current reading to be local random reading.
2. The method according to claim 1 , further comprising:
in a case where the current reading is not local random reading, setting the number of instances of sequential reading and the data amount of sequential reading to zero.
3. The method according to claim 1 , wherein the local random reading window and the maximum readahead size are stored in global configuration.
4. A device for local random readahead of a file in a distributed file system, comprising a memory storing instructions and a processor in communication with the memory, wherein the processor is configured to execute the instructions to:
create a readahead structure pointer in an operation handle of a file;
in response to reading of the file, access and update information in the readahead structure pointer, and determine whether the reading is local random reading; and
perform readahead in a case where the reading is local random reading, the number of instances of continuous reading in the updated information is greater than 1, and a current reading position is greater than or equal to a readahead trigger position in the updated information;
wherein the information in the readahead structure pointer comprises: a last reading end position, a last readahead end position, a last readahead size, the number of instances of sequential reading, a data amount of sequential reading, a local random reading window, a maximum readahead size, and the readahead trigger position;
wherein the processor, when being configured to execute the instructions to access and update information in the readahead structure pointer, is configured to execute the instructions to:
where a current reading position is the same as the last reading end position, add 1 to the number of instances of sequential reading, and add a preset length to the data amount of sequential reading;
wherein the processor, when being configured to execute the instructions to determine whether the reading is local random reading, is configured to execute the instructions to:
where the current reading position is different from the last reading end position, determine whether a data amount that differs between a current reading offset and the last reading end position is within one random window; the one random window being the local random reading window; and
in response to where the data amount that differs between the current reading offset and the last reading end position is within one random window, determine the current reading to be local random reading.
5. The device according to claim 4 , wherein the processor is further configured to execute the instructions to, in a case where the current reading is not local random reading, set the number of instances of sequential reading and the data amount of sequential reading to zero.
6. The device according to claim 4 , wherein the local random reading window and the maximum readahead size are stored in global configuration.
7. The method according to claim 1 , wherein when performing the readahead, a readahead offset is the last readahead end position, and a readahead size is twice the last readahead size.
8. The method according to claim 7 , wherein when performing the readahead, in a case where the last readahead size is 0, the readahead size is set to be equal to the data amount of the current sequential reading.
9. The method according to claim 7 , wherein when performing the readahead, an actual readahead size is determined according to the maximum readahead size, and the readahead size is updated as the current actual readahead size.
10. The method according to claim 9 , wherein the readahead trigger position is updated as [(the reading end position+the current actual readahead size)/2].
11. The method according to claim 9 , wherein a readahead end position is updated as the last readahead end position plus the current readahead size.
12. The method according to claim 1 , further comprising:
in a case where the data amount that differs between the current reading offset and the last reading end position is within one random window, adding 1 to the number of instances of sequential reading, and adding a preset length to the data amount of sequential reading.
13. The method according to claim 1 , wherein determining whether the data amount that differs between the current reading offset and the last reading end position is within one random window comprises:
determining whether |current reading offset−last_pos|<=fw, wherein last_pos is the last reading end position, and fw is the local random reading window; and
in a case where |current reading offset−last_pos|<=fw, determining that the data amount that differs between the current reading offset and the last reading end position is within one random window.
14. The method according to claim 1 , wherein the local random reading is a reading process in which out-of-order reading is performed within the local random reading window but the overall reading is sequential reading.
15. The method according to claim 2 , further comprising:
in a case where the current reading is not local random reading, updating a current reading end position to a current reading offset plus a current reading length.
16. The device according to claim 4 , wherein the processor is further configured to execute the instructions to, in a case where the data amount that differs between the current reading offset and the last reading end position is within one random window, add 1 to the number of instances of sequential reading, and add a preset length to the data amount of sequential reading.