Method and device for storing data
A method and device for storing data are provided. The method includes: selecting at least one block from a plurality of blocks in a storage device as a evicting block or a target block based on an expected expiration time of each of the plurality of blocks in response to a request for garbage collection, wherein the expected expiration time of each block is obtained based on an expected expiration time of valid data of each block; and performing garbage collection based on the selected at least one block.
1. A method of storing data, the method comprising:
obtaining a score for each of a plurality of blocks based on a block attribute and an expected expiration time of a respective block among the plurality of blocks;
selecting, based on a request for garbage collection, at least one first block from a plurality of blocks in a storage as an evicting block or a target block based on the score of each of the plurality of blocks, the expected expiration time of each of the plurality of blocks being obtained based on an expected expiration time of valid data of each of the plurality of blocks; and
performing garbage collection based on the selected at least one first block.
2. The method according to claim 1 , wherein the selecting the at least one first block as the evicting block or the target block comprises:
selecting the at least one first block from the plurality of blocks in the storage as the evicting block,
wherein the expected expiration time of the selected evicting block is greater than or equal to expected expiration times of one or more second blocks, other than the evicting block, in the plurality of blocks.
3. The method according to claim 2 , further comprises:
selecting at least one second block satisfying a condition from among the one or more second blocks as the target block based on an expected expiration time of the valid data of the evicting block and expected expiration times of the one or more second blocks in the block.
4. The method according to claim 3 , wherein the condition comprises:
a difference between the expected expiration time of the valid data of the evicting block and the expected expiration time of the target block being less than or equal to a reference value.
5. The method according to claim 1 , wherein the selecting the at least one first block from the plurality of blocks as the evicting block comprises:
selecting the at least one first block from the plurality of blocks as the evicting block based on the expected expiration time and at least one of an amount of invalid data, an amount of valid data or a valid data ratio of each of the plurality of blocks,
wherein the valid data ratio of each block indicates a ratio between the amount of valid data for the block and a total amount of data in the plurality of blocks.
6. The method according to claim 1 , wherein the expected expiration time of each block is an average of the expected expiration times of the valid data of each of the plurality of blocks.
7. The method according to claim 1 , wherein the data of each of the plurality of blocks is data of a data file of a log structure merge tree (LSM-Tree) database, and
wherein the expected expiration time of the valid data of each block is obtained according to an expected expiration time of the data file to which the valid data belongs.
8. The method according to claim 7 , wherein the method further comprises:
obtaining operating parameters of the LSM-Tree database, the operating parameters comprising at least one of a number of files waiting for compaction compacted, a number of files being compacted, a number of files waiting for flush, or a number of files being flushed in the LSM-Tree database;
determining whether input or output of the LSM-Tree database is busy using a pre-trained machine learning model with the operating parameters as inputs; and
generating the request for garbage collection when the input or output of the LSM-Tree database is not busy.
9. A non-transitory computer-readable storage medium storing a computer program, which, when executed by a processor, implements the method of storing data according to claim 1 .
10. A device for storing data, the device comprising:
a storage divided into a plurality of blocks for storing data; and
a processor configured to:
obtaining a score for each of the plurality of blocks based on a block attribute and an expected expiration time of a respective block among the plurality of blocks;
select, based on a request for garbage collection, at least one first block from a plurality of blocks in a storage as an evicting block or a target block based on the score of each of the plurality of blocks, the expected expiration time of each of the plurality of blocks being obtained based on an expected expiration time of valid data of each of the plurality of blocks; and
perform garbage collection based on the selected at least one first block.
11. The device according to claim 10 , wherein the processor selects the at least one first block from the plurality of blocks in the storage as the evicting block, and
wherein the expected expiration time of the selected evicting block is greater than or equal to expected expiration times of one or more second blocks, other than the evicting block, in the plurality of blocks.
12. The device according to claim 11 , wherein the processor is configured to:
select at least one second block satisfying a condition from among the one or more second blocks as the target block based on an expected expiration time of the valid data of the evicting block and expected expiration times of the one or more second blocks in the plurality of blocks.
13. The device according to claim 12 , wherein the condition comprises:
a difference between the expected expiration time of the valid data of the evicting block and the expected expiration time of the selected target block being less than or equal to a preset value.
14. The device according to claim 10 , wherein the processor is configured to:
select the at least one first block from the plurality of blocks as the evicting block based on the expected expiration time and at least one of an amount of invalid data, an amount of valid data or a valid data ratio of each of the plurality of blocks,
wherein the valid data ratio of each block indicates a ratio between the amount of valid data for the block and a total amount of data in the block.
15. The device according to claim 10 , wherein the expected expiration time of each block is an average of the expected expiration times of the valid data of each of the plurality of blocks.
16. The device according to claim 10 , wherein the data of each of the plurality of blocks is data of a data file of a log structure merge tree (LSM-Tree) database, and
wherein the expected expiration time of the valid data of each block is obtained according to an expected expiration time of the data file to which the valid data belongs.
17. The device according to claim 16 , wherein the processor is further configured to:
obtain operating parameters of the LSM-Tree database, the operating parameters comprising at least one of a number of files waiting for compaction compacted, a number of files being compacted, a number of files waiting for flush, or a number of files being flushed in the LSM-Tree database;
determine whether input or output of the LSM-Tree database is busy using a pre-trained machine learning model with the operating parameters as inputs; and
generate the request for garbage collection when the input or output of the LSM-Tree database is not busy.
18. An electronic system, the electronic system comprises:
a memory storing one or more instructions; and
a storage device divided into a plurality of blocks for storing data; and
a processor configured to execute the one or more instructions to:
obtain a score for each of the plurality of blocks based on a block attribute and an expected expiration time of a respective block among the plurality of blocks;
select, based on a request for garbage collection, at least one first block from the plurality of blocks in the storage device as an evicting block or a target block based on the score of each of the plurality of blocks, the expected expiration time of each of the plurality of blocks being obtained based on an expected expiration time of valid data of each of the plurality of blocks; and
perform garbage collection based on the selected at least one first block.