METHODS, APPARATUSES AND COMPUTER PROGRAM PRODUCTS FOR MANAGING METADATA OF STORAGE OBJECT
Metadata of a storage object is managed. For instance, in response to metadata of a storage object being updated, a first index structure for indexing the metadata of the storage object and a page table corresponding to the first index structure in a memory are updated, wherein the first index structure and the page table have been stored in a persistent storage device. Updates of the page table are recorded in at least one page table journal and the updated first index structure and the at least one page table journal are stored in the persistent storage device. Embodiments can significantly increase the speed of failover and persistence of metadata in a distributed object storage system.
1 . A method for managing metadata of a storage object, comprising:
in response to the metadata of the storage object being updated, updating, by a system comprising a processor, a first index structure for indexing the metadata of the storage object and a page table corresponding to the first index structure in a memory, resulting in an updated first index structure, wherein the first index structure records a mapping relationship between a first identifier of the storage object and a second identifier of a page where the metadata of the storage object is located, wherein the page table records a mapping relationship between the second identifier and a page address of the page, and wherein the first index structure and the page table have been stored in a persistent storage device;
recording updates of the page table in at least one page table journal; and
storing the updated first index structure and the at least one page table journal in the persistent storage device.
2 . The method of claim 1 , further comprising:
determining whether the at least one page table journal in the persistent storage device is to be merged with the page table;
in response to determining that the at least one page table journal is to be merged with the page table, merging the page table and the at least one page table journal into an updated page table; and
storing the updated page table in the persistent storage device.
3 . The method of claim 2 , wherein the determining whether the at least one page table journal is to be merged with the page table comprises:
determining whether a merge condition is satisfied; and
in response to the merge condition being satisfied, determining that the at least one page table journal is to be merged with the page table,
wherein the merge condition comprises at least one of:
a time since a last merge of page table journals exceeding a threshold time, or
an amount of the updates of the page table indicated by the at least one page table journal exceeding a threshold amount.
4 . The method of claim 2 , wherein the storing the updated page table in the persistent storage device comprises:
dividing the updated page table into a plurality of blocks;
storing the plurality of blocks in the persistent storage device respectively;
generating a second index structure for recording respective addresses of the plurality of blocks in the persistent storage device; and
storing the second index structure in the persistent storage device.
5 . The method of claim 1 , wherein the first index structure is implemented as a B+ tree.
6 . A method for managing metadata of a storage object, comprising:
reading, from a persistent storage device into a memory by a system comprising a processor, a first index structure for indexing the metadata of the storage object and at least a part of a page table corresponding to the first index structure, wherein the first index structure records a mapping relationship between a first identifier of the storage object and a second identifier of a page where the metadata of the storage object is located, and the page table records a mapping relationship between the second identifier and a page address of the page; and
in response to receiving a first request to access the metadata of the storage object, accessing the metadata of the storage object based on the first index structure and at least the part of the page table.
7 . The method of claim 6 , wherein the page table stored in the persistent device comprises a plurality of blocks and a second index structure for recording respective addresses of the plurality of blocks in the persistent storage device, and reading at least the part of the page table comprises:
reading the second index structure from the persistent storage device.
8 . The method of claim 7 , wherein the accessing the metadata of the storage object comprises:
extracting the first identifier of the storage object from the first request;
determining, by searching the first index structure, the second identifier of the page where the metadata of the storage object is located;
determining, from the plurality of blocks, a block associated with the page based on the second identifier;
determining an address of the block in the persistent storage device by searching the second index structure;
reading the block from the address in the persistent storage device;
searching the block for a page address of the page based on the second identifier; and
accessing the metadata of the storage object from the page address in the persistent storage device.
9 . The method of claim 7 , further comprising:
reading, based on the second index structure, the plurality of blocks from the persistent storage device into the memory to restore the page table in the memory, wherein the first index structure further indexes further metadata of a further storage object; and
in response to receiving a second request to access the further metadata of the further storage object, accessing the further metadata of the further storage object based on the first index structure and the page table.
10 . The method of claim 6 , wherein the page table stored in the persistent device comprises a previous page table and at least one page table journal for recording updates of the page table relative to the previous page table, wherein the previous page table comprises a plurality of blocks and a second index structure for recording respective addresses of the plurality of blocks in the persistent storage device, wherein the reading at least the part of the page table comprises reading the at least one page table journal and the second index structure from the persistent storage device, and wherein the accessing the metadata of the storage object comprises:
extracting the first identifier of the storage object from the first request;
determining, by searching the first index structure, the second identifier of the page where the metadata of the storage object is located;
searching the at least one page table journal for a page address of the page based on the second identifier; and
in response to the page address of the page being found in the at least one page table journal, accessing the metadata of the storage object from the page address in the persistent storage device.
11 . The method of claim 10 , further comprising:
in response to the page address of the page not being found in the at least one page table journal, determining, from the plurality of blocks, a block associated with the page based on the second identifier;
determining an address of the block in the persistent storage device by searching the second index structure;
reading the block from the address in the persistent storage device;
searching the block for a page address of the page based on the second identifier; and
accessing the metadata of the storage object from the page address in the persistent storage device.
12 . The method of claim 10 , further comprising:
reading, based on the second index structure, the plurality of blocks from the persistent storage device into the memory to restore the previous page table in the memory; and
restoring the page table in the memory by merging the previous page table and the at least one page table journal.
15 . The method of claim 6 , wherein the first index structure is implemented as a B+ tree.
16 . An apparatus for managing metadata of a storage object, comprising:
at least one processing unit;
at least one memory coupled to the at least one processing unit and storing instructions for execution by the at least one processing unit, the instructions, when executed by the at least one processing unit, causing the apparatus to perform actions comprising:
in response to the metadata of the storage object being updated, updating a first index structure for indexing the metadata of the storage object and a page table corresponding to the first index structure in a memory, the updating the first index structure resulting in an updated first index structure, wherein the first index structure records a mapping relationship between a first identifier of the storage object and a second identifier of a page where the metadata of the storage object is located, wherein the page table records a mapping relationship between the second identifier and a page address of the page, and wherein the first index structure and the page table have been stored in a persistent storage device;
recording updates of the page table in at least one page table journal; and
storing the updated first index structure and the at least one page table journal in the persistent storage device.
17 . The apparatus of claim 16 , wherein the actions further comprise:
determining whether the at least one page table journal in the persistent storage device is to be merged with the page table;
in response to determining that the at least one page table journal is to be merged with the page table, merging the page table and the at least one page table journal into an updated page table; and
storing the updated page table in the persistent storage device.
18 . An apparatus for managing metadata of a storage object, comprising:
at least one processing unit;
at least one memory coupled to the at least one processing unit and storing instructions for execution by the at least one processing unit, the instructions, when executed by the at least one processing unit, causing the apparatus to perform actions comprising:
reading, from a persistent storage device into a memory, a first index structure for indexing the metadata of the storage object and at least a part of a page table corresponding to the first index structure, wherein the first index structure records a mapping relationship between a first identifier of the storage object and a second identifier of a page where the metadata of the storage object is located, and wherein the page table records a mapping relationship between the second identifier and a page address of the page; and
in response to receiving a first request to access the metadata of the storage object, accessing the metadata of the storage object based on the first index structure and at least the part of the page table.
19 . The apparatus of claim 18 , wherein the page table stored in the persistent device comprises blocks and a second index structure for recording respective addresses of the blocks in the persistent storage device, wherein the reading at least the part of the page table comprises reading the second index structure from the persistent storage device, and wherein the actions further comprise reading, based on the second index structure, the blocks from the persistent storage device into the memory to restore the page table in the memory.
20 . The apparatus of claim 18 , wherein the first index structure is implemented as a B+ tree.