Method, device and computer program product for storing metadata
In an example, a method comprises determining, based on a set of metadata items to be stored, a first sequence and a second sequence, wherein the first sequence and the second sequence each include a plurality of key-value pairs, each key-value pair including a metadata item from the set of metadata items and a keyword corresponding to the metadata item in the set. The method also comprises causing a first co-routine to utilize available computing resources to process the first sequence. The method further comprises in response to an amount of computing resources available for the first co-routine to process the first sequence being below a first threshold, causing a second co-routine to process the second sequence, wherein the second co-routine is different from the first co-routine. This method can increase the utilization rate of the processor and more rapidly stores the metadata into the external storage device.
1. A method of storing metadata, the method comprising:
identifying, by a system comprising a processor and based on a set of metadata items to be stored, a first group of key-value pairs and a second group of key-value pairs, each key-value pair including a metadata item from the set of metadata items and a keyword corresponding to the metadata item;
causing a first co-routine to utilize available computing resources to process the first group of key-value pairs to store the first group of key-value pairs in a storage device; and
in response to an amount of computing resources available for the first co-routine to process the first group of key-value pairs being below a first threshold amount of available computing resources, causing a second co-routine to process the second group of key-value pairs to store the second group of key-value pairs in the storage device wherein the second co-routine is different from the first co-routine.
2. The method of claim 1 , further comprising:
in response to a number of Input/Output operations associated with the first group of key-value pairs and processed by the first co-routine exceeding a threshold number of Input/Output operations, determining that the amount of computing resources used by the first co-routine is below the first threshold amount of available computing resources.
3. The method of claim 1 , further comprising:
in response to the amount of computing resources available for the first co-routine to process the first group of key-value pairs being below the first threshold amount of available computing resources, causing the first co-routine to stop processing the first group of key value pairs.
4. The method of claim 3 , further comprising:
in response to the amount of computing resources available for the first co-routine to process the first group of key-value pairs exceeding the first threshold amount of available computing resources, causing the first co-routine to continue the processing of the first group of key-value pairs.
5. The method of claim 1 , wherein the causing the first co-routine to process the first group of key-value pairs comprises causing the first co-routine to:
determine, in a storage device, an associated key-value pair matching a first key-value pair in the first group of key-value pairs, a first keyword in the first key-value pair matching a second keyword in the associated key-value pair;
obtain a first set of key-value pairs from a critical storage page in the storage device, the critical storage page including the associated key-value pair;
update the first set of key-value pairs with the first key-value pair, resulting in an updated first set of key-value pairs; and
in response to the first co-routine and the second co-routine completing processing of the first group of key-value pairs and the second group of key-value pairs, store the updated first set of key-value pairs into the critical storage page.
6. The method of claim 5 , wherein the causing the first co-routine to process the first group of key-value pairs comprises causing the first co-routine to:
determine, in a storage device, a group of associated key-value pairs matching remaining key-value pairs in the first group of key-value pairs other than the first key-value pair, keywords in the remaining key-value pairs matching keywords in the group of associated key-value pairs respectively;
obtain a group of storage pages to be updated from the storage device, the group of storage pages including the group of associated key-value pairs and excluding the critical storage page;
update, with the remaining key-value pairs, a second set of key-value pairs obtained from the group of storage pages to be updated, resulting in an updated second set of key-value pairs;
store the updated second set of key-value pairs into a group of target storage pages respectively; and
cause the storage device to utilize addresses of the group of target storage pages to update addresses of the group of storage pages to be updated.
7. An electronic device for storing metadata, comprises:
a processor; and
a memory having computer program instructions stored thereon, the processor executing the computer program instructions in a memory to control the electronic device to perform acts comprising:
identifying, based on a set of metadata items to be stored, a first plurality of key-value pairs and a second plurality of key-value pairs, each key-value pair including a metadata item from the set of metadata items and a keyword corresponding to the metadata item;
causing a first co-routine to utilize available computing resources to process the first plurality of key-value pairs to store the first plurality of key-value pairs in a storage device; and
in response to an amount of computing resources available for the first co-routine to process the first plurality of key-value pairs being below a first threshold amount of available computing resources, causing a second co-routine to process the second plurality of key-value pairs to store the second plurality of key-value pairs in the storage device, wherein the second co-routine is different from the first co-routine.
8. The electronic device of claim 7 , wherein the actions further comprise:
in response to a number of Input/Output operations associated with the first plurality of key-value pairs and processed by the first co-routine exceeding a threshold number of Input/Output operations, determining that the amount of computing resources used by the first co-routine is below the first threshold amount of available computing resources.
9. The electronic device of claim 7 , wherein the actions further comprise:
in response to the amount of computing resources available for the first co-routine to process the first plurality of key-value pairs being below the first threshold amount of available computing resources, causing the first co-routine to stop processing the first plurality of key-value pairs.
10. The electronic device of claim 9 , wherein the actions further comprise:
in response to the amount of computing resources available for the first co-routine to process the first plurality of key-value pairs exceeding the first threshold amount of available computing resources, causing the first co-routine to continue the processing of the first plurality of key-value pairs.
11. The electronic device of claim 7 , wherein the causing the first co-routine to process the first plurality of key-value pairs comprises causing the first co-routine to:
determine, in a storage device, an associated key-value pair matching a first key-value pair in the first plurality of key-value pairs, a first keyword in the first key-value pair matching a second keyword in the associated key-value pair;
obtain a first set of key-value pairs from a critical storage page in the storage device, the critical storage page including the associated key-value pair;
update the first set of key-value pairs with the first key-value pair, resulting in an updated first set of key-value pairs; and
in response to the first co-routine and the second co-routine completing processing of the first plurality of key-value pairs and the second plurality of key-value pairs, storing the updated first set of key-value pairs into the critical storage page.
12. The electronic device of claim 7 , wherein the causing the first co-routine to process the first plurality of key-value pairs comprises causing the first co-routine to:
determine, in a storage device, a group of associated key-value pairs matching remaining key-value pairs in the first plurality of key-value pairs other than the first key-value pair, keywords in the remaining key-value pairs matching keywords in the group of associated key-value pairs respectively;
obtain a group of storage pages to be updated from the storage device, the group of storage pages including the group of associated key-value pairs and excluding the critical storage page;
update, with the remaining key-value pairs, a second set of key-value pairs obtained from the group of storage pages to be updated, resulting in an updated second set of key-value pairs;
store the updated second set of key-value pairs into a group of target storage pages respectively; and
cause the storage device to utilize addresses of the group of target storage pages to update addresses of the group of storage pages to be updated.
13. A computer program product being stored on a non-transitory computer-readable medium and comprising machine-executable instructions which, when executed, cause a machine to perform operations, comprising:
identifying, based on metadata items to be stored, a first group of key-value pairs and a second group of key-value pairs, each key-value pair including a metadata item from the metadata items and a keyword corresponding to the metadata item;
causing a first co-routine to utilize available computing resources to process the first group of key-value pairs to store the first group of key-value pairs in a storage device; and
in response to an amount of computing resources available for the first co-routine to process the first group of key-value pairs being below a first threshold amount of available computing resources, causing a second co-routine to process the second group of key-value pairs to store the second group of key-value pairs in the storage device, wherein the second co-routine is different from the first co-routine.
14. The computer program product of claim 13 , wherein the operations further comprising:
in response to a number of Input/Output operations associated with the first group of key-value pairs and processed by the first co-routine exceeding a threshold number of Input/Output operations, determining that the amount of computing resources used by the first co-routine is below the first threshold amount of available computing resources.
15. The computer program product of claim 13 , wherein the operations further comprise:
in response to the amount of computing resources available for the first co-routine to process the first group of key-value pairs being below the first threshold amount of available computing resources, causing the first co-routine to stop processing the first group of key-value pairs.
16. The computer program product of claim 15 , wherein the operations further comprise:
in response to the amount of computing resources available for the first co-routine to process the first group of key-value pairs exceeding the first threshold amount of available computing resources, causing the first co-routine to continue the processing of the first group of key-value pairs.
17. The computer program product of claim 13 , wherein the causing the first co-routine to process the first group of key-value pairs comprises causing the first co-routine to:
determine, in a storage device, an associated key-value pair matching a first key-value pair in the first group of key-value pairs, a first keyword in the first key-value pair matching a second keyword in the associated key-value pair; and
obtain a first set of key-value pairs from a critical storage page in the storage device, the critical storage page including the associated key-value pair.
18. The computer program product of claim 17 , wherein the causing the first co-routine to process the first group of key-value pairs further comprises causing the first co-routine to:
update the first set of key-value pairs with the first key-value pair, resulting in an updated first set of key-value pairs; and
in response to the first co-routine and the second co-routine completing processing of the first group of key-value pairs and the second group of key-value pairs, store the updated first set of key-value pairs into the critical storage page.
19. The computer program product of claim 18 , wherein the causing the first co-routine to process the first group of key-value pairs comprises causing the first co-routine to:
determine, in a storage device, a group of associated key-value pairs matching remaining key-value pairs in the first group of key-value pairs other than the first key-value pair, keywords in the remaining key-value pairs matching keywords in the group of associated key-value pairs respectively;
obtain a group of storage pages to be updated from the storage device, the group of storage pages including the group of associated key-value pairs and excluding the critical storage page; and
update, with the remaining key-value pairs, a second set of key-value pairs obtained from the group of storage pages to be updated, resulting in an updated second set of key-value pairs.
20. The computer program product of claim 19 , wherein the causing the first co-routine to process the first group of key-value pairs comprises causing the first co-routine to:
store the updated second set of key-value pairs into a group of target storage pages respectively; and
cause the storage device to utilize addresses of the group of target storage pages to update addresses of the group of storage pages to be updated.