Data storage control method, device and storage medium
A data storage control method, a device, and a storage medium are provided. The method includes: constructing an equipment room allocation model for a distributed storage system based on a dependency relationship between storage objects in the distributed storage system, the equipment room allocation strategy is used to indicate a target equipment room allocated to the storage object in the distributed storage system, and the migration evaluation result is used to indicate overheads caused by data transition processing of migration from an original equipment room to the target equipment room for storage; determining, based on the equipment room allocation model, an equipment room allocation strategy corresponding to a minimum migration evaluation result as a target equipment room allocation strategy; and performing data migration on the distributed storage system based on the target equipment room allocation strategy.
1 . A data storage control method, wherein the method comprises:
constructing an equipment room allocation model for a distributed storage system based on a dependency relationship between storage objects in the distributed storage system, wherein the equipment room allocation model is used to output a corresponding migration evaluation result based on an equipment room allocation strategy, the equipment room allocation strategy is used to indicate a target equipment room allocated to the storage object in the distributed storage system, and the migration evaluation result is used to indicate overheads caused by data transition processing of migration from an original equipment room to the target equipment room for storage;
invoking the equipment room allocation model to determine, based on a migration evaluation result corresponding to each equipment room allocation strategy, an equipment room allocation strategy corresponding to a minimum migration evaluation result as a target equipment room allocation strategy; and
performing data migration on the distributed storage system based on the target equipment room allocation strategy.
2 . The method according to claim 1 , wherein the migration evaluation result comprises migration costs corresponding to the distributed storage system and corresponding cross-equipment room access traffic generated after equipment room allocation is performed based on the equipment room allocation strategy, and the migration costs comprise total costs required for migrating the storage object in the distributed storage system from the original equipment room corresponding to the storage object to the target equipment room corresponding to the storage object in the equipment room allocation strategy for storage.
3 . The method according to claim 2 , wherein the storage object comprises a data path and a computing task, and the constructing an equipment room allocation model for a distributed storage system based on a dependency relationship between storage objects in the distributed storage system comprises:
constructing a directed acyclic graph based on a dependency relationship between the data path and the computing task, wherein each node in the directed acyclic graph comprises a data path and a computing task, and when the data path and the computing task are connected by an edge, a first edge weight of the edge is used to represent historical access traffic of the computing task to the data path; and
constructing the equipment room allocation model by using the directed acyclic graph.
4 . The method according to claim 3 , wherein the migration costs are related to migration costs for a single storage object and a number of storage objects to be migrated, and the number of storage objects to be migrated is related to the original equipment room and the equipment room allocation strategy; and
the cross-equipment room access traffic is related to a first edge weight corresponding to a cross-equipment room node group in the directed acyclic graph, the cross-equipment room node group comprises a node where a computing task is located and a node where a data path is located in the directed acyclic graph, which are connected by an edge, and the computing task and the data path in the cross-equipment room node group correspond to different target equipment rooms in the equipment room allocation strategy.
5 . The method according to claim 4 , wherein the invoking the equipment room allocation model to determine, based on a migration evaluation result corresponding to each equipment room allocation strategy, an equipment room allocation strategy corresponding to a minimum migration evaluation result as a target equipment room allocation strategy comprises:
determining, based on the equipment room allocation model under a preset constraint, the equipment room allocation strategy corresponding to the minimum migration evaluation result as the target equipment room allocation strategy, wherein the preset constraint comprises: each storage object is allocated to one target equipment room, a total data amount of a storage object allocated to each target equipment room is less than or equal to a capacity of the target equipment room, and a number of storage objects allocated to each target equipment room is within a preset number range, wherein the preset number range comprises a value range of a ratio of a number of storage objects to a number of equipment rooms in the distributed storage system.
6 . The method according to claim 4 , wherein the performing data migration on the distributed storage system based on the target equipment room allocation strategy comprises:
determining a migration order based on the directed acyclic graph; and
performing data migration on the distributed storage system based on the target equipment room allocation strategy, the original equipment room, and the migration order.
7 . The method according to claim 6 , wherein the determining a migration order based on the directed acyclic graph comprises:
connecting an upstream node of a target node and a downstream node of the target node in the directed acyclic graph, to delete the target node from the directed acyclic graph to obtain a directed graph for migration; and
determining the migration order based on the directed graph for migration.
8 . The method according to claim 7 , wherein the determining the migration order based on the directed graph for migration comprises:
if the directed graph for migration comprises a cyclic structure, determining, based on a first edge weight between the target node and each of the upstream node and the downstream node in the directed acyclic graph, a second edge weight of an edge between the upstream node and the downstream node in the directed graph for migration;
breaking an edge with a smallest second edge weight in the cyclic structure, to obtain a directed acyclic graph for migration; and
determining the migration order based on a dependency relationship between nodes in the directed acyclic graph for migration.
9 . The method according to claim 7 , wherein the target node comprises one of the following: a node where the computing task is located and a node where the data path is located.
10 . The method according to claim 3 , wherein the invoking the equipment room allocation model to determine, based on a migration evaluation result corresponding to each equipment room allocation strategy, an equipment room allocation strategy corresponding to a minimum migration evaluation result as a target equipment room allocation strategy comprises:
determining, based on the equipment room allocation model under a preset constraint, the equipment room allocation strategy corresponding to the minimum migration evaluation result as the target equipment room allocation strategy, wherein the preset constraint comprises: each storage object is allocated to one target equipment room, a total data amount of a storage object allocated to each target equipment room is less than or equal to a capacity of the target equipment room, and a number of storage objects allocated to each target equipment room is within a preset number range, wherein the preset number range comprises a value range of a ratio of a number of storage objects to a number of equipment rooms in the distributed storage system.
11 . The method according to claim 2 , wherein the invoking the equipment room allocation model to determine, based on a migration evaluation result corresponding to each equipment room allocation strategy, an equipment room allocation strategy corresponding to a minimum migration evaluation result as a target equipment room allocation strategy comprises:
determining, based on the equipment room allocation model under a preset constraint, the equipment room allocation strategy corresponding to the minimum migration evaluation result as the target equipment room allocation strategy, wherein the preset constraint comprises: each storage object is allocated to one target equipment room, a total data amount of a storage object allocated to each target equipment room is less than or equal to a capacity of the target equipment room, and a number of storage objects allocated to each target equipment room is within a preset number range, wherein the preset number range comprises a value range of a ratio of a number of storage objects to a number of equipment rooms in the distributed storage system.
12 . The method according to claim 1 , wherein the invoking the equipment room allocation model to determine, based on a migration evaluation result corresponding to each equipment room allocation strategy, an equipment room allocation strategy corresponding to a minimum migration evaluation result as a target equipment room allocation strategy comprises:
determining, based on the equipment room allocation model under a preset constraint, the equipment room allocation strategy corresponding to the minimum migration evaluation result as the target equipment room allocation strategy, wherein the preset constraint comprises: each storage object is allocated to one target equipment room, a total data amount of a storage object allocated to each target equipment room is less than or equal to a capacity of the target equipment room, and a number of storage objects allocated to each target equipment room is within a preset number range, wherein the preset number range comprises a value range of a ratio of a number of storage objects to a number of equipment rooms in the distributed storage system.
13 . An electronic device, comprising at least one processor and at least one memory,
wherein the at least memory stores computer-executable instructions; and
the at least one processor executes the computer-executable instructions stored in the at least one memory, to cause the electronic device to implement a data storage control method and the method comprises:
constructing an equipment room allocation model for a distributed storage system based on a dependency relationship between storage objects in the distributed storage system, wherein the equipment room allocation model is used to output a corresponding migration evaluation result based on an equipment room allocation strategy, the equipment room allocation strategy is used to indicate a target equipment room allocated to the storage object in the distributed storage system, and the migration evaluation result is used to indicate overheads caused by data transition processing of migration from an original equipment room to the target equipment room for storage;
invoking the equipment room allocation model to determine, based on a migration evaluation result corresponding to each equipment room allocation strategy, an equipment room allocation strategy corresponding to a minimum migration evaluation result as a target equipment room allocation strategy; and
performing data migration on the distributed storage system based on the target equipment room allocation strategy.
14 . The electronic device according to claim 13 , wherein the migration evaluation result comprises migration costs corresponding to the distributed storage system and corresponding cross-equipment room access traffic generated after equipment room allocation is performed based on the equipment room allocation strategy, and the migration costs comprise total costs required for migrating the storage object in the distributed storage system from the original equipment room corresponding to the storage object to the target equipment room corresponding to the storage object in the equipment room allocation strategy for storage.
15 . The electronic device according to claim 14 , wherein the storage object comprises a data path and a computing task, and the constructing an equipment room allocation model for a distributed storage system based on a dependency relationship between storage objects in the distributed storage system comprises:
constructing a directed acyclic graph based on a dependency relationship between the data path and the computing task, wherein each node in the directed acyclic graph comprises a data path and a computing task, and when the data path and the computing task are connected by an edge, a first edge weight of the edge is used to represent historical access traffic of the computing task to the data path; and
constructing the equipment room allocation model by using the directed acyclic graph.
16 . The electronic device according to claim 15 , wherein the migration costs are related to migration costs for a single storage object and a number of storage objects to be migrated, and the number of storage objects to be migrated is related to the original equipment room and the equipment room allocation strategy; and
the cross-equipment room access traffic is related to a first edge weight corresponding to a cross-equipment room node group in the directed acyclic graph, the cross-equipment room node group comprises a node where a computing task is located and a node where a data path is located in the directed acyclic graph, which are connected by an edge, and the computing task and the data path in the cross-equipment room node group correspond to different target equipment rooms in the equipment room allocation strategy.
17 . The electronic device according to claim 16 , wherein the performing data migration on the distributed storage system based on the target equipment room allocation strategy comprises:
determining a migration order based on the directed acyclic graph; and
performing data migration on the distributed storage system based on the target equipment room allocation strategy, the original equipment room, and the migration order.
18 . The electronic device according to claim 17 , wherein the determining a migration order based on the directed acyclic graph comprises:
connecting an upstream node of a target node and a downstream node of the target node in the directed acyclic graph, to delete the target node from the directed acyclic graph to obtain a directed graph for migration; and
determining the migration order based on the directed graph for migration.
19 . The electronic device according to claim 13 , wherein the invoking the equipment room allocation model to determine, based on a migration evaluation result corresponding to each equipment room allocation strategy, an equipment room allocation strategy corresponding to a minimum migration evaluation result as a target equipment room allocation strategy comprises:
determining, based on the equipment room allocation model under a preset constraint, the equipment room allocation strategy corresponding to the minimum migration evaluation result as the target equipment room allocation strategy, wherein the preset constraint comprises: each storage object is allocated to one target equipment room, a total data amount of a storage object allocated to each target equipment room is less than or equal to a capacity of the target equipment room, and a number of storage objects allocated to each target equipment room is within a preset number range, wherein the preset number range comprises a value range of a ratio of a number of storage objects to a number of equipment rooms in the distributed storage system.
20 . A non-transitory computer-readable storage medium, storing computer-executable instructions that, when executed by a processor, cause a computing device to implement a data storage control method, and the method comprising:
constructing an equipment room allocation model for a distributed storage system based on a dependency relationship between storage objects in the distributed storage system, wherein the equipment room allocation model is used to output a corresponding migration evaluation result based on an equipment room allocation strategy, the equipment room allocation strategy is used to indicate a target equipment room allocated to the storage object in the distributed storage system, and the migration evaluation result is used to indicate overheads caused by data transition processing of migration from an original equipment room to the target equipment room for storage;
invoking the equipment room allocation model to determine, based on a migration evaluation result corresponding to each equipment room allocation strategy, an equipment room allocation strategy corresponding to a minimum migration evaluation result as a target equipment room allocation strategy; and
performing data migration on the distributed storage system based on the target equipment room allocation strategy.