Data synchronization method, apparatus, and system
Embodiments of the disclosure provide a data synchronization method, apparatus and system. The data synchronization method includes: receiving first modification data in primary storage; adding the first modification data to an asynchronous message queue; and sending the first modification data in the asynchronous message queue to standby storage. Embodiments of the present disclosure add modification data in primary storage to an asynchronous message queue, and send the modification data in the asynchronous message queue to standby storage by enabling one or more message sending threads, so that data synchronization between the primary storage and the standby storage is no longer restricted by the resources of the primary storage. The method reduces the time delay between the primary storage and the standby storage during data synchronization by sending data via multiple threads.
1. A data synchronization method, comprising:
receiving, by a data synchronization apparatus, first modification data sent by a primary storage;
adding the first modification data to an asynchronous message queue of the data synchronization apparatus;
sending, via a message sending thread, the first modification data in the asynchronous message queue to a standby storage;
receiving, from the standby storage, information indicating a successful storage of the first modification data, the information is fed back by the standby storage when a proportion of storage units having the first modification data successfully stored to all storage units of the standby storage reaches a pre-set proportion threshold; and
in response to receiving the information, deleting the first modification data.
2. The method according to claim 1 , wherein receiving, by the data synchronization apparatus, the first modification data sent by the primary storage further comprises:
receiving a data snapshot of the first modification data sent by the primary storage.
3. The method according to claim 1 , further comprising:
receiving, by the data synchronization apparatus, a backlog data query request for backlog data in the asynchronous message queue; and
feeding back the backlog data in the asynchronous message queue to a query art.
4. The method according to claim 1 , wherein sending, via a message sending thread, the first modification data in the asynchronous message queue to the standby storage further comprises:
sending, via a message sending thread, the first modification data in the asynchronous message queue to a standby memory storage, wherein a data expiration time of the standby memory storage is greater than a time delay of the primary storage synchronizing data to a second standby storage, the data expiration time of the standby memory storage indicates a time by which the first modification data stored in the standby memory storage is deleted.
5. A data synchronization apparatus, comprising:
a memory storing a set of instructions; and
a processor configured to execute the set of instructions to cause the data synchronization apparatus to perform:
receiving first modification data sent by a primary storage;
adding the first modification data to an asynchronous message queue of the data synchronization apparatus;
sending, via a message sending thread, the first modification data in the asynchronous message queue to a standby storage;
receiving, from the standby storage, information indicating a successful storage of the first modification data, the information is fed back by the standby storage when a proportion of storage units having the first modification data successfully stored to all storage units of the standby storage reaches a pre-set proportion threshold; and
deleting the first modification data in response to receiving the information.
6. The apparatus according to claim 5 , wherein the processor is further configured to execute the set of instructions to cause the data synchronization apparatus to perform:
receiving a data snapshot of the first modification data sent by the primary storage.
7. The apparatus according to claim 5 , wherein the processor is further configured to execute the set of instructions to cause the data synchronization apparatus to perform:
receiving a backlog data query request for backlog data in the asynchronous message queue; and
feedbacking the backlog data in the asynchronous message queue to a query party.
8. The apparatus according to claim 5 , wherein the processor is further configured to execute the set of instructions to cause the data synchronization apparatus to perform:
sending, via a message sending thread, the first modification data in the asynchronous message queue to standby memory storage, wherein a data expiration time of the standby memory storage is greater than a time delay of the primary storage synchronizing data to second standby storage.
9. A data synchronization system, comprising:
a primary storage;
an asynchronous message queue system; and
a standby storage comprising a standby memory storage and a second standby storage,
wherein the asynchronous message queue system is configured to:
receive first modification data sent by the primary storage;
add the first modification data to an asynchronous message queue;
send, via a message sending thread, the first modification data in the asynchronous message queue to the standby memory storage;
receive, from the standby memory storage, information indicating a successful storage of the first modification data, the information is fed back by the standby memory storage when a proportion of storage units having the first modification data successfully stored to all storage units of the standby storage reaches a pre-set proportion threshold; and
in response to receiving the information, delete the first modification data, and
wherein the second standby storage is configured to receive and store synchronous data of the primary storage.
10. A non-transitory computer readable medium that stores a set of instructions that is executable by at least one processor of a data synchronization device to cause the data synchronization device to perform a data synchronization method, the method comprising:
receiving, by the data synchronization device, first modification data sent by primary storage;
adding the first modification data to an asynchronous message queue of the data synchronization device;
sending, via a message sending thread, the first modification data in the asynchronous message queue to a standby storage;
receiving, from the standby storage, information indicating a successful storage of the first modification data, the information is fed back by the standby storage when a proportion of storage units having the first modification data successfully stored to all storage units of the standby storage reaches a pre-set proportion threshold; and
in response to receiving the information, deleting the first modification data.
11. The non-transitory computer readable medium according to claim 10 , wherein receiving, by the data synchronization device, first modification data sent by the primary storage further comprises:
receiving, by the data synchronization device, a data snapshot of the first modification data sent by the primary storage.
12. The non-transitory computer readable medium according to claim 10 , wherein the set of instructions is executable by the at least one processor of the data synchronization device to cause the data synchronization device to further perform:
receiving, by the data synchronization device, a backlog data query request for backlog data in the asynchronous message queue; and
feeding back the backlog data in the asynchronous message queue to a query party.
13. The non-transitory computer readable medium according to claim 10 , wherein sending, via a message sending thread, the first modification data in the asynchronous message queue to standby storage further comprises:
sending, via a message sending thread, the first modification data in the asynchronous message queue to standby memory storage, wherein a data expiration time of the standby memory storage is greater than a time delay of the primary storage synchronizing data to second standby storage.