Systems and methods for data replication synchronization
Systems and methods are provided for storing a first data object comprising a first set of immutable components, the first data object being associated with a corresponding second data object stored by a remote replication system. A difference is determined between the first set of immutable components of the first data object and a second set of immutable components of the corresponding second data object. A subset of immutable components is identified from the first set of immutable components based on the difference. The subset of immutable components from the first set of immutable components is provided to the remote replication system over a communication network.
1 . A system comprising:
a first server communicatively coupled to different servers and to one or more clients;
the different servers;
the one or more clients;
one or more processors; and
a memory storing instructions that, when executed by the one or more processors, cause the system to perform:
storing a first data object at the first server, the first data object being associated with a corresponding second data object stored by a second server of the one or more different servers and a corresponding third data object stored by a third server of the one or more different servers;
receiving an update to the second data object from the second server;
generating a version vector representing the updated second data object;
generating a condensed form of the version vector, wherein the condensed form hides second server client information of any associated clients of the second server;
transmitting the condensed form of the version vector to the first server or the third server;
prior to or during scheduling of one or more import actions from the second server to the first server and scheduling of one or more export actions from the first server to the third server:
generating a condensed import version vector representing an import data object to be imported by the one or more import actions, wherein the condensed import version vector hides the second server client information or hides different client information of any associated clients of a different server besides the second server; and
generating a condensed export version vector representing an export data object to be exported by the one or more export actions, wherein the condensed export version vector hides first server client information of any associated clients of the first server, hides the second server client information, or hides the different client information; and
selectively scheduling the one or more import action and the one or more export actions, according to ordering rules of the one or more import actions from the second server to the first server relative to the one or more export actions from the first server to the third server to synchronize, among the first server, the second server and the third server, the first data object, the second data object, and the third data object.
2 . The system of claim 1 , wherein hiding of second server client information results in identity information of any second associated clients associated with the second server being invisible from a perspective of the first server, and wherein identity information of any third associated clients associated with the third server is invisible from the perspective of the first server.
3 . The system of claim 1 , wherein the selective scheduling comprises refraining from reordering a priority of an import action or an export action with respect to a different import action.
4 . The system of claim 1 , wherein the selective scheduling comprises refraining from reordering a first export action to occur ahead of a second export action.
5 . The system of claim 1 , wherein the selective scheduling is based on a predicted saturation state of a network connection resulting from an export action.
6 . The system of claim 1 , wherein the instructions further cause the system to:
hold, at the first server, separate version clock acknowledgements for each of different data objects stored within the first server, and for different updates from each of the different servers;
compare the condensed form of the version vector associated with an update to the second data object or the third data object to a condensed form of a local version vector of the first server associated with an update to the first data object to determine any concurrent changes;
upon detecting a difference between the condensed form of the version vector associated with the update, hold the update to the first data object from replication to the second server and to the third server;
resolve a conflict associated with the concurrent changes; and
in response to resolving the conflict, update the condensed form of the local version vector of the first server and transmit the updated first data object and the updated condensed form of the local version vector to the second server or the third server.
7 . The system of claim 1 , wherein the instructions further cause the system to update the first data object at the first server in response to the update to the second data object and propagate the updated first data object to a first client without propagating the updated first data object to any other clients.
8 . The system of claim 1 , wherein the selectively scheduling comprises:
determining that a local update to the first data object takes precedence over the update received from the second server;
updating the first data object based on the local update;
propagating the updated first data object to the second server; and
propagating the updated first data object to the third server without propagating the updated second data object.
9 . The system of claim 1 , wherein the selectively scheduling is based on a predicted transfer time of propagating the second update to the first server and the third server, the selectively scheduling comprising refraining from scheduling in response to the predicted transfer time being longer than a threshold time.
10 . The system of claim 1 , wherein the update comprises a first update, and the condensed form of the version vector comprises a first condensed form of the version vector; and the instructions that, when executed by the one or more processors, cause the system to perform:
receiving an indication of an additional client of the second server;
receiving a second update to the first updated second data object from the second server;
generating a second version vector representing the second updated second data object;
generating a second condensed form of the second version vector, wherein the second condensed form hides second server client information regarding an identity of the additional client;
and transmitting the second condensed form of the second version vector to the first server or the third server.
11 . The system of claim 1 , wherein the second server client information comprises identify information of one or more identities of the any associated clients of the second server.
12 . A method being implemented by a computing system including one or more physical processors and storage media storing machine-readable instructions, the method comprising:
storing, at a first server, a first data object, the first data object being associated with a corresponding second data object stored by a second server of the one or more different servers and a corresponding third data object stored by a third server of the one or more different servers;
receiving an update to the second data object from the second server;
generating a version vector representing the updated second data object;
generating a condensed form of the version vector, wherein the condensed form hides second server client information of any associated clients of the second server;
transmitting the condensed form of the version vector to the first server or the third server;
prior to or during scheduling of one or more import actions from the second server to the first server and scheduling of one or more export actions from the first server to the third server:
generating a condensed import version vector representing an import data object to be imported by the one or more import actions, wherein the condensed import version vector hides the second server client information or hides different client information of any associated clients of a different server besides the second server; and
generating a condensed export version vector representing an export data object to be exported by the one or more export action, wherein the condensed export version vector hides first server client information of any associated clients of the first server, hides the second server client information, or hides the different client information; and
selectively scheduling the one or more import action and the one or more export actions, according to ordering rules of the one or more import actions from the second server to the first server relative to the one or more export actions from the first server to the third server to synchronize, among the first server, the second server and the third server, the first data object, the second data object, and the third data object.
13 . The method of claim 12 , wherein hiding of second server client information results in identity information of any second associated clients associated with the second server being invisible from a perspective of the first server, and wherein identity information of any third associated clients associated with the third server is invisible from the perspective of the first server.
14 . The method of claim 12 , wherein the selective scheduling is based on a predicted saturation state of a network connection resulting from an export action.
15 . The method of claim 12 , further comprising:
holding, at the first server, separate version clock acknowledgements for each of different data objects stored within the first server, and for different updates from each of the different servers;
comparing the condensed form of the version vector associated with an update to the second data object or the third data object to a condensed form of a local version vector of the first server associated with an update to the first data object to determine any concurrent changes;
upon detecting a difference between the condensed form of the version vector associated with the update, holding the update to the first data object from replication to the second server and to the third server;
resolving a conflict associated with the concurrent changes; and
in response to resolving the conflict, updating the condensed form of the local version vector of the first server and transmit the updated first data object and the updated condensed form of the local version vector to the second server or the third server.
16 . The method of claim 12 , further comprising:
updating the first data object at the first server in response to the update to the second data object and propagating the updated first data object to a first client without propagating the updated first data object to any other clients.
17 . The method of claim 12 , wherein the selectively scheduling comprises:
determining that a local update to the first data object takes precedence over the update received from the second server;
updating the first data object based on the local update;
propagating the updated first data object to the second server; and
propagating the updated first data object to the third server without propagating the updated second data object.
18 . The method of claim 12 , wherein the selectively scheduling is based on a predicted transfer time of propagating the second update to the first server and the third server, the selectively scheduling comprising refraining from scheduling in response to the predicted transfer time being longer than a threshold time.
19 . The system of claim 10 , wherein the instructions that, when executed by the one or more processors, cause the system to perform:
receiving a third update to the second updated second data object from the third server;
generating a third version vector representing the third updated second data object;
generating a third condensed form of the third version vector, wherein the third condensed form hides third server client information regarding one or more identities of clients of the third server; and
transmitting the third condensed form of the third update to the first server or the second server.