Replication barriers for dependent data transfers between data stores

1. A system, comprising:

at least one processor;

at least one memory;

an operations log, stored in the at least one memory, configured to store a plurality of time-based log operation entries for completed storage operations targeting a first plurality of data objects in a first object data store, wherein the first plurality of data objects includes a dependency set of data objects;

a barrier engine, stored in the at least one memory for execution by the at least one processor, configured to:

identify, based on timestamps in the plurality of time-based log operation entries for the dependency set of data objects, a dependency between the dependency set of data objects and a barrier object in the first plurality of data objects;

identify the barrier object in the operations log; and

set, in a time-based operations log entry for the barrier object in the operations log, a barrier object identifier value; and

a replication manager, stored in the at least one memory for execution by the at least one processor, configured to transfer the first plurality of data objects from the first object data store to a second object data store by:

transferring the dependency set of data objects to the second object data store;

determining, responsive to the barrier object identifier value in the operations log, the barrier object;

delaying, while the replication manager is transferring the dependency set of data objects to the second object data store, the transfer of the barrier object to the second object data store; and

releasing the barrier object responsive to completion of transferring the dependency set of data objects, wherein the timestamps for the dependency set of data objects are earlier than a timestamp for the barrier object in the plurality of time-based log operation entries.

2. The system of claim 1 , wherein the replication manager is further configured to use a plurality of replication engines operating in parallel to transfer the first plurality of data objects.

3. The system of claim 2 , wherein:

at least one later data object from the plurality of data objects:

has a first time value later than a second time value of the barrier object; and

is not included in the dependency set of data objects; and

the plurality of replication engines is configured to transfer the at least one later data object before the barrier object.

4. The system of claim 1 , wherein the barrier object comprises a snapshot of the dependency set of data objects.

5. The system of claim 4 , wherein the barrier engine is further configured to generate the barrier object based on the dependency set of data objects.

6. The system of claim 1 , wherein:

the first object data store is further configured to include a plurality of dependency sets in the first plurality of data objects over an operating period;

the barrier engine is further configured to identify a plurality of barrier objects for the plurality of dependency sets at predetermined intervals during the operating period; and

the replication manager is further configured to transfer each dependency set of the plurality of dependency sets prior to transferring each corresponding barrier object of the plurality of barrier objects.

7. The system of claim 1 , wherein the barrier engine is further configured to receive a dependent object identifier for the barrier object from a client application.

8. The system of claim 1 , wherein the barrier engine is further configured to identify a dependent object identifier from metadata for the barrier object.

9. The system of claim 1 , wherein the barrier engine is further configured to:

remove, responsive to the replication manager completing transfer of the dependency set of data objects, the barrier object identifier value from the time-based operation log entry for the barrier object.

10. A computer-implemented method, comprising:

storing a first plurality of data objects in a first object data store, wherein the first plurality of data objects includes a dependency set of data objects;

identifying, based on timestamps in time-based log operation entries for the dependency set of data objects, a dependency between the dependency set of data objects and a barrier object in the first plurality of data objects;

identifying the barrier object in an operations log, wherein the operations log stores a plurality of time-based log operation entries for completed storage operations targeting the first plurality of data objects in the first object data store;

setting, in a time-based operations log entry for the barrier object in the operations log, a barrier object identifier value; and

transferring the first plurality of data objects from the first object data store to a second object data store by:

transferring the dependency set of data objects to the second object data store;

determining, responsive to the barrier object identifier value in the operations log, the barrier object;

delaying, while transferring the dependency set of data objects to the second object data store, the transfer of the barrier object to the second object data store; and

releasing the barrier object responsive to completion of transferring the dependency set of data objects, wherein the timestamps for the dependency set of data objects are earlier than a timestamp for the barrier object in the time-based log operation entries.

11. The computer-implemented method of claim 10 , wherein transferring the dependency set of data objects to the second object data store includes using a plurality of replication engines operating in parallel to transfer the dependency set of data objects.

12. The computer-implemented method of claim 11 , wherein:

at least one later data object from the first plurality of data objects:

has a first time value later than a second time value of the barrier object; and

is not included in the dependency set of data objects; and

the plurality of replication engines transfers the at least one later data object before the barrier object.

13. The computer-implemented method of claim 10 , wherein the barrier object comprises a snapshot of the dependency set of data objects.

14. The computer-implemented method of claim 13 , further comprising:

generating the barrier object based on the dependency set of data objects.

15. The computer-implemented method of claim 10 , further comprising:

storing a plurality of dependency sets in the first data store over an operating period;

identifying a plurality of barrier objects for the plurality of dependency sets at predetermined intervals during the operating period; and

transferring each dependency set of the plurality of dependency sets prior to transferring each corresponding barrier object of the plurality of barrier objects.

16. The computer-implemented method of claim 10 , further comprising:

receiving a dependent object identifier for the barrier object from a client application.

17. The computer-implemented method of claim 10 , further comprising:

detecting a dependent object identifier from metadata for the barrier object.

18. The computer-implemented method of claim 10 , further comprising:

detecting a dependent object identifier from a data object name for the barrier object.

19. The computer-implemented method of claim 10 , further comprising:

removing, responsive to completing transfer of the dependency set of data objects, the barrier object identifier value from the time-based log operation entry for the barrier object.

20. A system, comprising:

at least one processor;

at least one memory;

an operations log, stored in the at least one memory, configured to store a plurality of time-based log operation entries for completed storage operations targeting a plurality of data objects in a first object data store, wherein the plurality of data objects includes a dependency set of data objects;

means for identifying, based on timestamps in the plurality of time-based log operation entries for the dependency set of data objects, a dependency between the dependency set of data objects and a barrier object in the plurality of data objects;

means for identifying the barrier object in the operations log;

means for setting, in a time-based operations log entry for the barrier object in the operations log, a barrier object identifier value; and

means for transferring the plurality of data objects from the first object data store to a second object data store by:

transferring the dependency set of data objects to the second object data store;

determining, responsive to the barrier object identifier value in the operations log, the barrier object;

delaying, while transferring the dependency set of data objects to the second object data store, the transfer of the barrier object to the second object data store; and

releasing the barrier object responsive to completion of transferring the dependency set of data objects, wherein the timestamps for the dependency set of data objects are earlier than a timestamp for the barrier object in the plurality of time-based log operation entries.