Data cluster migration using an incremental synchronization

1. A method comprising:

performing, by a computing device, a replication process using an image of a first data set to replicate the image of the first data set to a second data set;

receiving, by the computing device, a set of events that occurred to alter data in the first data set after the image of the first data set is taken, wherein the set of events describes a set of operators that was performed on the first data set;

analyzing, by the computing device, the set of events to determine if an exception to a first set of rules for performing the set of operators on the second data set for the set of events occurs, wherein the exception occurs when one or more of the set of operators are processed for the set of events after a delay, and wherein the delay from performing an operator in the set of operators is a result of a file not existing in the first data set when performing the operator in the set of operators;

selecting, by the computing device, a second set of rules for the exception based on analyzing the set of events; and

processing, by the computing device, the set of operators for the set of events according to the second set of rules on the second data set to synchronize data from the first data set to the second data set, the processing of the set of operators using the second set of rules being different from processing of the set of operators using the first set of rules, wherein processing the set of operators according to the second set of rules comprises:

performing a validation process to validate whether the file is in the first data set, and

marking an operator in the one or more of the set of operators as an operator to skip when the file is not existing in the first data set.

2. The method of claim 1 , further comprising:

selecting the exception from a plurality of exceptions.

3. The method of claim 2 , wherein each exception in the plurality of exceptions is associated with a different second set of rules.

4. The method of claim 1 , wherein:

the file not existing in the first data set is deleted, and

the operator in the set of operators is processed to create the file in the second data set after the file in the first data set is deleted.

5. The method of claim 1 , wherein when the operator in the one or more of the set of operators is marked as the operator to skip, the operator in the one or more of the set of operators is not processed again.

6. The method of claim 1 , wherein:

the exception occurs when an operator in the set of operators is for a rename of a first directory to a second directory that results in not synchronizing a file in the first data set to the second data set.

7. The method of claim 6 , wherein processing the set of operators according to the second set of rules comprises:

splitting a rename directory operator in the set of events into a first sync directory operator that synchronizes files in the first directory and a second sync directory operator that synchronizes files in the second directory.

8. The method of claim 7 , wherein:

the first sync directory operator synchronizes all files in the first directory from the first data set to the second data set, and

the second sync directory operator synchronizes all files in the second directory from the first data set to the second data set.

9. The method of claim 1 , wherein:

the exception occurs when one or more operators are processed out of order for the set of events resulting in an error in processing the set of operators.

10. The method of claim 9 , wherein:

an order of metadata operators is not changed in the second set of rules,

a data operator is not moved before a metadata operator,

a data operator is moveable to after the metadata operator, and

neighboring data operators can exchange order in the set of operators.

11. The method of claim 9 , wherein:

a delete directory operator in the set of operators is processed out of order with a create file operator in the set of operators.

12. A non-transitory computer-readable storage medium having stored thereon computer executable instructions, which when executed by a computing device, cause the computing device to be operable for:

performing a replication process using an image of a first data set to replicate the image of the first data set to a second data set;

receiving a set of events that occurred to alter data in the first data set after the image of the first data set is taken, wherein the set of events describes a set of operators that was performed on the first data set;

analyzing the set of events to determine if an exception to a first set of rules for performing the set of operators on the second data set for the set of events occurs, wherein the exception occurs when one or more of the set of operators are processed for the set of events after a delay, and wherein the delay from performing an operator in the set of operators is a result of a file not existing in the first data set when performing the operator in the set of operators;

selecting a second set of rules for the exception based on analyzing the set of events; and

processing the set of operators for the set of events according to the second set of rules on the second data set to synchronize data from the first data set to the second data set, the processing of the set of operators using the second set of rules being different from processing of the set of operators using the first set of rules, wherein processing the set of operators according to the second set of rules comprises:

performing a validation process to validate whether the file is in the first data set, and

marking an operator in the one or more of the set of operators as an operator to skip when the file is not existing in the first data set.

13. The non-transitory computer-readable storage medium of claim 12 , wherein:

the exception occurs when an operator in the set of operators is for a rename of a first directory to a second directory that results in not synchronizing a file in the first data set to the second data set.

14. The non-transitory computer-readable storage medium of claim 13 , wherein processing the set of operators according to the second set of rules comprises:

splitting a rename directory operator in the set of events into a first sync directory operator that synchronizes files in the first directory and a second sync directory operator that synchronizes files in the second directory.

15. The non-transitory computer-readable storage medium of claim 12 , wherein:

the exception occurs when one or more operators are processed out of order for the set of events resulting in an error in processing the set of operators.

16. The non-transitory computer-readable storage medium of claim 15 , wherein:

an order of metadata operators is not changed in the second set of rules,

a data operator is not moved before a metadata operator,

a data operator is moveable to after the metadata operator, and

neighboring data operators can exchange order in the set of operators.

17. The non-transitory computer-readable storage medium of claim 12 , wherein:

the file not existing in the first data set is deleted, and

the operator in the set of operators is processed to create the file in the second data set after the file in the first data set is deleted.

18. The non-transitory computer-readable storage medium of claim 12 , wherein when the operator in the one or more of the set of operators is marked as the operator to skip, the operator in the one or more of the set of operators is not processed again.

19. The non-transitory computer-readable storage medium of claim 12 , further operable for:

selecting the exception from a plurality of exceptions.

20. An apparatus comprising:

one or more computer processors; and

a computer-readable storage medium comprising instructions for controlling the one or more computer processors to be operable for:

performing a replication process using an image of a first data set to replicate the image of the first data set to a second data set;

receiving a set of events that occurred to alter data in the first data set after the image of the first data set is taken, wherein the set of events describes a set of operators that was performed on the first data set;

analyzing the set of events to determine if an exception to a first set of rules for performing the set of operators on the second data set for the set of events occurs, wherein the exception occurs when one or more of the set of operators are processed for the set of events after a delay, and wherein the delay from performing an operator in the set of operators is a result of a file not existing in the first data set when performing the operator in the set of operators;

selecting a second set of rules for the exception based on analyzing the set of events; and

processing the set of operators for the set of events according to the second set of rules on the second data set to synchronize data from the first data set to the second data set, the processing of the set of operators using the second set of rules being different from processing of the set of operators using the first set of rules, wherein processing the set of operators according to the second set of rules comprises:

performing a validation process to validate whether the file is in the first data set, and

marking an operator in the one or more of the set of operators as an operator to skip when the file is not existing in the first data set.