Hierarchical coherency log for managing a distributed data storage system

1. A computing device comprising:

a memory containing machine readable medium comprising machine executable code having stored thereon instructions for performing a method of managing a coherency relationship in a distributed storage environment;

a processor coupled to the memory, the processor configured to execute the machine executable code to cause the processor to:

controlling a mirror storage device containing a mirror dataset, wherein the mirror dataset includes a coherent copy of a primary dataset at a primary storage device, wherein controlling the mirror storage device includes implementing a hierarchical change log that tracks a coherence state for the mirror dataset, wherein the hierarchical change log includes a first sub-log and a second sub-log, wherein the first sub-log and the second sub-log are each independently configured based on one of: a property of a corresponding data transaction, a property of the mirror dataset, and a property of a query, and wherein a block range of the first sub-log overlaps a block range of the second sub-log; and

performing and input/output (I/O) operation to modify a portion of the mirror dataset, wherein the I/O operation affects the block range, including determining that the I/O operation affects the block range in a higher-priority one of the first or second sub-logs and in response to the determining creating a third sub-log for the I/O operation.

2. The computing device of claim 1 , wherein the hierarchical change log defines a priority relationship between the first sub-log and the second sub-log.

3. The computing device of claim 1 , wherein the first sub-log and the second sub-log are different in one of a representation and a block size.

4. The computing device of claim 3 , wherein the first sub-log includes a sparse matrix representation and wherein the second sub-log includes a bitmap representation.

5. A method comprising:

creating a first sub-log of a hierarchical change log containing a coherence state for a mirror relationship between a primary dataset and a mirror dataset, wherein the first sub-log is created based on one of: a property of a corresponding data transaction, a property of the mirror dataset, and a property of a query;

in response to a first data transaction affecting the mirror dataset, creating a second sub-log of the hierarchical change log to track the data transaction, wherein a block range of the second sub-log overlaps a block range of the first sub-log, wherein there is a priority relationship between the first sub-log and the second sub-log, and wherein the second sub-log is created based on one of: the property of the corresponding data transaction, the property of the mirror dataset, and the property of the query; and

performing a second data transaction to modify a portion of the mirror dataset, wherein the second data transaction affects the block range, including determining that the second data transaction affects the block range in a higher-priority one of the first or second sub-logs and in response to the determining creating a third sub-log for the second data transaction.

6. The method of claim 5 , wherein the first sub-log and the second sub-log are different in one of a representation and a block size.

7. The method of claim 5 , wherein the creating of the second sub-log includes optimizing the second sub-log based on one of: a property of the first data transaction, a property of the mirror dataset, and a property of a query of a sub-log.

8. The method of claim 5 , wherein the first data transaction includes a synchronization of the primary dataset and the mirror dataset, and wherein the second sub-log includes a sparse matrix representation of the coherence state.

9. The method of claim 5 further comprising flattening the first sub-log and the second sub-log.

10. The method of claim 9 , wherein the flattening is performed after completing the first data transaction.

11. A non-transitory machine readable medium having stored thereon instructions for performing a method comprising machine executable code which when executed by at least one machine, causes the machine to perform operations comprising:

initiating a first data transaction affecting a mirror dataset;

modifying a higher-priority sub-log of a hierarchical change log in response to the first data transaction, the higher-priority sub-log containing a coherence state for a bit range of the mirror dataset, wherein the modifying extends a bit range of the higher-priority sub-log to overlap a bit range of a lower-priority sub-log of the hierarchical change log, and wherein the higher-priority sub-log and the lower-priority sub-log are each independently configured based on one of: a property of a corresponding data transaction, a property of the mirror dataset, and a property of a query; and

performing a second data transaction to modify a portion of the mirror dataset, wherein the second data transaction affects the bit range, including determining that the second data transaction affects the bit range in the higher-priority sub-log and in response to the determining creating an additional sub-log for the second data transaction, wherein the additional sub-log has a priority higher than the higher-priority sub-log.

12. The non-transitory machine-readable medium of claim 11 , wherein the data transaction includes a synchronization of the primary dataset and the mirror dataset, and wherein the sub-log includes a sparse matrix representation of the coherence state.

13. The non-transitory machine-readable medium of claim 11 , wherein the modifying of the sub-log configures the sub-log based on one of: a property of the data transaction, a property of the mirror dataset, and a property of a query.

14. The non-transitory machine-readable medium of claim 11 , wherein the computer program has further instructions that carry out completing the data transaction and discarding the lower-priority sub-log after completing the data transaction.

15. The non-transitory machine-readable medium of claim 11 , wherein the sub-log and the lower-priority sub-log are different in one of a representation and a block size.