Monitoring changes to data items using associated metadata
Embodiments are directed towards managing and tracking item identification of a plurality of items to determine if an item is a new or existing item, where an existing item has been previously processed. In some embodiments, two or more item identifiers may be generated. In one embodiment, generating the two or more item identifiers may include analyzing the item using a small item size characteristic, a compressed item, or for an identifier collision. The two or more item identifiers may be employed to determine if the item is a new or existing item. In one embodiment, the two or more item identifiers may be compared to a record about an existing item to determine if the item is a new or existing item. If the item is an existing item, then the item may be further processed to determine if the existing item has actually changed.
1. A method, comprising:
comparing, after expiration of a time value corresponding to an item, first metadata corresponding to the item to second metadata corresponding to the item, the first metadata determined prior to the second metadata;
based at least in part on the comparing, determining whether the item has been changed;
updating the time value associated with the item to an updated time value based at least in part on:
decreasing the time value when the item is determined to have changed; or
one of maintaining the time value or increasing the time value when the item is not determined to have changed; and
after expiration of the updated time value, comparing the second metadata to third metadata corresponding to the item, the third metadata determined prior to the second metadata.
2. The method of claim 1 , wherein at least one of the first metadata, the second metadata, or the third metadata corresponds to at least one of an item size, a timestamp, an access time, a creation time, a modification time, or a permissions attribute.
3. The method of claim 1 , further comprising, when another updated time value includes a maximum time value, ceasing comparing metadata corresponding to the item.
4. The method of claim 1 , further comprising:
determining that the item is a new item; and
based at least in part on the item being the new item, setting a default time value as the time value.
5. The method of claim 1 , wherein the determining whether the item has been changed includes determining whether a size of the item has changed.
6. The method of claim 5 , wherein the determining whether the size of the item has changed includes comparing a first end position of the item from the first metadata to a second end position of the item from the second metadata.
7. The method of claim 1 , wherein the method is executed, at least in part, using a computing device in a cloud computing architecture.
8. A system comprising:
one or more processors;
one or more memory devices storing instructions thereon that, when executed by the one or more processors, cause the one or more processors to execute operations comprising:
comparing, after expiration of a time value corresponding to an item, first metadata corresponding to the item to second metadata corresponding to the item, the first metadata determined prior to the second metadata;
based at least in part on the comparing, determining whether the item has been changed;
updating the time value associated with the item to an updated time value based at least in part on:
decreasing the time value when the item is determined to have changed; or
one of maintaining the time value or increasing the time value when the item is not determined to have changed; and
after expiration of the updated time value, comparing the second metadata to third metadata corresponding to the item, the third metadata determined prior to the second metadata.
9. The system of claim 8 , wherein at least one of the first metadata, the second metadata, or the third metadata corresponds to at least one of an item size, a timestamp, an access time, a creation time, a modification time, or a permissions attribute.
10. The system of claim 8 , wherein the operations further comprise, when another updated time value includes a maximum time value, ceasing comparing metadata corresponding to the item.
11. The system of claim 8 , wherein the operations further comprise:
determining that the item is a new item; and
based at least in part on the item being the new item, setting a default time value as the time value.
12. The system of claim 8 , wherein the determining whether the item has been changed includes determining whether a size of the item has changed.
13. The system of claim 12 , wherein the determining whether the size of the item has changed includes comparing a first end position of the item from the first metadata to a second end position of the item from the second metadata.
14. The system of claim 8 , wherein the system is comprised in a cloud computing architecture.
15. One or more non-transitory computer-storage media storing instructions that, when executed by one or more processors, cause the one or more processors to execute operations comprising:
comparing, after expiration of a time value corresponding to an item, first metadata corresponding to the item to second metadata corresponding to the item, the first metadata determined prior to the second metadata;
based at least in part on the comparing, determining whether the item has been changed;
updating the time value associated with the item to an updated time value based at least in part on:
decreasing the time value when the item is determined to have changed; or
one of maintaining the time value or increasing the time value when the item is not determined to have changed; and
after expiration of the updated time value, comparing the second metadata to third metadata corresponding to the item, the third metadata determined prior to the second metadata.
16. The one or more non-transitory computer-storage media of claim 15 , wherein at least one of the first metadata, the second metadata, or the third metadata corresponds to at least one of an item size, a timestamp, an access time, a creation time, a modification time, or a permissions attribute.
17. The one or more non-transitory computer-storage media of claim 15 , wherein the operations further comprise, when another updated time value includes a maximum time value, ceasing comparing metadata corresponding to the item.
18. The one or more non-transitory computer-storage media of claim 15 , wherein the operations further comprise:
determining that the item is a new item; and
based at least in part on the item being the new item, setting a default time value as the time value.
19. The one or more non-transitory computer-storage media of claim 15 , wherein the determining whether the item has been changed includes determining whether a size of the item has changed.
20. The one or more non-transitory computer-storage media of claim 19 , wherein the determining whether the size of the item has changed includes comparing a first end position of the item from the first metadata to a second end position of the item from the second metadata.