Facilitating existing item determinations
Embodiments are directed towards managing and tracking item identification of a plurality of items to determine if an item is a new or existing item, where an existing item has been previously processed. In some embodiments, two or more item identifiers may be generated. In one embodiment, generating the two or more item identifiers may include analyzing the item using a small item size characteristic, a compressed item, or for an identifier collision. The two or more item identifiers may be employed to determine if the item is a new or existing item. In one embodiment, the two or more item identifiers may be compared to a record about an existing item to determine if the item is a new or existing item. If the item is an existing item, then the item may be further processed to determine if the existing item has actually changed.
1. A computer-implemented method comprising:
determining, on a computing device, that an item is waiting to be processed;
determining an item identifier for the item;
using the identifier to find a record associated with the item;
obtaining, from the record associated with the item, a stored end of the item identifier associated with the item, wherein an end of the item identifier is a unique identifier based on content of the item and is stored for the item during previous processing of the item;
obtaining, from the record associated with the item, a stored end of a pointer associated with the item, wherein the stored end of the pointer references an end of the item stored during previous processing of the item;
determining a current end of the item identifier associated with the item, wherein determining the current end includes using the stored end of the pointer;
comparing the stored end of the item identifier with the current end of the item identifier;
determining that the item is an existing item when the stored end of the item identifier matches the current end of the item identifier; and
providing the item to a calling process to be processed as the existing item.
2. The computer-implemented method of claim 1 , wherein determining the item identifier includes using a name associated with the item.
3. The computer-implemented method of claim 1 , wherein determining the item identifier includes using content associated with the item.
4. The computer-implemented method of claim 1 further comprising determining that the item is a new item when the stored end of the item identifier does not match the current end of the item identifier.
5. The computer-implemented method of claim 1 further comprising decompressing the item when the item was previously compressed.
6. The computer-implemented method of claim 1 further comprising:
determining a size of the item;
comparing the stored end of the pointer with the size of the item; and
determining a stored item name identifier associated with the item when the stored end of the pointer is greater than the size of the item, wherein an item name identifier corresponding to an item is stored when an associated item is previously processed.
7. The computer-implemented method of claim 6 further comprising:
determining a current item name identifier using a name associated with the item;
comparing the stored item name identifier with the current item name identifier; and
determining that the item is an existing item when the stored item name identifier matches the current name identifier.
8. The computer-implemented method of claim 6 further comprising:
determining a current item name identifier using an item name associated with the item;
comparing the stored item name identifier with the current item name identifier; and
indicating an unresolved identifier when the stored item name identifier does not match the current name identifier, wherein an unresolved identifier indicates that the item cannot be determined to be a new item or an existing item.
9. The computer-implemented method of claim 1 , further comprising:
determining that the item is complete;
determining a possible change to content associated with the item; and
scheduling the item for re-evaluation.
10. The computer-implemented method of claim 1 , wherein the item is scheduled for re-evaluation at a higher frequency when a possible change to the content of the item is determined.
11. One or more computer-readable storage media having instructions stored thereon, wherein the instructions, when executed by a computing device, cause the computing device to:
determine, on a computing device, that an item is waiting to be processed;
determine an item identifier for the item;
use the identifier to find a record associated with the item;
obtain, from the record associated with the item, a stored end of the item identifier associated with the item, wherein an end of the item identifier is a unique identifier based on content of the item and is stored for the item during previous processing of the item;
obtain, from the record associated with the item, a stored end of a pointer associated with the item, wherein the stored end of the pointer references an end of the item stored during previous processing of the item;
determine a current end of the item identifier associated with the item, wherein determining the current end includes using the stored end of the pointer;
compare the stored end of the item identifier with the current end of the item identifier;
determine that the item is an existing item when the stored end of the item identifier matches the current end of the item identifier; and
provide the item to a calling process to be processed as the existing item.
12. The one or more computer-readable storage media of claim 11 , wherein determining the item identifier includes using a name associated with the item.
13. The one or more computer-readable storage media of claim 11 , wherein determining the item identifier includes using content associated with the item.
14. The one or more computer-readable storage media of claim 11 , wherein the instructions further cause the computing device to determine that the item is a new item when the stored end of the item identifier does not match the current end of the item identifier.
15. The one or more computer-readable storage media of claim 11 , wherein the instructions further cause the computing device to decompress the item when the item was previously compressed.
16. The one or more computer-readable storage media of claim 11 , wherein the instructions further cause the computing device to:
determine a size of the item;
compare the stored end of the pointer with the size of the item; and
determine a stored item name identifier associated with the item when the stored end of the pointer is greater than the size of the item, wherein an item name identifier corresponding to an item is stored when an associated item is previously processed.
17. A computing system comprising:
one or more processors; and
a memory coupled with the one or more processors, the memory having instructions stored thereon, wherein the instructions, when executed by the one or more processors, cause the computing system to:
determine, on a computing device, that an item is waiting to be processed;
determine an item identifier for the item;
use the identifier to find a record associated with the item;
obtain, from the record associated with the item, a stored end of the item identifier associated with the item, wherein an end of the item identifier is a unique identifier based on content of the item and is stored for the item during previous processing of the item;
obtain, from the record associated with the item, a stored end of a pointer associated with the item, wherein the stored end of the pointer references an end of the item stored during previous processing of the item;
determine a current end of the item identifier associated with the item, wherein determining the current end includes using the stored end of the pointer;
compare the stored end of the item identifier with the current end of the item identifier;
determine that the item is an existing item when the stored end of the item identifier matches the current end of the item identifier; and
provide the item to a calling process to be processed as the existing item.
18. The computing system of claim 17 , wherein the item is scheduled for re-evaluation at a higher frequency when a possible change to the content of the item is determined.
19. The computing system of claim 17 , wherein determining the item identifier includes using a name associated with the item.
20. The computing system of claim 17 , wherein determining the item identifier includes using content associated with the item.