Detecting and resolving computer system errors using fast file change monitoring
Embodiments are directed towards managing and tracking item identification of a plurality of items to determine if an item is a new or existing item, where an existing item has been previously processed. In some embodiments, two or more item identifiers may be generated. In one embodiment, generating the two or more item identifiers may include analyzing the item using a small item size characteristic, a compressed item, or for an identifier collision. The two or more item identifiers may be employed to determine if the item is a new or existing item. In one embodiment, the two or more item identifiers may be compared to a record about an existing item to determine if the item is a new or existing item. If the item is an existing item, then the item may be further processed to determine if the existing item has actually changed.
1. A method, comprising:
periodically evaluating, according to a current expiration time value, a file in at least a portion of a file system for changes to the file;
during each evaluation of the periodically evaluating:
calculating a checksum using at least a portion of data contained in the file; and
comparing the checksum to a previously computed checksum corresponding to a prior evaluation of the file;
determining, based at least in part on the comparing the checksum to the previously computed checksum, whether the file has changed;
when the file is determined to have changed:
determining a last accessed position in the file;
analyzing data in the file from the last accessed position to a current end of the file; and
updating the current expiration time value to a first updated expiration time value for a next evaluation of the file, the first updated expiration time value being less than the current expiration time value; and
when the file is determined not to have changed, updating the current expiration time value to a second updated expiration time value for the next evaluation of the file, the second updated expiration time value being equal to or greater than the current expiration time value.
2. The method as recited in claim 1 , further comprising:
storing the analyzed data.
3. The method as recited in claim 1 , wherein the determining whether the file has changed further comprises determining if a current size of the file is different than a previously saved size for the file.
4. The method as recited in claim 1 , wherein, during each evaluation of the periodically evaluating, the method further comprises determining whether metadata associated with the file has changed.
5. The method as recited in claim 1 , wherein, during each evaluation of the periodically evaluating, the method further comprises detecting whether there is a change in at least a portion of a directory listing.
6. The method as recited in claim 1 , wherein the updating the current expiration time value to the first updated expiration time value is based at least in part a frequency of changes detected for the file.
7. The method as recited in claim 1 , wherein, when the second updated expiration time value exceeds a maximum expiration time value, the file is changed to a no action state.
8. The method as recited in claim 1 , wherein the checksum corresponds to a first end of item identifier for a current evaluation of the file and the previously computed checksum corresponds to a second end of item identifier for the prior evaluation of the file.
9. One or more non-transitory computer-readable storage media, storing one or more sequences of instructions, which when executed by one or more processors cause performance of:
periodically evaluating, according to a current expiration time value, a file in at least a portion of a file system for changes to the file;
during each evaluation of the periodically evaluating:
calculating a checksum using at least a portion of data contained in the file; and
comparing the checksum to a previously computed checksum corresponding to a prior evaluation of the file;
determining, based at least in part on the comparing the checksum to the previously computed checksum, whether the file has changed;
when the file is determined to have changed:
determining a last accessed position in the file;
analyzing data in the file from the last accessed position to a current end of the file; and
updating the current expiration time value to a first updated expiration time value for a next evaluation of the file, the first updated expiration time value being less than the current expiration time value; and
when the file is determined not to have changed, updating the current expiration time value to a second updated expiration time value for the next evaluation of the file, the second updated expiration time value being equal to or greater than the current expiration time value.
10. The one or more non-transitory computer-readable storage media as recited in claim 9 , wherein the one or more sequences of instructions, when executed by the one or more processors, further cause performance of storing the analyzed data.
11. The one or more non-transitory computer-readable storage media as recited in claim 9 , wherein the determining whether the file has changed further comprises determining if a current size of the file is different than a previously saved size for the file.
12. The one or more non-transitory computer-readable storage media as recited in claim 9 , wherein the one or more sequences of instructions, when executed by the one or more processors, further cause performance of, during each evaluation of the periodically evaluating, detecting whether there is a change in at least a portion of a directory listing.
13. The one or more non-transitory computer-readable storage media as recited in claim 9 , wherein, when the second updated expiration time value exceeds a maximum expiration time value, the file is changed to a no action state.
14. The one or more non-transitory computer-readable storage media as recited in claim 9 , wherein the checksum corresponds to a first end of item identifier for a current evaluation of the file and the previously computed checksum corresponds to a second end of item identifier for the prior evaluation of the file.
15. An apparatus, comprising:
a file processor, implemented at least partially in hardware, configured to:
periodically evaluate, according to a current expiration time value, a file in at least a portion of a file system for changes to the file;
during each evaluation of the periodically evaluating:
calculate a checksum using at least a portion of data contained in the file; and
compare the checksum to a previously computed checksum corresponding to a prior evaluation of the file;
determine, based at least in part on comparing the checksum to the previously computed checksum, whether the file has changed;
when the file is determined to have changed:
determine a last accessed position in the file;
analyze data in the file from the last accessed position to a current end of the file; and
update the current expiration time value to a first updated expiration time value for a next evaluation of the file, the first updated expiration time value being less the current expiration time value; and
when the file is determined not to have changed, update the current expiration time value to a second updated expiration time value for the next evaluation of the file, the second updated expiration time value being equal to or greater than the current expiration time value.
16. The apparatus as recited in claim 15 , wherein the file processor is further configured to store the analyzed data.
17. The apparatus as recited in claim 15 , wherein the determining whether the file has changed further comprises determining if a current size of the file is different than a previously saved size for the file.
18. The apparatus as recited in claim 15 , wherein, during each evaluation of the periodically evaluating, the file processor is further configured to detect whether there is a change in at least a portion of a directory listing.
19. The apparatus as recited in claim 15 , wherein, when the second updated expiration time value exceeds a maximum expiration time value, the file is changed to a no action state.
20. The apparatus as recited in claim 15 , wherein the checksum corresponds to a first end of item identifier for a current evaluation of the file and the previously computed checksum corresponds to a second end of item identifier for the prior evaluation of the file.