Grouping and differentiating files based on content
View Patent ↗Methods and apparatus teach a digital spectrum of a file. The digital spectrum is used to map a file's position. This position relative to another file's position reveals distances between the files. Representatively, files have a plurality of symbols representing an underlying data stream of original bits of data. The number of occurrences of each symbol in each file is compared to like symbols in other files. This can occur via algorithms, mapping, or both. In certain instances, comparison reveals a difference in counts between the symbols of the files. This difference is then squared, added together, and a square root taken. Comparing “distance values” reveals file adjacency, grouping, or the like. Also, normalizing, weighting, filtering functions and/or other statistical computations are applied in certain instances.
1. In a computing system environment, a method of differentiating files stored on one or more computing devices, each file having a plurality of symbols derived from an underlying data stream of all original bits of raw data of said each file, comprising:
encoding said each file as a plurality of symbols representing an underlying data stream of all original bits of binary data of the file;
determining a number of occurrences of each said symbol in said each file; and
computing a distance between said each file and every other file based on the determined number of occurrences.
2. The method of claim 1 , further including normalizing the determined number of occurrences for said each symbol by comparing to a total number of symbols in a source file.
3. The method of claim 1 , further including weighting the determined number of occurrences for said each symbol according to a bit length of said original bits of data represented by the symbols.
4. The method of claim 3 , wherein the weighting further includes multiplying the determined number of occurrences for said each symbol by a corresponding said bit length.
5. The method of claim 1 , further including grouping together closest neighbors of the files.
6. The method of claim 1 , further including sorting into an ordered list the computed said distances between said each file and every other file.
7. The method of claim 1 , further including normalizing and weighting the determined number of occurrences for said each symbol.
8. The method of claim 7 , wherein the normalizing further includes comparing the determined number of occurrences for said each symbol to a total number of symbols in a source file.
9. The method of claim 7 , wherein the weighting further includes multiplying the determined number of occurrences for said each symbol by a bit length of said original bits of data represented by the symbols.
10. In a computing system environment, a method of differentiating files stored on one or more computing devices, each file having a plurality of symbols derived from an underlying data stream of all original bits of raw data of said each file, comprising encoding said each file as a plurality of symbols representing an underlying data stream of all original bits of binary data of the file, determining a number of occurrences of each said symbol in said each file, and computing a distance in an N-dimensional space between said each file and every other file based on said determined number of occurrences, said N-dimensional space being defined by a total number N of different symbols in the plurality of symbols.
11. The method of claim 10 , wherein the computing the distance further includes determining for like symbols a difference in said determined number of occurrences.
12. The method of claim 11 , further including squaring the determined difference for said like symbols.
13. The method of claim 12 , further including adding together the squared determined differences.
14. The method of claim 13 , further including taking a square root of the added together squared determined differences.
15. In a computing system environment, a method of determining closest files stored on one or more computing devices, each file having a plurality of symbols derived from an underlying data stream of all original bits of raw data of said each file, comprising:
encoding said each file as a plurality of symbols representing an underlying data stream of all original bits of binary data of the file;
determining a number of occurrences of each said symbol in said each file;
computing a distance value between said each file and every other file based on said number of occurrences; and
concluding a closest two files based on the computed distance value.
16. The method of claim 15 , further including normalizing the determined number of occurrences for said each symbol.
17. The method of claim 15 , further including weighting the determined number of occurrences for said each symbol.
18. The method of claim 15 , further including considering only a portion of the determined number of occurrences when said computing the distance.
19. In a computing system environment, a method of differentiating files stored on one or more computing devices, each file having a plurality of symbols derived from an underlying data stream of all original bits of raw data of said each file, comprising:
encoding said each file as a plurality of symbols representing an underlying data stream of all original bits of binary data of the file;
defining an N-dimensional space wherein N is a total number of different symbols in the plurality of symbols;
determining a number of occurrences of each said symbol in said each file; and
computing a distance in the N-dimensional space between said each file and every other file of the stored files based on the determined number of occurrences.