Efficient change analysis in poly-lingual corpus hierarchies

1. A method comprising:

obtaining, by a data processing system, directory level checksums and rolling checksums for a source set of code in a first data structure that is a hierarchical data structure;

obtaining, by the data processing system, a target set of code in a second data structure that is a hierarchical data structure;

calculating, by the data processing system, directory level checksums for the target set of code, wherein the directory level checksums are calculated for each directory level node of the second data structure;

calculating, by the data processing system, rolling checksums for the target set of code based on the directory level checksums for the target set of code;

comparing, by the data processing system, the directory level checksums and the rolling checksums calculated for the target set of code against the directory level checksums and the rolling checksums calculated for the source set of code;

identifying, by the data processing system, a location of change data of the target set of code at directory level within the second data structure based on the comparison of the directory level checksums and the rolling checksums calculated for the target set of code against the directory level checksums and the rolling checksums obtained for the source set of code; and

outputting, by the data processing system, the location of the change data;

wherein upon calculating the directory level checksums for each directory level node, the directory level checksums are stored and maintained for downstream processing including the comparison of the directory level checksums and the rolling checksums calculated for the target set of code against the directory level checksums and the rolling checksums calculated for the source set of code; and rolling checksums are calculated for each directory level node of the second data structure as a concatenation of the directory level checksum for the directory level node and the directory level checksums for all descendent directory level nodes of the directory level node.

2. The method of claim 1 , wherein the obtaining the directory level checksums and rolling checksums for the source set of code, comprises:

creating directories for the first data structure;

creating source files for the source set of code;

populating the directories with the source files;

calculating the directory level checksums for the source set of code; and

calculating the rolling checksums for the source set of code based on the directory level checksums for the source set of code.

3. The method of claim 1 , further comprising:

creating, by the data processing system, directories for the second data structure;

creating, by the data processing system, target files for the target set of code; and

populating, by the data processing system, the directories with the target files.

4. The method of claim 1 , wherein the hierarchical data structure is in native format, and the comparing comprises a top down approach starting at root level comparison and working down each branch for the directory level checksum and the rolling checksum comparisons.

5. The method of claim 1 , wherein the hierarchical data structure is in reverse format, and the comparing comprises a bottom up approach starting at root level comparison and working up each segment for the directory level checksum and the rolling checksum comparisons.

6. The method of claim 1 , further comprising obtaining, by the data processing system, file level checksums for the source set of code in the first data structure, and calculating, by the data processing system, file level checksums for the target set of code, wherein:

the comparing comprises comparing the file level checksums, the directory level checksums, and the rolling checksums calculated for the target set of code against the file level checksums, the directory level checksums, and the rolling checksums calculated for the source set of code;

the identifying comprises identifying the location of the change data of the target set of code at the directory level and file level within the second data structure based on the comparison of the file level checksums, the directory level checksums, and the rolling checksums calculated for the target set of code against the file level checksums, the directory level checksums, and the rolling checksums calculated for the source set of code; and

outputting, by the data processing system, the location of the change data.

7. A non-transitory computer-readable memory storing a plurality of instructions executable by one or more processors, the plurality of instructions comprising instructions that when executed by the one or more processors cause the one or more processors to perform processing comprising:

obtaining directory level checksums and rolling checksums for a source set of code in a first data structure that is a hierarchical data structure;

obtaining a target set of code in a second data structure that is a hierarchical data structure;

calculating directory level checksums for the target set of code, wherein the directory level checksums are calculated for each directory level node of the second data structure;

calculating rolling checksums for the target set of code based on the directory level checksums for the target set of code;

comparing the directory level checksums and the rolling checksums calculated for the target set of code against the directory level checksums and the rolling checksums calculated for the source set of code;

identifying a location of change data of the target set of code at directory level within the second data structure based on the comparison of the directory level checksums and the rolling checksums calculated for the target set of code against the directory level checksums and the rolling checksums obtained for the source set of code; and

outputting the location of the change data,

wherein upon calculating the directory level checksums for each directory level node, the directory level checksums are stored and maintained for downstream processing including the comparison of the directory level checksums and the rolling checksums calculated for the target set of code against the directory level checksums and the rolling checksums calculated for the source set of code; and rolling checksums are calculated for each directory level node of the second data structure as a concatenation of the directory level checksum for the directory level node and the directory level checksums for all descendent directory level nodes of the directory level node.

8. The non-transitory computer-readable memory of claim 7 , wherein the obtaining the directory level checksums and rolling checksums for the source set of code, comprises:

creating directories for the first data structure;

creating source files for the source set of code;

populating the directories with the source files;

calculating the directory level checksums for the source set of code; and

calculating the rolling checksums for the source set of code based on the directory level checksums for the source set of code.

9. The non-transitory computer-readable memory of claim 7 , wherein the processing further comprises:

creating directories for the second data structure;

creating target files for the target set of code; and

populating the directories with the target files.

10. The non-transitory computer-readable memory of claim 7 , wherein the hierarchical data structure is in native format, and the comparing comprises a top down approach starting at root level comparison and working down each branch for the directory level checksum and the rolling checksum comparisons.

11. The non-transitory computer-readable memory of claim 7 , wherein the hierarchical data structure is in reverse format, and the comparing comprises a bottom up approach starting at root level comparison and working up each segment for the directory level checksum and the rolling checksum comparisons.

12. The non-transitory computer-readable memory of claim 7 , wherein the processing further comprises:

the comparing comprises comparing the file level checksums, the directory level checksums, and the rolling checksums calculated for the target set of code against the file level checksums, the directory level checksums, and the rolling checksums calculated for the source set of code;

the identifying comprises identifying the location of the change data of the target set of code at the directory level and file level within the second data structure based on the comparison of the file level checksums, the directory level checksums, and the rolling checksums calculated for the target set of code against the file level checksums, the directory level checksums, and the rolling checksums calculated for the source set of code; and

outputting the location of the change data.

13. A system comprising:

one or more processors; and

a memory coupled to the one or more processors, the memory storing a plurality of instructions executable by the one or more processors, the plurality of instructions comprising instructions that when executed by the one or more processors cause the one or more processors to perform processing comprising:

obtaining directory level checksums and rolling checksums for a source set of code in a first data structure that is a hierarchical data structure;

obtaining a target set of code in a second data structure that is a hierarchical data structure;

calculating directory level checksums for the target set of code, wherein the directory level checksums are calculated for each directory level node of the second data structure;

calculating rolling checksums for the target set of code based on the directory level checksums for the target set of code;

comparing the directory level checksums and the rolling checksums calculated for the target set of code against the directory level checksums and the rolling checksums obtained for the source set of code;

identifying a location of change data of the target set of code at directory level within the second data structure based on the comparison of the directory level checksums and the rolling checksums calculated for the target set of code against the directory level checksums and the rolling checksums calculated for the source set of code; and

outputting the location of the change data,

wherein upon calculating the directory level checksums for each directory level node, the directory level checksums are stored and maintained for downstream processing including the comparison of the directory level checksums and the rolling checksums calculated for the target set of code against the directory level checksums and the rolling checksums calculated for the source set of code; and rolling checksums are calculated for each directory level node of the second data structure as a concatenation of the directory level checksum for the directory level node and the directory level checksums for all descendent directory level nodes of the directory level node.

14. The system of claim 13 , wherein the obtaining the directory level checksums and rolling checksums for the source set of code, comprises:

creating directories for the first data structure;

creating source files for the source set of code;

populating the directories with the source files;

calculating the directory level checksums for the source set of code; and

calculating the rolling checksums for the source set of code based on the directory level checksums for the source set of code.

15. The system of claim 13 , wherein the hierarchical data structure is in native format, and the comparing comprises a top down approach starting at root level comparison and working down each branch for the directory level checksum and the rolling checksum comparisons.

16. The system of claim 13 , wherein the hierarchical data structure is in reverse format, and the comparing comprises a bottom up approach starting at root level comparison and working up each segment for the directory level checksum and the rolling checksum comparisons.

17. The system of claim 13 , wherein the processing further comprises:

the comparing comprises comparing the file level checksums, the directory level checksums, and the rolling checksums calculated for the target set of code against the file level checksums, the directory level checksums, and the rolling checksums calculated for the source set of code;

the identifying comprises identifying the location of the change data of the target set of code at the directory level and file level within the second data structure based on the comparison of the file level checksums, the directory level checksums, and the rolling checksums calculated for the target set of code against the file level checksums, the directory level checksums, and the rolling checksums calculated for the source set of code; and

outputting the location of the change data.