IP Library Granted Patent US 9,817,659
Granted Patent B2
US 9,817,659 · App. 15/336,383 · Granted Nov 14, 2017

Artifact normalization

Inventors: Julian Tibble (Oxford, GB); Pavel Avgustinov (Oxford, GB)
Assignee: Semmle Limited
G06F8/75G06F8/71
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 9,817,659
App. No.
15/336,383
Granted
Nov 14, 2017
Kind
B2
Abstract

Methods, systems, and apparatus, including computer programs encoded on computer storage media, for generating normalized analysis artifacts for a source code base. One of the methods includes receiving a request to perform an analysis of a collection of source code files in a particular file system, wherein each source code file in the particular file system has a respective associated original file path. One or more normalized analysis artifacts are generated for the collection of source code files, including applying a file path transformation function to the original file path of each source code file referenced by the analysis artifacts to generate a normalized file path for the source code file, wherein each reference in the normalized analysis artifacts to a particular source code file uses the normalized file path for the particular source code file instead of the original file path for the particular source code file.

Claims (51)

1. A computer-implemented method comprising:

receiving a request to perform an analysis of a collection of source code files, wherein each source code file in the collection of source code files has a respective associated original file path;

determining that a first original file path of a first source code file of the collection of source code files includes a plurality of symbolic links;

generating a resolved file path for the first original file path of the first source code file, including evaluating each symbolic link of the plurality of symbolic links in the first original file path according to one or more resolution criteria, wherein the one or more resolution criteria specify when to resolve a symbolic link in an original file path, including:

determining whether the symbolic link in the first original file path satisfies the one or more resolution criteria,

whenever the symbolic link in the first original file path satisfies any of the one or more resolution criteria, resolving the symbolic link and updating a resolved file path for the first source code file with a resolved path component corresponding to the symbolic link, and

whenever the symbolic link in the first original file path does not satisfy any of the one or more resolution criteria, leaving the symbolic link in the resolved file path for the first source code file,

wherein the resolved file path for the first original file path includes resolved path components for one or more but fewer than all of the plurality of symbolic links in the first original file path;

applying a file path transformation function to the resolved file path for the first source code file to generate a normalized file path for the first source code file; and

generating one or more normalized analysis artifacts for the first source code file, wherein each reference, in the one or more normalized analysis artifacts, to the first source code file, uses the normalized file path for the first source code file instead of the original file path for the first source code file.

2. The method of claim 1 , wherein after evaluating each symbolic link of the plurality of symbolic links in the first original file path, the resolved file path for the first source code file includes at least one of the plurality of symbolic links.

3. The method of claim 1 , wherein evaluating each symbolic link of the plurality of symbolic links in the first original file path comprises evaluating the symbolic links in order from a filename component of the original file path to each parent component of the filename component of the original file path.

4. The method of claim 1 , wherein determining whether the symbolic link satisfies the one or more resolution criteria comprises determining that one or more of the resolution criteria are satisfied because a particular symbolic link of the first original file path does not identify a file having a computer-generated name in a content-addressable file system.

5. The method of claim 1 , wherein determining whether the symbolic link satisfies the one or more resolution criteria comprises determining that one or more of the resolution criteria are satisfied because a particular symbolic link of the first original file path is a symbolic link to a directory and not to a file.

6. The method of claim 1 , wherein determining whether the symbolic link satisfies the one or more resolution criteria comprises determining that one or more of the resolution criteria are satisfied because the symbolic link is a link to a location within a user-specified directory tree.

7. The method of claim 1 , wherein determining whether the symbolic link satisfies the one or more resolution criteria comprises determining that one or more of the resolution criteria are satisfied because a particular symbolic link of the first original file path identifies a file path that matches a portion of a checkout root directory.

8. The method of claim 1 , further comprising computing, from the one or more normalized analysis artifacts, respective values of one or more source code attributes for one or more of the source code files in response to the request.

9. The method of claim 8 , wherein at least two source code files, in the collection of source code files, that have different respective original parent directories according to respective original file paths of the two source code files have a same normalized parent directory according to respective normalized file paths of the two source code files.

10. The method of claim 9 , wherein the two source code files have different original file paths and a same normalized file path, and

wherein computing, from the one or more normalized analysis artifacts, respective values of one or more source code attributes for one or more of the source code files comprises computing a single respective value of each source code attribute for the normalized file path of the two source code files.

11. A system comprising:

one or more computers and one or more storage devices storing instructions that are operable, when executed by the one or more computers, to cause the one or more computers to perform operations comprising:

receiving a request to perform an analysis of a collection of source code files, wherein each source code file in the collection of source code files has a respective associated original file path;

determining that a first original file path of a first source code file of the collection of source code files includes a plurality of symbolic links;

generating a resolved file path for the first original file path of the first source code file, including evaluating each symbolic link of the plurality of symbolic links in the first original file path according to one or more resolution criteria, wherein the one or more resolution criteria specify when to resolve a symbolic link in an original file path, including:

determining whether the symbolic link in the first original file path satisfies the one or more resolution criteria,

whenever the symbolic link in the first original file path satisfies any of the one or more resolution criteria, resolving the symbolic link and updating a resolved file path for the first source code file with a resolved path component corresponding to the symbolic link, and

whenever the symbolic link in the first original file path does not satisfy any of the one or more resolution criteria, leaving the symbolic link in the resolved file path for the first source code file,

wherein the resolved file path for the first original file path includes resolved path components for one or more but fewer than all of the plurality of symbolic links in the first original file path;

applying a file path transformation function to the resolved file path for the first source code file to generate a normalized file path for the first source code file; and

generating one or more normalized analysis artifacts for the first source code file, wherein each reference, in the one or more normalized analysis artifacts, to the first source code file, uses the normalized file path for the first source code file instead of the original file path for the first source code file.

12. The system of claim 11 , wherein after evaluating each symbolic link of the plurality of symbolic links in the first original file path, the resolved file path for the first source code file includes at least one of the plurality of symbolic links.

13. The system of claim 11 , wherein evaluating each symbolic link of the plurality of symbolic links in the first original file path comprises evaluating the symbolic links in order from a filename component of the original file path to each parent component of the filename component of the original file path.

14. The system of claim 11 , wherein determining whether the symbolic link satisfies the one or more resolution criteria comprises determining that one or more of the resolution criteria are satisfied because a particular symbolic link of the first original file path does not identify a file having a computer-generated name in a content-addressable file system.

15. The system of claim 11 , wherein determining whether the symbolic link satisfies the one or more resolution criteria comprises determining that one or more of the resolution criteria are satisfied because a particular symbolic link of the first original file path is a symbolic link to a directory and not to a file.

16. The system of claim 11 , wherein determining whether the symbolic link satisfies the one or more resolution criteria comprises determining that one or more of the resolution criteria are satisfied because the symbolic link is a link to a location within a user-specified directory tree.

17. The system of claim 11 , wherein determining whether the symbolic link satisfies the one or more resolution criteria comprises determining that one or more of the resolution criteria are satisfied because a particular symbolic link of the first original file path identifies a file path that matches a portion of a checkout root directory.

18. The system of claim 11 , wherein the operations further comprise computing, from the one or more normalized analysis artifacts, respective values of one or more source code attributes for one or more of the source code files in response to the request.

19. The system of claim 11 , wherein at least two source code files, in the collection of source code files, that have different respective original parent directories according to respective original file paths of the two source code files have a same normalized parent directory according to respective normalized file paths of the two source code files.

20. The system of claim 19 , wherein the two source code files have different original file paths and a same normalized file path, and

wherein computing, from the one or more normalized analysis artifacts, respective values of one or more source code attributes for one or more of the source code files comprises computing a single respective value of each source code attribute for the normalized file path of the two source code files.

21. A computer program product, encoded on one or more non-transitory computer storage media, comprising instructions that when executed by one or more computers cause the one or more computers to perform operations comprising:

receiving a request to perform an analysis of a collection of source code files, wherein each source code file in the collection of source code files has a respective associated original file path;

determining that a first original file path of a first source code file of the collection of source code files includes a plurality of symbolic links;

generating a resolved file path for the first original file path of the first source code file, including evaluating each symbolic link of the plurality of symbolic links in the first original file path according to one or more resolution criteria, wherein the one or more resolution criteria specify when to resolve a symbolic link in an original file path, including:

determining whether the symbolic link in the first original file path satisfies the one or more resolution criteria,

whenever the symbolic link in the first original file path satisfies any of the one or more resolution criteria, resolving the symbolic link and updating a resolved file path for the first source code file with a resolved path component corresponding to the symbolic link, and

whenever the symbolic link in the first original file path does not satisfy any of the one or more resolution criteria, leaving the symbolic link in the resolved file path for the first source code file,

wherein the resolved file path for the first original file path includes resolved path components for one or more but fewer than all of the plurality of symbolic links in the first original file path;

applying a file path transformation function to the resolved file path for the first source code file to generate a normalized file path for the first source code file; and

generating one or more normalized analysis artifacts for the first source code file, wherein each reference, in the one or more normalized analysis artifacts, to the first source code file, uses the normalized file path for the first source code file instead of the original file path for the first source code file.

Assignments (3)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jan 27, 2020
From: GITHUB SOFTWARE UK LTD.
To: MICROSOFT TECHNOLOGY LICENSING, LLC
Reel/Frame 051710/0252 →
CHANGE OF NAME Recorded Dec 11, 2019
From: SEMMLE LTD.
To: GITHUB SOFTWARE UK LTD.
Reel/Frame 052027/0912 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Feb 1, 2017
From: TIBBLE, JULIAN; AVGUSTINOV, PAVEL
To: SEMMLE LIMITED
Reel/Frame 041589/0739 →
Continuity (2)
Continuation 14928776 · Oct 30, 2015
Related Publication 20170123791A1 · May 4, 2017