Software composition analysis (SCA) to identify AI-generated code that is copied code from third-party libraries

1. A computer-implemented method comprising:

selecting a code file in a development system to create a selected code file;

identifying a programming language included in the selected code file;

selecting a language-specific parser based on the programming language;

segmenting the selected code file, using the language-specific parser, to create multiple parsed blocks comprising:

function definition blocks;

file license blocks; and

at least one remaining block;

selecting a block of the multiple parsed blocks to create a selected block;

creating, using a cryptographic hash function and based on the selected block, a block signature that uniquely identifies the selected block;

comparing the block signature to multiple signatures of blocks in third-party libraries, the multiple signatures located in an index;

based at least in part on determining that the block signature matches one of the multiple signatures of blocks in third-party libraries:

determining, using the index, a source file from which the selected block was copied; and

adding a source file identifier associated with the source file to a software composition report of the selected code file;

identifying, based on the software composition report, one or more vulnerabilities included in the source file to create identified vulnerabilities;

identifying, based on the one or more vulnerabilities, one or more fixes to address the vulnerabilities to create identified fixes; and

providing the software composition report, the identified vulnerabilities, and the identified fixes to a software developer associated with the selected code file to enable the software developer to apply the identified fixes to the identified vulnerabilities.

2. The computer-implemented method of claim 1 , further comprising:

determining, based on the software composition report, one or more licensing incompatibilities associated with the selected code file; and

providing one or more suggestions on addressing the one or more licensing incompatibilities associated with the selected code file.

3. The computer-implemented method of claim 1 , further comprising:

creating, using a machine learning embedding model and based on the selected block, a block embedding comprising a floating-point vector representation of the selected block;

based at least in part on determining that the block signature fails to match any of the multiple signatures of the blocks in the third-party libraries, comparing the block embedding to multiple embeddings of the blocks in the third-party libraries, the multiple embeddings located in the index; and

determining that the block embedding matches, in the index, one or more embeddings of the multiple embeddings of the blocks in the third-party libraries;

determining, using the index and based on the one or more embeddings that match the block embedding, one or more additional source files; and

adding the one or more additional source files to the software composition report of the selected code file.

4. The computer-implemented method of claim 3 , further comprising:

ordering the one or more embeddings that match the block embedding based on a similarity measure to create an ordered set of matching embeddings ordered from closest match to least closest match.

5. The computer-implemented method of claim 4 , wherein the similarity measure comprises a vector cosine distance.

6. The computer-implemented method of claim 1 , further comprising:

determining a size of the at least one remaining block; and

based on determining that the size of the at least one remaining block exceeds a predetermined size, dividing the at least one remaining block into smaller remaining blocks.

7. The computer-implemented method of claim 1 , further comprising:

identifying, based on the software composition report, one or more licensing incompatibilities; and

providing at least one suggestion to address the one or more licensing incompatibilities.

8. A server comprising:

one or more processors; and

one or more non-transitory computer readable media storing instructions executable by the one or more processors to perform operations comprising:

selecting a code file in a development system to create a selected code file;

identifying a programming language included in the selected code file;

selecting a language-specific parser based on the programming language;

segmenting the selected code file, using the language-specific parser, to create multiple parsed blocks comprising:

function definition blocks;

file license blocks; and

at least one remaining block;

selecting a block of the multiple parsed blocks to create a selected block;

creating, using a cryptographic hash function and based on the selected block, a block signature that uniquely identifies the selected block;

comparing the block signature to multiple signatures of blocks in third-party libraries, the multiple signatures located in an index;

based at least in part on determining that the block signature matches one of the multiple signatures of blocks in third-party libraries:

determining, using the index, a source file from which the selected block was copied; and

adding a source file identifier associated with the source file to a software composition report of the selected code file;

identifying, based on the software composition report, one or more vulnerabilities included in the source file to create identified vulnerabilities;

identifying, based on the one or more vulnerabilities, one or more fixes to address the vulnerabilities to create identified fixes; and

providing the software composition report, the identified vulnerabilities, and the identified fixes to a software developer associated with the selected code file to enable the software developer to apply the identified fixes to the identified vulnerabilities.

9. The server of claim 8 , wherein creating the index comprises:

determining a dependency graph for individual third-party software libraries of multiple software libraries to create multiple dependency graphs of the multiple software libraries;

merging the dependency graphs to create a consolidated dependency graph; and

creating, using a ranking algorithm, a prioritized set of libraries ranked in order of importance, the prioritized set of libraries including no more than a predetermined number of libraries.

10. The server of claim 9 , wherein:

the ranking algorithm comprises PageRank.

11. The server of claim 9 , the operations further comprising:

determining, based on the prioritized set of libraries, one or more libraries that do not include copied code;

designating the one or more libraries that do not include copied code as base libraries; and

creating the index using the base libraries.

12. The server of claim 8 , the operations further comprising:

receiving a request to add a new library to the index;

based on determining that a portion of the new library is already included in the index, identifying a particular library of the third-party libraries from which the portion originated; and

adding information to the index indicating that the portion of the new library originated from the particular library.

13. The server of claim 8 , wherein the source file identifier comprises:

a file identifier identifying the source file; and

a version identifier identifying a version of the source file.

14. The server of claim 8 , wherein the at least one remaining block includes data structure declarations and constants from the selected file.

15. One or more non-transitory computer readable media capable of storing instructions executable by one or more processors to perform operations comprising:

selecting a code file in a development system to create a selected code file;

identifying a programming language included in the selected code file;

selecting a language-specific parser based on the programming language;

segmenting the selected code file, using the language-specific parser, to create multiple parsed blocks comprising:

function definition blocks;

file license blocks; and

at least one remaining block;

selecting a block of the multiple parsed blocks to create a selected block;

creating, using a cryptographic hash function and based on the selected block, a block signature that uniquely identifies the selected block;

comparing the block signature to multiple signatures of blocks in third-party libraries, the multiple signatures located in an index;

based at least in part on determining that the block signature matches one of the multiple signatures of blocks in third-party libraries:

determining, using the index, a source file from which the selected block was copied; and

adding a source file identifier associated with the source file to a software composition report of the selected code file;

identifying, based on the software composition report, one or more vulnerabilities included in the source file to create identified vulnerabilities;

identifying, based on the one or more vulnerabilities, one or more fixes to address the vulnerabilities to create identified fixes; and

providing the software composition report, the identified vulnerabilities, and the identified fixes to a software developer associated with the selected code file to enable the software developer to apply the identified fixes to the identified vulnerabilities.

16. The one or more non-transitory computer readable media of claim 15 , the operations further comprising:

determining, based on the software composition report, one or more licensing incompatibilities associated with the selected code file; and

providing one or more suggestions on addressing the one or more licensing incompatibilities associated with the selected code file.

17. The one or more non-transitory computer readable media of claim 16 , the operations further comprising:

creating, using a machine learning embedding model and based on the selected block, a block embedding comprising a floating-point vector representation of the selected block;

based at least in part on determining that the block signature fails to match any of the multiple signatures of the blocks in the third-party libraries, comparing the block embedding to multiple embeddings of the blocks in the third-party libraries, the multiple embeddings located in the index; and

determining that the block embedding matches, in the index, one or more embeddings of the multiple embeddings of the blocks in the third-party libraries;

determining, using the index and based on the one or more embeddings that match the block embedding, one or more additional source files; and

adding the one or more additional source files to the software composition report of the selected code file.

18. The one or more non-transitory computer readable media of claim 17 , the operations further comprising:

ordering the one or more embeddings that match the block embedding based on a similarity measure to create an ordered set of matching embeddings ordered from closest match to least closest match, the similarity measure comprising a vector cosine distance.

19. The one or more non-transitory computer readable media of claim 18 , the operations further comprising:

determining a dependency graph for individual third-party software libraries of multiple software libraries to create multiple dependency graphs of the multiple software libraries;

merging the dependency graphs to create a consolidated dependency graph;

creating, using a ranking algorithm, a prioritized set of libraries ranked in order of importance, the prioritized set of libraries including no more than a predetermined number of libraries;

determining, based on the prioritized set of libraries, one or more libraries that do not include copied code;

designating the one or more libraries that do not include copied code as base libraries; and

creating the index using the base libraries.

20. The one or more non-transitory computer readable media of claim 15 , wherein:

the source file identifier comprises:

a file identifier identifying the source file; and

a version identifier identifying a version of the source file.