Compile-time native code software bill of material generator
Presented is a method for capturing the software bill of material (SBOM) for compiled-code languages (like C or C++), in the absence of a package manager (such as Conan). The method loads a library into the process space of all tools used in the build process (e.g. gcc, Id, as, etc), which allows it to observe all of the files opened by the build tool during the build process. By combining a process identification tree, PID tree, with the record of which files were opened or created by which build tools, the system is able to identify all of the files utilized in the build and create a tree of which files depend on which other files.
1 . A method for generating a software bill-of-material on a developers build system during the build process of a binary file, the software bill-of-material including unique identifiers for all files accessed during the build process and their dependencies, the method comprising:
assigning a library to preload as part of the build process when a build application is loaded into memory on the developers build system, said library comprising an initialize function, an exit function, and an open-detect function,
said build application comprising a series of code blocks which are executed as sequential events including an application construction, an application objective, and an application deconstruction, and
said developers build system having an operating system which includes an open-file function;
registering the initialize function to be executed during the application construction,
the exit function to be executed during the application deconstruction,
invoking the initialize function during the application construction, thereby causing the open-detect function to wrap the open-file function of the operating systems such that the open-detect function is executed with any subsequent open-file call performed on the developers build system, and creating a database and storing metadata relating to the build application;
exiting the application construction code block and proceeding to the application objective, wherein the application objective includes code to open a file;
calling the operating system's open-file function to open said file;
intercepting the open-file function call by the open-detect function, said open-detect function collecting information relative to the file, and storing said information to the database, and invoking the operating system's open-file function prior to returning to the application objective;
completing the application objective and entering the application deconstruction wherein the application deconstruction invokes the exit function, said exit function closing said database;
generating a software bill-of-material from the stored information from the database.
2 . The method of claim 1 , wherein assigning the library to preload comprises configuring the developers build system, prior to loading the build application into memory, to cause the operating system to load the library into a process space of the build application before any other library is loaded, such that symbols of the library, including the open-detect function, are resolved by the build application in preference to corresponding symbols of any other library.
3 . The method of claim 1 , wherein the information collected by the open-detect function comprises one or more of file type, author, publisher, name, version, hash, licenses, copyright, and process identifier (PID), and wherein the database comprises a dictionary data structure in which a path and filename of an accessed file serves as a key and the collected information serves as a value associated with the key.
4 . The method of claim 1 , further comprising, prior to computing a hash of a file being opened, checking the database to determine whether an entry corresponding to the path and filename of the file already exists, and, if the entry exists, omitting recomputation of the hash for that file.
5 . The method of claim 1 , further comprising collecting a process identifier (PID) and a parent process identifier (parent PID) for the build application, and mapping a dependency tree between files based at least in part on the PID and parent PID, wherein the build application invokes one or more child processes including at least one of a preprocessor, an assembler, a link preprocessor, and a linker, and wherein the library is inherited by each child process via environment variable inheritance such that the open-detect function intercepts open-file calls performed by each child process.
6 . The method of claim 1 , further comprising filtering the information collected by the open-detect function based on at least one of: whether a target file being opened is associated with a host tool of a build environment, such that information for host tools is excluded from the database; and a global variable indicating that the build application is associated with a compile-time security tool, such that a subset of data is collected when the global variable is set.
7 . The method of claim 1 , further comprising inspecting one or more command-line arguments passed to the build application when the build application is invoked, deriving additional information about the build process from said one or more command-line arguments, and storing the additional information in the database in association with the metadata relating to the build application.
8 . A system for generating a software bill-of-material on a developers build system during the build process of a binary file, the software bill-of-material including unique identifiers for all files accessed during the build process and their dependencies, the system comprising:
the developers build system having an operating system which includes an open-file function;
a build application loaded into memory on the developers build system, said build application comprising a series of code blocks which are executed as sequential events including an application construction, an application objective, and an application deconstruction; and
a library assigned to be preloaded as part of the build process when the build application is loaded into memory on the developers build system, said library comprising an initialize function, an exit function, and an open-detect function, wherein the initialize function is registered to be executed during the application construction and the exit function is registered to be executed during the application deconstruction;
wherein the initialize function, when invoked during the application construction, is configured to cause the open-detect function to wrap the open-file function of the operating system such that the open-detect function is executed with any subsequent open-file call performed on the developers build system, and to create a database and store metadata relating to the build application;
wherein, upon exiting the application construction code block and proceeding to the application objective, the application objective includes code to open a file and calls the operating system's open-file function to open said file;
wherein the open-detect function is configured to intercept the open-file function call, to collect information relative to the file, to store said information to the database, and to invoke the operating system's open-file function prior to returning to the application objective;
wherein, upon completing the application objective and entering the application deconstruction, the application deconstruction invokes the exit function, said exit function closing said database; and
wherein a software bill-of-material is generated from the stored information from the database.
9 . The system of claim 8 , wherein the developers build system is configured, prior to loading the build application into memory, to cause the operating system to load the library into a process space of the build application before any other library is loaded, such that symbols of the library, including the open-detect function, are resolved by the build application in preference to corresponding symbols of any other library.
10 . The system of claim 8 , wherein the information collected by the open-detect function comprises one or more of file type, author, publisher, name, version, hash, licenses, copyright, and process identifier (PID), and wherein the database comprises a dictionary data structure in which a path and filename of an accessed file serves as a key and the collected information serves as a value associated with the key.
11 . The system of claim 8 , wherein the open-detect function is further configured, prior to computing a hash of a file being opened, to check the database to determine whether an entry corresponding to the path and filename of the file already exists, and, if the entry exists, to omit recomputation of the hash for that file.
12 . The system of claim 8 , wherein the open-detect function is further configured to collect a process identifier (PID) and a parent process identifier (parent PID) for the build application, and to map a dependency tree between files based at least in part on the PID and parent PID, wherein the build application invokes one or more child processes including at least one of a preprocessor, an assembler, a link preprocessor, and a linker, and wherein the library is inherited by each child process via environment variable inheritance such that the open-detect function intercepts open-file calls performed by each child process.
13 . The system of claim 8 , wherein the open-detect function is further configured to filter the information collected based on at least one of: whether a target file being opened is associated with a host tool of a build environment, such that information for host tools is excluded from the database; and a global variable indicating that the build application is associated with a compile-time security tool, such that a subset of data is collected when the global variable is set.
14 . The system of claim 8 , wherein the library is further configured to inspect one or more command-line arguments passed to the build application when the build application is invoked, to derive additional information about the build process from said one or more command-line arguments, and to store the additional information in the database in association with the metadata relating to the build application.
15 . A non-transitory computer-readable medium storing instructions for generating a software bill-of-material on a developers build system during the build process of a binary file, the software bill-of-material including unique identifiers for all files accessed during the build process and their dependencies, the instructions, when executed by one or more processors of the developers build system, cause the processor to:
assign a library to preload as part of the build process when a build application is loaded into memory on the developers build system,
said library comprising an initialize function, an exit function, and an open-detect function,
said build application comprising a series of code blocks which are executed as sequential events including an application construction, an application objective, and an application deconstruction, and
said developers build system having an operating system which includes an open-file function;
register the initialize function to be executed during the application construction, the exit function to be executed during the application deconstruction;
invoke the initialize function during the application construction, thereby causing the open-detect function to wrap the open-file function of the operating system such that the open-detect function is executed with any subsequent open-file call performed on the developers build system, and create a database and store metadata relating to the build application;
exit the application construction code block and proceed to the application objective, wherein the application objective includes code to open a file;
call the operating system's open-file function to open said file;
intercept the open-file function call by the open-detect function, said open-detect function collecting information relative to the file, and store said information to the database, and invoke the operating system's open-file function prior to returning to the application objective;
complete the application objective and enter the application deconstruction wherein the application deconstruction invokes the exit function, said exit function closing said database; and
generate a software bill-of-material from the stored information from the database.
16 . The non-transitory computer-readable medium of claim 15 , wherein assigning the library to preload comprises configuring the developers build system, prior to loading the build application into memory, to cause the operating system to load the library into a process space of the build application before any other library is loaded, such that symbols of the library, including the open-detect function, are resolved by the build application in preference to corresponding symbols of any other library.
17 . The non-transitory computer-readable medium of claim 15 , wherein the instructions further cause the processor to: prior to computing a hash of a file being opened, check the database to determine whether an entry corresponding to the path and filename of the file already exists, and, if the entry exists, omit recomputation of the hash for that file.
18 . The non-transitory computer-readable medium of claim 15 , wherein the instructions further cause the processor to: collect a process identifier (PID) and a parent process identifier (parent PID) for the build application and map a dependency tree between files based at least in part on the PID and parent PID, wherein the build application invokes one or more child processes including at least one of a preprocessor, an assembler, a link preprocessor, and a linker, and wherein the library is inherited by each child process via environment variable inheritance such that the open-detect function intercepts open-file calls performed by each child process.
19 . The non-transitory computer-readable medium of claim 15 , wherein the instructions further cause the processor to: filter the information collected by the open-detect function based on at least one of: whether a target file being opened is associated with a host tool of a build environment, such that information for host tools is excluded from the database; and a global variable indicating that the build application is associated with a compile-time security tool, such that a subset of data is collected when the global variable is set.
20 . The non-transitory computer-readable medium of claim 15 , wherein the instructions further cause the processor to: inspect one or more command-line arguments passed to the build application when the build application is invoked, derive additional information about the build process from said one or more command-line arguments, and store the additional information in the database in association with the metadata relating to the build application.