Known-deployed file metadata repository and analysis engine
A known-deployed file metadata repository (KDFMR) and analysis engine enumerates reference lists of files stored on a software delivery point (SDP) and compares the enumerated list of files and associated metadata to previously stored values in the KDFMR. If newly stored or modified files are identified, the analysis engine acquires the files from the SDP. Each file is analyzed to determine whether the file is an atomic file or a container file and metadata is generated or extracted. Each file stored in a container file is recursively extracted and analyzed, where metadata is generated for each extracted file and each container file. The KDFMR periodically analyzes the files stored on the SDP for differences to maintain the currency of the KDFMR data with respect to files stored on the SDP. Storage or modification of files on the SDP triggers analysis of the associated file. KDFMR data is updated with metadata determined based on sandbox detonation of files and/or identified artifacts of known-deployed files.
1 . A system comprising:
a software distribution point (SDP) computing system storing files for distribution to computing devices on an enterprise network; and
a computing device comprising:
a processor; and
non-transitory memory storing instructions that, when executed by the processor, causes the computing device to:
enumerate available files stored on the SDP computing system;
retrieve, from the SDP computing system via a network, one or more new files based on a comparison of enumerated files to logical paths associated with the SDP computing system;
extract metadata from each of the one or more new files, wherein the metadata is recursively extracted from two or more stored files in a container file;
identify a match of metadata of a file of the one or more new files and metadata stored in a data store comprising information stored of “known-good” files;
update the matched metadata with an indication that the file is “known- deployed” to positively confirm that events associated with the one or more new files are related to deployment activity by the SDP, wherein the indication comprises artifact metadata only available during dynamic execution of a particular entry of an artifact by a known deployed file metadata analysis engine, and wherein the artifact metadata comprises an indication of the SDP computing system and an indication that the file was introduced to a host through methods associated with approved software distribution practices, and wherein artifact file metadata is enriched with labels to identify a functionality associated with the artifact, and wherein the artifact metadata associated with the SDP only available during dynamic execution of a particular entry of the artifact by the known deployed file metadata analysis engine comprises information that is variable based on a user associated with detonation of the one or more files; and
trigger, based on enrichment of the matched metadata, deployment of one or more files by the SDP, wherein the one or more files are associated with metadata indicating the file has a “known-deployed” identifier.
2 . The system of claim 1 , wherein the instructions further cause the computing device to:
calculate a first cryptographic hash of a topmost container of the container file and a second cryptographic hash of an immediate second container adjacent the topmost container; and
halt recursive file extraction and metadata generation for the container file based on an indication of a match between the first cryptographic hash or the second cryptographic hash to metadata stored in the data store.
3 . The system of claim 1 , wherein the instructions further cause the computing device to schedule analysis of the files stored on the SDP on a periodic basis.
4 . The system of claim 1 , wherein the instructions further cause the computing device to trigger analysis of the files stored on the SDP based on an indication that a modified file has been saved.
5 . The system of claim 1 , wherein the instructions further cause the computing device to trigger analysis of the files stored on the SDP based on an indication that a new file has been saved.
6 . The system of claim 5 , wherein the instructions further cause the computing device to trigger analysis of the new file based on the indication that the new file has been saved.
7 . The system of claim 1 , wherein the instructions further cause the computing device to enrich the file metadata stored in the data store with semantic labels to identify whether a file is known to be used for adversary purposes.
8 . The system of claim 1 , wherein the instructions further cause the computing device to enrich the file metadata stored in the data store with semantic labels to identify whether a file serves a specific purpose within an enterprise computing environment corresponding to one of a release purpose or a development purpose.
9 . A method comprising:
enumerating, by a known deployed file metadata analysis engine, files stored on a software distribution point (SDP) computing system;
retrieving, by the known deployed file metadata analysis engine from the SDP computing system via a network, one or more new files based on a comparison of enumerated files to logical paths associated with the SDP computing system;
extracting, by the known deployed file metadata analysis engine, metadata from each of the one or more new files, wherein the metadata is recursively extracted from two or more stored files in a container file;
identifying, by the known deployed file metadata analysis engine, a match of metadata of the one or more new files and metadata stored in a data store comprising information stored of “known-good” files;
updating, by the known deployed file metadata analysis engine, the matched metadata with an indication that the one or more new files is “known-deployed” to positively confirm deployment activity by the SDP, wherein the indication comprises artifact metadata associated with the SDP only available during dynamic execution of a particular entry of an artifact by the known deployed file metadata analysis engine and methods associated with approved software distribution practices, and wherein artifact file metadata is enriched with labels to identify a functionality associated with the artifact, and wherein the artifact metadata associated with the SDP only available during dynamic execution of a particular entry of the artifact by the known deployed file metadata analysis engine comprises information that is variable based on a host endpoint upon which the artifact is deployed; and
triggering, by the known deployed file metadata analysis engine and based on enrichment of the matched metadata, deployment of one or more files by the SDP.
10 . The method of claim 9 , further comprising recursively extracting metadata from each file stored in a container file of the one or more new files by:
calculating a first cryptographic hash of a topmost container of the container file and a second cryptographic hash of an immediate second container adjacent the topmost container; and
halting, by the known deployed file metadata analysis engine, recursive file extraction and metadata generation for the container file based on an indication of a match between the first cryptographic hash or the second cryptographic hash to metadata stored in the data store.
11 . The method of claim 9 , comprising scheduling, by the known deployed file metadata analysis engine, analysis of the files stored on the SDP on a periodic basis.
12 . The method of claim 9 , comprising triggering, by the known deployed file metadata analysis engine, analysis of the files stored on the SDP based on an indication that a modified file has been saved.
13 . The method of claim 9 , comprising triggering, by the known deployed file metadata analysis engine, analysis of the files stored on the SDP based on an indication that a new file has been saved.
14 . The method of claim 13 , comprising triggering analysis of the new file based on the indication that the new file has been saved.
15 . The method of claim 9 , comprising enriching the file metadata stored in the data store with semantic labels to identify whether a file is known to be used for adversary purposes.
16 . The method of claim 9 , comprising enriching the file metadata stored in the data store with semantic labels to identify whether a file serves a specific purpose within an enterprise computing environment, wherein the specific purpose corresponds to a production network for support of released software products.
17 . An apparatus comprising:
a processor; and
non-transitory memory storing instructions that, when executed by the processor, causes the apparatus to:
enumerate available files stored on a software distribution point (SDP) computing system;
retrieve, from the SDP computing system via a network, one or more new files based on a comparison of enumerated files to logical paths associated with the SDP computing system;
extract metadata from each of the one or more new files, wherein the metadata is recursively extracted from two or more stored files in a container file;
identify a match of metadata of a file of the one or more new files and metadata stored in a data store comprising information stored of “known-good” files;
update the matched metadata with an indication that the file is “known-deployed” to positively confirm that events associated with the one or more new files are related to deployment activity by the SDP, wherein the indication comprises artifact metadata only available during dynamic execution of a particular entry of an artifact by a known deployed file metadata analysis engine, and wherein the artifact metadata comprises an indication of the SDP computing system and an indication that the file was introduced to a host through methods associated with approved software distribution practices, and wherein artifact file metadata is enriched with labels to identify a functionality associated with the artifact, and wherein the artifact metadata associated with the SDP only available during dynamic execution of a particular entry of the artifact by the known deployed file metadata analysis engine comprises information that is variable based on one or both of a host endpoint upon which the artifact is deployed and a user associated with detonation of the one or more files; and
trigger, based on enrichment of the matched metadata, deployment of one or more files by the SDP computing system, wherein the one or more files are associated with metadata indicating the file has a “known-deployed” identifier.
18 . The apparatus of claim 17 , wherein the instructions further cause the apparatus to enrich the file metadata stored in the data store with semantic labels identifying a purpose served by an associated file within an enterprise computing environment and wherein the semantic labels correspond to a particular host endpoint.
19 . The apparatus of claim 17 , wherein the instructions further cause the apparatus to trigger analysis of the new file based on the indication that the new file has been saved.
20 . The apparatus of claim 17 , wherein the instructions further cause the apparatus to trigger analysis of the SDP computing system on a periodic basis.