Known-deployed file metadata repository and analysis engine
A known-deployed file metadata repository (KDFMR) and analysis engine enumerates reference lists of files stored on a software delivery point (SDP) and compares the enumerated list of files and associated metadata to previously stored values in the KDFMR. If newly stored or modified files are identified, the analysis engine acquires the files from the SDP. Each file is analyzed to determine whether the file is an atomic file or a container file and metadata is generated or extracted. Each file stored in a container file is recursively extracted and analyzed, where metadata is generated for each extracted file and each container file. The KDFMR periodically analyzes the files stored on the SDP for differences to maintain the currency of the KDFMR data with respect to files stored on the SDP. Storage or modification of files on the SDP triggers analysis of the associated file. KDFMR data is updated with metadata determined based on sandbox detonation of files and/or identified artifacts of known-deployed files.
1. A method comprising:
enumerating, by a known deployed file metadata analysis engine, available files stored on a software distribution point (SDP) computing system;
comparing, by the known deployed file metadata analysis engine, enumerated files to logical paths associated with the SDP computing system to identify one or more new files;
retrieving, by the known deployed file metadata analysis engine from the SDP computing system via a network, the one or more new files;
extracting, by the known deployed file metadata analysis engine, metadata from each of the one or more new files;
recursively extracting, based on an indication that a file of the one or more new files is a container file and by the known deployed file metadata analysis engine, metadata from each file stored in the container file of the one or more new files;
identifying, by the known deployed file metadata analysis engine, a match of metadata of a file of the one or more new files and metadata stored in a data store comprising information stored of “known-good” files and that comprises an indication of one or more of an associated internal development group and a trusted vendor providing “safe” applications;
enriching, by the known deployed file metadata analysis engine, the matched metadata with an indication that the file is “known-deployed” to positively confirm that events associated with the one or more new files are related to deployment activity, wherein the indication comprises artifact metadata only available during dynamic execution of a particular entry of an artifact by the known deployed file metadata analysis engine, and wherein the artifact metadata comprises an indication of the SDP and an indication that the file was introduced to a host through methods associated with approved software distribution practices and wherein artifact file metadata is enriched with labels to identify a functionality associated with the artifact; and
triggering, by the known deployed file metadata analysis engine and based on enrichment of the matched metadata, deployment of one or more files by the SDP, wherein the one or more files are associated with metadata indicating the file is “known-deployed”.
2. The method of claim 1 , wherein recursively extracting metadata from each file stored in the container file of the one or more new files comprises:
calculating a first cryptographic hash of a topmost container of the container file and a second cryptographic hash of an immediate second container adjacent the topmost container;
halting, by the known deployed file metadata analysis engine, recursive file extraction and metadata generation for the container file based on an indication of a match between the first cryptographic hash or the second cryptographic hash to metadata stored in the data store.
3. The method of claim 1 , comprising scheduling, by the known deployed file metadata analysis engine, analysis of the files stored on the SDP on a periodic basis.
4. The method of claim 1 , comprising triggering, by the known deployed file metadata analysis engine, analysis of the files stored on the SDP based on an indication that a modified file has been saved.
5. The method of claim 1 , comprising triggering, by the known deployed file metadata analysis engine, analysis of the files stored on the SDP based on an indication that a new file has been saved.
6. The method of claim 5 , comprising triggering analysis of the new file based on the indication that the new file has been saved.
7. The method of claim 1 , comprising enriching the file metadata stored in the data store with semantic labels to identify whether a file is known to be used for adversary purposes.
8. The method of claim 1 comprising enriching the file metadata stored in the data store with semantic labels to identify whether a file serves a specific purpose within an enterprise computing environment.
9. An apparatus comprising:
a processor; and
non-transitory memory storing instructions that, when executed by the processor, causes the apparatus to:
enumerate available files stored on a software distribution point (SDP) computing system;
compare enumerated files to logical paths associated with the SDP computing system to identify one or more new files;
retrieve, from the SDP computing system via a network, the one or more new files;
extract metadata from each of the one or more new files;
recursively extract, based on an indication that a file of the one or more new files is a container file, metadata from each file stored in the container file of the one or more new files;
identify a match of metadata of a file of the one or more new files and metadata stored in a data store comprising information stored of “known-good” files;
update the matched metadata with an indication that the file is “known-deployed” to positively confirm that events associated with the one or more new files are related to deployment activity, wherein the indication comprises artifact metadata only available during dynamic execution of a particular entry of an artifact by a known deployed file metadata analysis engine, and wherein the artifact metadata comprises an indication of a software distribution point and an indication that the file was introduced to a host through methods associated with approved software distribution practices and wherein artifact file metadata is enriched with labels to identify a functionality associated with the artifact; and
trigger, based on enrichment of the matched metadata, deployment of one or more files by the SDP, wherein the one or more files are associated with metadata indicating the file has a “known-deployed” identifier.
10. The apparatus of claim 9 , wherein the instructions further cause the apparatus to:
calculate a first cryptographic hash of a topmost container of the container file and a second cryptographic hash of an immediate second container adjacent the topmost container; and
halt recursive file extraction and metadata generation for the container file based on an indication of a match between the first cryptographic hash or the second cryptographic hash to metadata stored in the data store.
11. The apparatus of claim 9 , wherein the instructions further cause the apparatus to schedule analysis of the files stored on the SDP on a periodic basis.
12. The apparatus of claim 9 , wherein the instructions further cause the apparatus to trigger analysis of the files stored on the SDP based on an indication that a modified file has been saved.
13. The apparatus of claim 9 , wherein the instructions further cause the apparatus to trigger analysis of the files stored on the SDP based on an indication that a new file has been saved.
14. The apparatus of claim 13 , wherein the instructions further cause the apparatus to trigger analysis of the new file based on the indication that the new file has been saved.
15. The apparatus of claim 9 , wherein the instructions further cause the apparatus to enrich the file metadata stored in the data store with semantic labels to identify whether a file is known to be used for adversary purposes.
16. The apparatus of claim 9 , wherein the instructions further cause the apparatus to enrich the file metadata stored in the data store with semantic labels to identify whether a file serves a specific purpose within an enterprise computing environment.
17. Non-transitory computer-readable media storing instructions that, when executed by a computing device comprising at least one processor, memory, and a communication interface, cause the computing device to:
enumerate available files stored on a software distribution point (SDP) computing system;
compare enumerated files to logical paths associated with the SDP computing system to identify one or more new files;
retrieve, from the SDP computing system via a network, the one or more new files;
extract metadata from each of the one or more new files;
recursively extract, based on an indication that a file of the one or more new files is a container file, metadata from each file stored in the container file of the one or more new files;
identify a match of metadata of a file of the one or more new files and metadata stored in a data store comprising information stored of “known-good” files;
update the matched metadata with an indication that the file is “known-deployed” to positively confirm that events associated with the one or more new files are related to deployment activity, wherein the indication comprises artifact metadata only available during dynamic execution of a particular entry of an artifact by a known deployed file metadata analysis engine and wherein the artifact metadata comprises an indication of a software distribution point and an indication that the file was introduced to a host through methods associated with approved software distribution practices and wherein artifact file metadata is enriched with labels to identify a functionality associated with the artifact; and
trigger, based on the update to the matched metadata, deployment of one or more files by the SDP, wherein the one or more files are associated with metadata indicating the file has a “known-deployed” identifier.
18. The non-transitory computer readable media of claim 17 , wherein the instructions further cause the computing device to:
calculate a first cryptographic hash of a topmost container of the container file and a second cryptographic hash of an immediate second container adjacent the topmost container; and
halt recursive file extraction and metadata generation for the container file based on an indication of a match between the first cryptographic hash or the second cryptographic hash to metadata stored in the data store.
19. The non-transitory computer readable media of claim 17 , wherein the instructions further cause the computing device to schedule analysis of the files stored on the SDP on a periodic basis.
20. The non-transitory computer readable media of claim 17 , wherein the instructions further cause the computing device to trigger analysis of the new file based on the indication that the new file has been saved.