Method and apparatus for creating a dataset using file creation dates
A method and apparatus for generating a content detection dataset using file creation dates. The method accesses a database comprising data files. The files are analyzed by a machine learning model to determine file creation dates. The creation dates are used to identify relevant content files. The most relevant files are included into a content detection dataset as content samples. The dataset may be used for training machine learning based content detectors.
1 . A method for generating a content detection dataset comprising:
accessing a database comprising a plurality of data files including malware files and clean files;
accessing content of the data files and metadata associated with the data files;
analyzing, using a machine learning model, the content of the data files and the metadata to determine an estimate of a creation date for each data file, wherein the creation date is an indicia of file relevance;
selecting particular data files of the plurality of data files as content samples for a content detector based upon the creation dates and associated file relevance; and
storing the selected content samples and associated creation dates in the content detection dataset which is configured to train a machine learning model of the content detector to identify malware.
2 . The method of claim 1 , wherein the metadata comprises at least one of a creation date or indicia of a creation date.
3 . The method of claim 2 , wherein the indicia comprise at least one of references to specific dates in the data files, references to operating systems or application versions that were available after particular dates, header information, or compiler information.
4 . The method of claim 1 , wherein files having a recent creation date are more relevant than an older creation date.
5 . The method of claim 4 , wherein the malware files having the most recent creation dates are selected as content samples.
6 . The method of claim 1 , wherein analyzing the metadata and file content comprises processing the metadata using a machine learning model.
7 . Apparatus for generating a content detection dataset comprising at least one processor coupled to at least one non-transitory computer readable medium having instructions stored thereon, which, when executed by the at least one processor, cause the at least one processor to perform operations comprising:
accessing a database comprising a plurality of data files including malware files and clean files;
accessing content of the data files and metadata associated with the data files;
analyzing, using a machine learning model, the content of the data files and the metadata to determine an estimate of a creation date for each data file, wherein the creation date is an indicia of file relevance;
selecting particular data files of the plurality of data files as content samples for a content detector based upon the creation dates and associated file relevance; and
storing the selected content samples and associated creation dates in the content detection dataset which is configured to train a machine learning model of the content detector to identify malware.
8 . The apparatus of claim 7 , wherein the metadata comprises at least one of a creation date or indicia of a creation date.
9 . The apparatus of claim 8 , wherein the indicia comprise at least one of references to specific dates in the data files, references to operating systems or application versions that were available after particular dates, header information, or compiler information.
10 . The apparatus of claim 7 , wherein the malware files having a recent creation date are more relevant than an older creation date.
11 . The apparatus of claim 10 , wherein the malware files having the most recent creation dates are selected as content samples.
12 . The apparatus of claim 7 , wherein analyzing the metadata comprises processing the metadata using a machine learning model.