System and method for training machine learning applications
Digital object library management systems and methods for machine learning applications are taught herein. Such a method includes populating a digital object library with a number of machine readable digital objects, modifying the digital objects to include additional machine readable data about the digital objects or other digital objects and the relationships among existing digital objects, generating lists of objects for use in construction and verification of machine learning models used to classify unknown objects into one or more categories, building queries to generate object lists, initiating model generation, in which a machine learning model used to classify unknown objects into one or more categories is generated, initiating model evaluation, storing models, object lists, evaluation results, and associations among these objects, generating a visual display of object metadata, lists, relational information, and evaluation results and running distributable algorithms across the library of digital objects.
1. A method comprising:
generating, for a plurality of files, metadata indicating one or more properties of the plurality of files;
determining, based on the metadata and one or more criteria, a first portion of the plurality of files;
training, based on the first portion of the plurality of files, one or more machine learning models; and
determining, based on the one or more machine learning models, a classification for a file from a second portion of the plurality of files.
2. The method of claim 1 , wherein the generating the metadata comprises determining at least one feature associated with the plurality of files and a similarity metric indicating a number of occurrences of the at least one feature in the plurality of files.
3. The method of claim 2 , wherein the at least one feature comprises at least one of an n-gram, a header field value, image data, or a file length.
4. The method of claim 1 , wherein the one or more machine learning models is based on at least one machine learning algorithm comprising a naive bayes classifier, a decision tree, a random forest, or a neural network.
5. The method of claim 1 , wherein the training the one or more machine learning models is performed on a first cluster of computers and the plurality of files is stored on a second cluster of computers.
6. The method of claim 1 , wherein the generating the metadata is based on receiving a user input comprising at least a portion of the properties.
7. The method of claim 1 , further comprising:
determining, based on the one or more criteria, at least that the first portion are of a same file-type.
8. The method of claim 7 , wherein the second portion of the plurality of files are of the same file-type.
9. The method of claim 1 , wherein the classification comprises at least malign or benign.
10. The method of claim 1 , further comprising:
determining, based on the classification, an effectiveness of the generated one or more machine learning models; and
causing display of data indicating the effectiveness and the metadata.
11. The method of claim 1 , wherein the plurality of files comprises at least one of a video file, an audio file, a document file, or an executable file.
12. The method of claim 1 , wherein the one or more criteria restrict membership in the first portion based on values of the metadata to control training bias in the first portion.
13. A device comprising:
one or more processors; and
memory storing instructions that, when executed by the one or more processors, cause the device to:
generate, for a plurality of files, metadata indicating one or more properties of the plurality of files;
determine, based on the metadata and one or more criteria, a first portion of the plurality of files;
train, based on the first portion of the plurality of files one or more machine learning models; and
determine, based on the one or more machine learning models, a classification for a file from a second portion of the plurality of files.
14. The device of claim 13 , wherein the generating the metadata comprises determining at least one feature associated with the plurality of files and a similarity metric indicating a number of occurrences of the at least one feature in the plurality of files.
15. The device of claim 14 , wherein the at least one feature comprises at least one of an n-gram, a header field value, image data, or a file length.
16. The device of claim 13 , wherein the instructions, when executed by the one or more processors, further cause the device to:
determine, based on the one or more criteria, at least that the first portion are of a same file-type.
17. The device of claim 13 , wherein the classification comprises at least malign or benign.
18. The device of claim 13 , wherein the one or more criteria restrict membership in the first portion based on values of the metadata to control training bias in the first portion.
19. A non-transitory computer-readable storage medium storing computer-readable instructions that, when executed by a processor, cause:
generating, a plurality of files, metadata indicating one or more properties of the plurality of files;
determining, based on the metadata and one or more criteria, a first portion of the plurality of files;
training, based on the first portion of the plurality of files, one or more machine learning models; and
determining, based on the one or more machine learning models, a classification for a file from a second portion of the plurality of files.
20. The non-transitory computer-readable storage medium of claim 19 , wherein the generating the metadata determining at least one feature associated with the plurality of files and a similarity metric indicating a number of occurrences of the at least one feature in the plurality of files.
21. The non-transitory computer-readable storage medium of claim 20 , wherein the at least one feature comprises at least one of an n-gram, a header field value, image data, or a file length.
22. The non-transitory computer-readable storage medium of claim 19 , wherein the classification comprises at least malign or benign.
23. The non-transitory computer-readable storage medium of claim 19 , wherein the one or more criteria restrict membership in the first portion based on values of the metadata to control training bias in the first portion.