Method and apparatus for clone search
There is provided a method and apparatus for identifying malicious code. The method uses machine learning to compare a new code segment to known code segments of malicious code. Code segments are converted to vectors and the cosine similarity of two vectors is used to identify clones. Techniques to train a neural network for handling very long code sequences and obfuscated malicious code are used.
1 . A method at a computing device for identifying malicious software, comprising:
a) encoding a first software file using an encoder model to create a first encoded vector;
b) selecting a malware file from a database of malware files;
c) encoding the malware file using the encoder model to create a second encoded vector;
d) computing a cosine similarity between the first encoded vector and the second encoded vector; and
e) determining whether the first software file is a clone of the malware file based on the cosine similarity;
wherein the encoder model is trained with training data, the training data comprising a plurality of destination files, a plurality of seed files, and indications of which of the destination files are clones of which of the plurality of seed files.
2 . The method of claim 1 , further comprising: repeating steps b), c), d), and e), with each file from the database.
3 . The method of claim 2 , further comprising: when none of the malware files from the database are determined to be the clone of the first software file, tagging the first software file as safe.
4 . The method of claim 2 , further comprising: when at least one of the malware files from the database is determined to be the clone of the first software file, tagging the first software file as unsafe.
5 . The method of claim 1 , wherein the cosine similarity is computed based on
cos_sim
(
v
→
seed
,
v
→
dest
)
=
v
→
seed
·
v
→
dest
v
→
seed
·
v
→
dest
where cos_sim represents the cosine similarity, {right arrow over (v)} dest represents the first encoded vector, and {right arrow over (v)} seed represents the second encoded vector.
6 . The method of claim 1 , further comprising, prior to step a), abstracting the first software file using an abstraction model.
7 . The method of claim 6 , wherein the abstraction model is trained with training data, the training data comprising a plurality of destination files, a plurality of seed files, and indications of which of the destination files are clones of which of the plurality of seed files.
8 . The method of claim 1 , wherein step e) compares the cosine similarity to a threshold value.
9 . A computing device for identifying malicious software, comprising:
a processor; and
memory;
wherein the processor and the memory cooperate to:
a) encode a first software file using an encoder model to create a first encoded vector;
b) select a malware file from a database of malware files;
c) encode the malware file using the encoder model to create a second encoded vector;
d) compute a cosine similarity between the first encoded vector and the second encoded vector; and
e) determine whether the first software file is a clone of the malware file based on the cosine similarity;
wherein the encoder model is trained with training data, the training data comprising a plurality of destination files, a plurality of seed files, and indications of which of the destination files are clones of which of the plurality of seed files.
10 . The computing device of claim 9 , wherein the processor and the memory further cooperate to repeat steps b), c), d), and e), with each file from the database.
11 . The computing device of claim 10 , wherein the processor and the memory further cooperate to:
when none of the malware files from the database are determined to be the clone of the first software file, tag the first software file as safe.
12 . The computing device of claim 10 , wherein the processor and the memory further cooperate to:
when at least one of the malware files from the database is determined to be the clone of the first software file, tag the first software file as unsafe.
13 . The computing device of claim 9 , wherein the cosine similarity is computed based on
cos_sim
(
v
→
seed
,
v
→
dest
)
=
v
→
seed
·
v
→
dest
v
→
seed
·
v
→
dest
where cos_sim represents the cosine similarity, {right arrow over (v)} dest represents the first encoded vector, and {right arrow over (v)} seed represents the second encoded vector.
14 . The computing device of claim 9 , wherein the processor and the memory further cooperate to, prior to step a), abstract he first software file using an abstraction model.
15 . The computing device of claim 14 , wherein the abstraction model is trained with training data, the training data comprising a plurality of destination files, a plurality of seed files, and indications of which of the destination files are clones of which of the plurality of seed files.
16 . The computing device of claim 9 , wherein step e) compares the cosine similarity to a threshold value.
17 . A non-transitory computer readable medium having stored thereon executable code for execution by a processor of a computing device, the executable code comprising instructions for:
a) encoding a first software file using an encoder model to create a first encoded vector;
b) selecting a malware file from a database of malware files;
c) encoding the malware file using the encoder model to create a second encoded vector;
d) computing a cosine similarity between the first encoded vector and the second encoded vector; and
e) determining whether the first software file is a clone of the malware file based on the cosine similarity;
wherein the encoder model is trained with training data, the training data comprising a plurality of destination files, a plurality of seed files, and indications of which of the destination files are clones of which of the plurality of seed files.