Multi-modal models for detecting malicious emails
In some aspects, the techniques described herein relate to a method for detecting malicious emails, the method including: receiving an email, wherein the email is associated with a markup payload; determining, based on the markup payload, text data associated with the email; determining, using the text data and a first machine learning model, a first representation of the email representing text associated with the email; rendering the email to generate image data that represents a rendering of the email; determining, using the image data and a second machine learning model, a second representation of the email that represents at least the rendering of the email; and determining a prediction for the email based on the first representation and the second representation, wherein the prediction represents whether the email is predicted to be malicious based on the first representation and the second representation.
1. A method for detecting malicious emails, the method comprising:
receiving an email, wherein the email is associated with a markup payload:
determining, based on the markup payload, text data associated with the email;
determining, using the text data and a first machine learning model, a first representation of the email representing text associated with the email;
rendering the email to generate image data that represents a rendering of the email;
determining, using the image data and a second machine learning model, a second representation of the email that represents at least the rendering of the email; and
determining a prediction for the email based on the first representation and the second representation, wherein the prediction represents whether the email is predicted to be malicious based on the first representation and the second representation.
2. The method of claim 1 , wherein determining the prediction comprises:
providing the first representation and the second representation to a third machine learning model, wherein the third machine learning model is configured to determine the prediction based on the first representation and the second representation; and
receiving the prediction from the third machine learning model.
3. The method of claim 1 , further comprising:
based at least in part on determining that the prediction indicates that the email is malicious, determining a remedial action for the email; and
processing the email based on the remedial action.
4. The method of claim 1 , wherein the first machine learning model comprises an attention-based text encoder layer.
5. The method of claim 1 , wherein the second machine learning model comprises a convolutional neural network layer.
6. The method of claim 1 , wherein:
the method further comprises:
determining second image data based on an attached image file associated with the email,
providing the second image data to the second machine learning model, and
receiving, from the second machine learning model, a third representation of the email; and
the prediction is determined based on the first representation, the second representation, and the third representation.
7. The method of claim 1 , wherein:
the method further comprises:
determining third image data based on an image associated with a uniform resource locator (URL) included in the email,
providing the third image data to the second machine learning model, and
receiving, from the second machine learning model, a fourth representation of the email; and
the prediction is determined based on the first representation, the second representation, and the fourth representation.
8. The method of claim 1 , wherein:
the method further comprises:
determining second text data based on content data associated with a uniform resource locator (URL) included in the email,
providing the second text data to the first machine learning model, and
receiving, from the second machine learning model, a fourth representation of the email; and
the prediction is determined based on the first representation, the second representation, and the fourth representation.
9. A system comprising:
one or more processors; and
one or more non-transitory computer-readable media storing computer-executable instructions that, when executed by the one or more processors, cause the one or more processors to perform operations comprising:
receiving an email, wherein the email is associated with a markup payload:
determining, based on the markup payload, text data associated with the email:
determining, using the text data and a first machine learning model, a first representation of the email representing text associated with the email;
rendering the email to generate image data that represents a rendering of the email;
determining, using the image data and a second machine learning model, a second representation of the email that represents at least the rendering of the email; and
determining a prediction for the email based on the first representation and the second representation, wherein the prediction represents whether the email is predicted to be malicious based on the first representation and the second representation.
10. The system of claim 9 , wherein determining the prediction comprises:
providing the first representation and the second representation to a third machine learning model, wherein the third machine learning model is configured to determine the prediction based on the first representation and the second representation; and
receiving the prediction from the third machine learning model.
11. The system of claim 9 , the operations further comprising:
based at least in part on determining that the prediction indicates that the email is malicious, determining a remedial action for the email; and
processing the email based on the remedial action.
12. The system of claim 9 , wherein the first machine learning model comprises an attention-based text encoder layer.
13. The system of claim 9 , wherein the second machine learning model comprises a convolutional neural network layer.
14. The system of claim 9 , wherein:
the operations further comprise:
determining second image data based on an attached image file associated with the email,
providing the second image data to the second machine learning model, and
receiving, from the second machine learning model, a third representation of the email; and
the prediction is determined based on the first representation, the second representation, and the third representation.
15. The system of claim 9 , wherein:
the operations further comprise:
determining third image data based on an image associated with a uniform resource locator (URL) included in the email,
providing the third image data to the second machine learning model, and
receiving, from the second machine learning model, a fourth representation of the email; and
the prediction is determined based on the first representation, the second representation, and the fourth representation.
16. The system of claim 9 , wherein:
the operations further comprise:
determining second text data based on content data associated with a uniform resource locator (URL) included in the email,
providing the second text data to the first machine learning model, and
receiving, from the second machine learning model, a fourth representation of the email; and
the prediction is determined based on the first representation, the second representation, and the fourth representation.
17. One or more non-transitory computer-readable media storing computer-executable instructions that, when executed by one or more processors, cause the one or more processors to perform operations comprising:
receiving an email, wherein the email is associated with a markup payload:
determining, based on the markup payload, text data associated with the email;
determining, using the text data and a first machine learning model, a first representation of the email representing text associated with the email;
rendering the email to generate image data that represents a rendering of the email;
determining, using the image data and a second machine learning model, a second representation of the email that represents at least the rendering of the email; and
determining a prediction for the email based on the first representation and the second representation, wherein the prediction represents whether the email is predicted to be malicious based on the first representation and the second representation.
18. The one or more non-transitory computer-readable media of claim 17 , wherein determining the prediction comprises:
providing the first representation and the second representation to a third machine learning model, wherein the third machine learning model is configured to determine the prediction based on the first representation and the second representation; and
receiving the prediction from the third machine learning model.
19. The one or more non-transitory computer-readable media of claim 17 , further comprising:
based at least in part on determining that the prediction indicates that the email is malicious, determining a remedial action for the email; and
processing the email based on the remedial action.
20. The one or more non-transitory computer-readable media of claim 17 , wherein the first machine learning model comprises an attention-based text encoder layer.