Identification and deobfuscation of obfuscated text in digital content
An online system receives digital content and determines whether the digital content includes malicious content, such as obfuscated text, before presenting the digital content to a user. To determine whether the digital content contains malicious content, the online system renders the digital content. The online system performs optical character recognition on the content. The online system uses an obfuscation machine learning model to identify obfuscated text. The online system may deobfuscate the obfuscated text. The online system may prevent presentation of the digital content in response to detecting obfuscated text.
1. A method comprising: receiving a digital content item; rendering a portion of the digital content item;
performing optical character recognition on the portion of the digital content item to identify one or more strings of text; generating a series of character embeddings for the identified strings of text of the portion of the digital content item;
inputting series of character embeddings to an obfuscation detection model, wherein the obfuscation detection model is configured as a neural network model, and the obfuscation detection model is trained by storing a text obfuscation library, and training parameters of the obfuscation detection model using the text obfuscation library;
outputting from the obfuscation detection model, a probability of obfuscation for the series of character embeddings; and responsive to determining that the output probability is above a predetermined threshold, providing the digital content item to a policy system for review; and
determining that the digital content item contains an obfuscation.
2. The method of claim 1 , further comprising preventing display of the digital content item in response to determining that the obfuscation violates presentation specifications of the policy system.
3. The method of claim 1 , further comprising assigning an attention weight to each character in the series of character embeddings, each attention weight indicating a contribution of a respective character to the probability of obfuscation.
4. The method of claim 1 , wherein the digital content item comprises an ad request.
5. The method of claim 1 , further comprising responsive to determining that the digital content item includes an obfuscation, deobfuscating the series of character embeddings to generate a deobfuscated version of the identified strings of text.
6. A computer program product comprising a non-transitory computer-readable storage medium containing computer program code for: receiving a digital content item;
rendering a portion of the digital content item; performing optical character recognition on the portion of the digital content item to identify one or more strings of text; generating a series of character embeddings for the identified strings of text of the portion of the digital content item;
inputting series of character embeddings to an obfuscation detection model, wherein the obfuscation detection model is configured as a neural network model, and the obfuscation detection model is trained by storing a text obfuscation library, and training parameters of the obfuscation detection model using the text obfuscation library;
outputting from the obfuscation detection model, a probability of obfuscation for the series of character embeddings; and responsive to determining that the output probability is above a predetermined threshold, providing the digital content item to a policy system for review; and
determining that the digital content item contains an obfuscation;
outputting from the obfuscation detection model, a probability of obfuscation for the series string of character embeddings; and responsive to determining that the output probability is above a predetermined threshold, providing the digital content item to a policy system for review.
7. The computer program product of claim 6 , further containing computer program code for preventing display of the digital content item in response to determining that the obfuscation violates presentation specifications of the policy system.
8. The computer program product of claim 6 , further containing computer program code for assigning an attention weight to each character in the series of character embeddings, each attention weight indicating a contribution of a respective character to the probability of obfuscation.
9. The computer program product of claim 6 , wherein the digital content item comprises an ad request.
10. The computer program product of claim 6 , further containing computer program code for responsive to determining that the digital content item includes an obfuscation, deobfuscating the series of character embeddings to generate a deobfuscated version of the identified strings of text.
11. A method comprising: receiving an ad request; rendering the ad request; performing optical character recognition on the ad request to identify one or more strings of text in the ad request; generating a series of character embeddings for the identified strings of text in the ad request;
inputting the series of character embeddings to an obfuscation detection model, wherein the obfuscation detection model is configured as a neural network model, and the obfuscation detection model is trained by storing a text obfuscation library, and training parameters of the obfuscation detection model using the text obfuscation library;
outputting from the obfuscation detection model, a probability of obfuscation for the series of character embeddings; and
responsive to determining that the output probability is above a predetermined threshold, providing the digital content item to a policy system for review; and
determining that the digital content item contains an obfuscation.
12. The method of claim 11 , further comprising assigning an attention weight to each character in series of character embeddings for the ad request, each attention weight indicating a contribution of a respective character to the probability of obfuscation.
13. The method of claim 12 , wherein the attention weights indicate a likelihood of obfuscation.
14. The method of claim 11 , further comprising using an out-of-vocabulary detection module to detect unknown words in the ad request.
15. The method of claim 11 , further comprising responsive to determining that the digital content item includes an obfuscation, deobfuscating the series of character embeddings to generate a deobfuscated version of the identified strings of text.
16. The method of claim 5 , wherein the deobfuscated version of the identified strings of text is generated by the obfuscation detection model.
17. The computer program product of claim 10 , wherein the deobfuscated version of the identified strings of text is generated by the obfuscation detection model.
18. The method of claim 11 , further comprising responsive to determining that the ad request includes an obfuscation and that the obfuscation violates presentation specifications of the policy system, preventing display of the ad request on one or more client devices.