Application of computer visual classification to security events

1. A method for identifying security events from a set of web requests, the method comprising:

obtaining, for each individual web request of the set of web requests, a screenshot of a corresponding web path resulting from the individual web request;

applying a perceptual hash to each obtained screenshot to generate a set of hashed screenshots;

associating a value of the perceptual hash with the corresponding obtained screenshot;

applying a cryptographic hash to each of the obtained screenshots;

associating a value of the cryptographic hash with the corresponding obtained screenshot;

grouping the set of hashed screenshots into multiple grouped sets of hashed screenshots based on corresponding hash value of the set of hashed screenshots, the multiple grouped set of hashed screenshots including a first grouped set of hashed screenshots having a value within a first predetermined threshold of each other and a second grouped set of hashed screenshots having a value within a second predetermined threshold of each other;

comparing a first screenshot of the set of hashed screenshots with the first grouped set of hashed screenshots and the second grouped set of hashed screenshots; and

determining whether a security event exists based on the comparison of the first screenshot and based on a set of values of the perceptual hash associated with the set of web requests and a set of values of the cryptographic hash associated with the set of web requests;

ordering the set of hashed screenshots by hash value;

removing, from the set of hashed screenshots, a subset of the hashed screenshots that have a similar hashed value;

wherein determining whether the security event exists comprises: recommending, based on a similarity metric applied to the corresponding subset of the set of hashed screenshots, removal of the first screenshot of a corresponding first web path resulting from a first web request of the set of web requests, the first screenshot comprising an automated response to the first web request that indicates a non-normal response for the first web request, the non-normal response comprising a response that indicates a resource is not found or an illegitimate response is provided; and

wherein removing the subset of the hashed screenshots comprises: removing the first screenshot to the corresponding first web path.

2. The method of claim 1 , wherein applying the perceptual hash further comprises:

selecting a similarity metric for the perceptual hash, wherein the similarity metric is measured based on a number of bytes of the resulting hash.

3. The method of claim 1 , further comprising:

providing for display the set of hashed screenshots, ordered by corresponding hash value.

4. The method of claim 3 , wherein grouping further comprises:

grouping multiple subsets of the set of hashed screenshots based on corresponding hash value of the set of hashed screenshots; and

wherein providing for display further comprises:

providing for display a representative screenshot for each of the multiple grouped sets of screenshots.

5. The method of claim 1 , wherein the set of the web requests belong to domains owned by different organizations.

6. The method of claim 1 ,

wherein grouping further comprises:

grouping multiple subsets of the hashed screenshots based on corresponding hash value of the set of hashed screenshots; and

wherein determining further comprises:

training a machine-learning model with the multiple grouped sets of screenshots; and

using the machine-learning model for the determination.

7. A non-transitory machine-readable storage medium comprising instructions executable by a hardware processor of a computing device for identifying security events from a set of web pages, the machine-readable storage medium comprising:

instructions to obtain, for each individual web request of the set of web requests, a screenshot of the corresponding web path resulting from the individual web request;

instructions to apply a perceptual hash to each obtained screenshot to generate a set of hashed screenshots;

instructions to associate a value of the perceptual hash with the corresponding obtained screenshot;

instructions to apply a cryptographic hash to each of the obtained screenshots;

instructions to associate a value of the cryptographic hash with the corresponding obtained screenshot;

instructions to group the set of hashed screenshots into multiple grouped sets of hashed screenshots based on corresponding hash value of the set of hashed screenshots, the multiple grouped set of hashed screenshots including a first grouped set of hashed screenshots having a value within a first predetermined threshold of each other and a second grouped set of hashed screenshots having a value within a second predetermined threshold of each other;

instructions to compare a first screenshot of the set of hashed screenshots with the first grouped set of hashed screenshots and the second grouped set of hashed screenshots;

instructions to determine whether a security event exists based on the comparison of the first screenshot and based on a set of values of the perceptual hash associated with the set of web requests and a set of values of the cryptographic hash associated with the set of web requests;

instructions to order the set of hashed screenshots by hash value;

instructions to remove, from the set of hashed screenshots, a subset of the hashed screenshots that have a similar hashed value;

wherein determining whether the security event exists comprises: recommending, based on a similarity metric applied to the corresponding subset of the set of hashed screenshots, removal of the first screenshot of a corresponding first web path resulting from a first web request of the set of web requests, the first screenshot comprising an automated response to the first web request that indicates a non-normal response for the first web request, the non-normal response comprising a response that indicates a resource is not found or an illegitimate response is provided; and

wherein removing the subset of the hashed screenshots comprises: removing the first screenshot to the corresponding first web path.

8. The non-transitory machine-readable storage medium of claim 7 , wherein the instructions to apply the hash comprise:

instructions to select one or multiple types of hashes to be applied to the set of screenshots, wherein the types of hashes include: a perceptual hash; or a cryptographic hash;

instructions to select, for each hash to be applied to the set of screenshots, a similarity metric;

instructions to apply, for each hash to be applied, the corresponding hash with the similarity metric to each of the obtained screenshots.

9. The non-transitory machine-readable storage medium of claim 7 ,

wherein the instructions to group further comprise: instructions to group multiple subsets of the set of hashed screenshots based on corresponding hash value of the screenshots; and

instructions to provide for display a representative screenshot for each of the multiple grouped sets of screenshots.

10. A system for identifying security events from a set of web requests, the system comprising a hardware processor implementing machine-readable instructions that cause the system to:

obtain, for each individual web request of the set of web requests, a screenshot of a corresponding web path resulting from the individual web request;

apply a perceptual hash to each obtained screenshot to generate a set of hashed screenshots;

associate a value of the perceptual hash with the corresponding obtained screenshot;

apply a cryptographic hash to each of the obtained screenshots;

associate a value of the cryptographic hash with the corresponding obtained screenshot;

group the set of hashed screenshots into multiple grouped sets of hashed screenshots based on corresponding hash value of the set of hashed screenshots, the multiple grouped set of hashed screenshots including a first grouped set of hashed screenshots having a value within a first predetermined threshold of each other and a second grouped set of hashed screenshots having a value within a second predetermined threshold of each other;

compare a first screenshot of the set of hashed screenshots with the first grouped set of hashed screenshots and the second grouped set of hashed screenshots

determine whether a security event exists based on the comparison of the first screenshot and based on a set of values of the perceptual hash associated with the set of web requests and a set of values of the cryptographic hash associated with the set of web requests;

order the set of hashed screenshots by hash value;

remove, from the set of hashed screenshots, a subset of the hashed screenshots that have a similar hashed value;

wherein determining whether the security event exists comprises: recommending, based on a similarity metric applied to the corresponding subset of the set of hashed screenshots, removal of the first screenshot of a corresponding first web path resulting from a first web request of the set of web requests, the first screenshot comprising an automated response to the first web request that indicates a non-normal response for the first web request, the non-normal response comprising a response that indicates a resource is not found or an illegitimate response is provided; and

wherein removing the subset of the hashed screenshots comprises: removing the first screenshot to the corresponding first web path.

11. The system of claim 10 , wherein applying the perceptual hash further comprises:

selecting a similarity metric for the perceptual hash, wherein the similarity metric is measured based on a number of bytes of the resulting hash.

12. The system of claim 10 , further comprising:

providing for display the set of hashed screenshots, ordered by corresponding hash value.

13. The system of claim 12 , wherein grouping further comprises:

grouping multiple subsets of the set of hashed screenshots based on corresponding hash value of the set of hashed screenshots; and

wherein providing for display further comprises:

providing for display a representative screenshot for each of the multiple grouped sets of screenshots.

14. The system of claim 10 , wherein the set of the web requests belong to domains owned by different organizations.

15. The system of claim 10 ,

wherein grouping further comprises:

grouping multiple subsets of the hashed screenshots based on corresponding hash value of the set of hashed screenshots; and

wherein determining further comprises:

training a machine-learning model with the multiple grouped sets of screenshots; and

using the machine-learning model for the determination.