IP Library Granted Patent US 11,895,137
Granted Patent B2
US 11,895,137 · App. 18/061,195 · Granted Feb 6, 2024

Phishing data item clustering and analysis

Inventors: David Cohen (Mountain View, CA); Jason Ma (Mountain View, CA); Bing Jie Fu (Redwood City, CA); Ilya Nepomnyashchiy (Mountain View, CA); Steven Berler (Menlo Park, CA); Alex Smaliy (Palo Alto, CA); Jack Grossman (Albuquerque, NM); James Thompson (London, GB); Julia Boortz (Menlo Park, CA); Matthew Sprague (Palo Alto, CA); Parvathy Menon (San Jose, CA); Michael Kross (Palo Alto, CA); Michael Harris (Palo Alto, CA); Adam Borochoff (New York, NY)
Assignee: Palantir Technologies Inc.
H04L63/1425G06F16/285G06Q40/12H04L63/145H04L63/1408
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,895,137
App. No.
18/061,195
Granted
Feb 6, 2024
Kind
B2
Abstract

Embodiments of the present disclosure relate to a data analysis system that may automatically generate memory-efficient clustered data structures, automatically analyze those clustered data structures, and provide results of the automated analysis in an optimized way to an analyst. The automated analysis of the clustered data structures (also referred to herein as data clusters) may include an automated application of various criteria or rules so as to generate a compact, human-readable analysis of the data clusters. The human-readable analyses (also referred to herein as “summaries” or “conclusions”) of the data clusters may be organized into an interactive user interface so as to enable an analyst to quickly navigate among information associated with various data clusters and efficiently evaluate those data clusters in the context of, for example, a fraud investigation. Embodiments of the present disclosure also relate to automated scoring of the clustered data structures.

Claims (128)

1. A computer-implemented method comprising:

by one or more hardware computer processors executing code:

communicating with one or more electronic data structures configured to store:

a data clustering strategy; and

a plurality of data items including at least:

a plurality of email data items, each of the plurality of email data items including at least a subject and a sender, each of the plurality of email data items potentially associated with phishing activity; and

a plurality of phishing-related data items related to a communications network of an organization, the plurality of phishing-related data items including at least one of: internal Internet Protocol addresses of the communications network, computerized devices of the communications network, users of particular computerized devices, organizational positions associated with users of particular computerized devices, or URLs and/or external domains visited by users of particular computerized devices;

accessing an email data item transmitted to one or more of the users of respective computerized devices within the network of the organization, the email data item including at least a subject and a sender, the email data item potentially associated with phishing activity;

designating the accessed email data item as a seed; and

generating a data item cluster based on the data clustering strategy by at least:

adding the seed to the data item cluster;

determining the subject and the sender associated with the seed;

identifying one or more of the plurality of email data items having a same subject as the determined subject or a same sender as the determined sender;

adding the identified one or more email data items to the data item cluster;

parsing one or more URLs from the email data items of the data item cluster;

adding the parsed URLs to the data item cluster;

identifying one or more users who are both recipients of at least one of the email data items of the data item cluster and visitors of one of the URLs of the data item cluster;

adding the identified one or more users, including data related to the one or more users, to the data item cluster;

identifying additional one or more data items associated with any data items of the data item cluster; and

adding, to the data item cluster, the additional one or more data items.

2. The computer-implemented method of claim 1 , wherein generating the data item cluster based on the data clustering strategy further comprises:

determining any new subjects or new senders associated with email data items of the data item cluster that are different from the determined subjects or the determined senders;

identifying a second one or more of the plurality of email data items having a same subject as the determined new subject, or a same sender as the determined new sender; and

adding the identified second one or more email data items to the data item cluster.

3. The computer-implemented method of claim 1 , wherein the identified one or more email data items are added to the data item cluster only if received by one or more computerized devices within the network within a predetermined period of time from a time that the seed was received.

4. The computer-implemented method of claim 3 , wherein the period of time comprises at least one of a number of hours, a number of days, or a number of weeks.

5. The computer-implemented method of claim 3 , wherein the predetermined period of time is further determined based on other email data items in the data item cluster.

6. The computer-implemented method of claim 1 , wherein identifying the one or more users further comprises:

scanning communications on the communications network of the organization so as to generate phishing-related data items including URLs visited by particular users;

extracting recipients of the email data items of the data item cluster associated with respective parsed URLs; and

for any parsed URL matching a URL visited by a particular user, if the extracted recipient of the email data item associated with the parsed URL matches the particular user, then identifying the user.

7. The computer-implemented method of claim 6 , wherein the communications are continuously scanned via a proxy.

8. The computer-implemented method of claim 1 , further comprising:

by the one or more hardware computer processors executing code:

continuously receiving email data items from users of respective computing devices of the organization, designate the received email data items as seeds, and generate data items clusters based on the data clustering strategy.

9. The computer-implemented method of claim 1 , wherein the data related to the one or more users includes an organizational position associated with the user.

10. The computer-implemented method of claim 1 , wherein:

the one or more electronic data structures are further configured to store a plurality of data cluster analysis rules associated with the data clustering strategy, and

the computer-implemented method further comprises:

for the generated data item cluster:

accessing the plurality of data cluster analysis rules associated with the data clustering strategy;

analyzing the data item cluster based on the accessed data cluster analysis rules; and

based on the analysis of the data item cluster:

determining an alert score for the data item cluster; and

generating one or more human-readable conclusions regarding the data item cluster.

11. A computer-implemented method comprising:

by one or more hardware computer processors executing code:

communicating with one or more electronic data structures configured to store:

a data clustering strategy; and

a plurality of data items including at least:

a plurality of email data items, each of the plurality of email data items including at least a subject and a sender, each of the plurality of email data items potentially associated with phishing activity; and

a plurality of phishing-related data items related to customers of an organization, the plurality of phishing-related data items including indicators of at least one of: customers of the organization or URLs identified as malicious by a third-party service;

receiving a plurality of email data items from customers of the organization, each of the email data items including at least a subject and a sender, each of the email data items potentially associated with phishing activity;

designating each of the received email data items as seeds; and

for each of the designated seeds, generating a data item cluster based on the data clustering strategy by at least:

adding the seed to the data item cluster;

determining the subject and the sender associated with the seed;

accessing the one or more electronic data structures and identifying one or more of the plurality of email data items having a same subject as the determined subject or a same sender as the determined sender;

adding the identified one or more email data items to the data item cluster;

parsing one or more URLs from the email data items of the data item cluster;

adding the URLs to the data item cluster;

in response to determining that the data item cluster includes at least a predetermined threshold quantity of email data items, designating the data item cluster as a campaign cluster;

identifying additional one or more data items associated with any data items of the data item cluster; and

adding, to the data item cluster, the additional one or more data items.

12. The computer-implemented method of claim 11 further comprising:

by the one or more hardware computer processors executing code:

for each campaign cluster, initiating further automated investigation including at least:

comparing URLs included in the campaign cluster with URLs previously identified as malicious by a third-party service;

based on the comparing, identifying any URLs included in the campaign cluster not previously identified as malicious by the third-party service; and

notifying the third-party service of the identified URLs.

13. The computer-implemented method of claim 11 , wherein the predetermined threshold quantity is 1, 2, 3, 4, or 5.

14. The computer-implemented method of claim 11 , wherein generating the data item cluster based on the data clustering strategy further comprises:

determining any new subjects or new senders associated with email data items of the data item cluster that are different from the determined subjects or senders;

identifying a second one or more of the plurality of email data items having a subject that is the same as the determined new subject, or a sender that is the same as the determined new sender; and

adding the identified second one or more email data items to the data item cluster.

15. The computer-implemented method of claim 11 , wherein the identified one or more email data items are added to the data item cluster only if they were received by an initial recipient within a period of time determined based on the seed.

16. The computer-implemented method of claim 15 , wherein the period of time comprises at least one of a number of hours, a number of days, or a number of weeks, and wherein the received email data item designated as the seed was received by an initial recipient within the time period.

17. The computer-implemented method of claim 15 , wherein the period of time is further determined based on other email data items in the data item cluster.

18. The computer-implemented method of claim 11 further comprising:

by the one or more hardware computer processors executing code:

continuously receiving email data items from customers of the organization, store the received email data items, designate the received email data items as seeds, and generate, for each of the designated seeds, data items clusters based on the data clustering strategy.

19. The computer-implemented method of claim 11 , wherein:

the one or more electronic data structures are further configured to store a plurality of data cluster analysis rules associated with the data clustering strategy, and

the computer-implemented method further comprises:

for each generated data item cluster:

accessing the plurality of data cluster analysis rules associated with the data clustering strategy;

analyzing the data item cluster based on the accessed data cluster analysis rules; and

based on the analysis of the data item cluster:

determining an alert score for the data item cluster; and

generating one or more human-readable conclusions regarding the data item cluster,

wherein the alert score indicates a degree of correlation between characteristics of the data item cluster and the accessed data cluster analysis rules.

20. One or more non-transitory computer-readable storage devices storing software instructions, the software instructions executable by one or more processors to cause the one or more processors to perform operations comprising:

communicating with one or more electronic data structures configured to store:

a data clustering strategy; and

a plurality of data items including at least:

a plurality of email data items, each of the plurality of email data items including at least a subject and a sender, each of the plurality of email data items potentially associated with phishing activity; and

a plurality of phishing-related data items related to a communications network of an organization, the plurality of phishing-related data items including at least one of: internal Internet Protocol addresses of the communications network, computerized devices of the communications network, users of particular computerized devices, organizational positions associated with users of particular computerized devices, or URLs and/or external domains visited by users of particular computerized devices;

accessing an email data item transmitted to one or more of the users of respective computerized devices within the network of the organization, the email data item including at least a subject and a sender, the email data item potentially associated with phishing activity;

designating the accessed email data item as a seed; and

generating a data item cluster based on the data clustering strategy by at least:

adding the seed to the data item cluster;

determining the subject and the sender associated with the seed;

identifying one or more of the plurality of email data items having a same subject as the determined subject or a same sender as the determined sender;

adding the identified one or more email data items to the data item cluster;

parsing one or more URLs from the email data items of the data item cluster;

adding the parsed URLs to the data item cluster;

identifying one or more users who are both recipients of at least one of the email data items of the data item cluster and visitors of one of the URLs of the data item cluster;

adding the identified one or more users, including data related to the one or more users, to the data item cluster;

identifying additional one or more data items associated with any data items of the data item cluster; and

adding, to the data item cluster, the additional one or more data items.

21. One or more non-transitory computer-readable storage devices storing software instructions, the software instructions executable by one or more processors to cause the one or more processors to perform operations comprising:

communicating with one or more electronic data structures configured to store:

a data clustering strategy; and

a plurality of data items including at least:

a plurality of email data items, each of the plurality of email data items including at least a subject and a sender, each of the plurality of email data items potentially associated with phishing activity; and

a plurality of phishing-related data items related to customers of an organization, the plurality of phishing-related data items including indicators of at least one of: customers of the organization or URLs identified as malicious by a third-party service;

receiving a plurality of email data items from customers of the organization, each of the email data items including at least a subject and a sender, each of the email data items potentially associated with phishing activity;

designating each of the received email data items as seeds; and

for each of the designated seeds, generating a data item cluster based on the data clustering strategy by at least:

adding the seed to the data item cluster;

determining the subject and the sender associated with the seed;

accessing the one or more electronic data structures and identifying one or more of the plurality of email data items having a same subject as the determined subject or a same sender as the determined sender;

adding the identified one or more email data items to the data item cluster;

parsing one or more URLs from the email data items of the data item cluster;

adding the URLs to the data item cluster;

in response to determining that the data item cluster includes at least a predetermined threshold quantity of email data items, designating the data item cluster as a campaign cluster;

identifying additional one or more data items associated with any data items of the data item cluster; and

adding, to the data item cluster, the additional one or more data items.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Dec 22, 2022
From: COHEN, DAVID; MA, JASON; FU, BING JIE; NEPOMNYASHCHIY, ILYA; BERLER, STEVEN; SMALIY, ALEX; GROSSMAN, JACK; THOMPSON, JAMES; BOORTZ, JULIA; SPRAGUE, MATTHEW; MENON, PARVATHY; KROSS, MICHAEL; HARRIS, MICHAEL; BOROCHOFF, ADAM
To: PALANTIR TECHNOLOGIES INC.
Reel/Frame 062210/0363 →
Continuity (6)
Continuation 17003398 · Aug 26, 2020
Continuation 15961431 · Apr 24, 2018
Continuation 14487021 · Sep 15, 2014
Continuation 14473920 · Aug 29, 2014
Provisional Application 62020876 · Jul 3, 2014
Related Publication 20230096596A1 · Mar 30, 2023