IP Library Granted Patent US 9,923,931
Granted Patent B1
US 9,923,931 · App. 15/017,388 · Granted Mar 20, 2018

Systems and methods for identifying violation conditions from electronic communications

Inventors: John Wagster (Franklin, TN); Robert Metcalf (Franklin, TN); Keith Ellis Massey (Nashville, TN); Kenneth Loran Graham (Nashville, TN); Sarah Cannon (Franklin, TN); Adam Jaggers (Franklin, TN); Vishnuvardhan Balluru (Franklin, TN); Bill Dipietro (Franklin, TN)
Assignee: Digital Reasoning Systems, Inc.
H04L63/30H04L43/10
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 9,923,931
App. No.
15/017,388
Granted
Mar 20, 2018
Kind
B1
Abstract

Some aspects of the present disclosure relate to systems and methods for identifying potential violation conditions from electronic communications. In one embodiment, a method includes receiving data associated with an electronic communication and detecting, from the received data, and using a trainable model, an indicator of a potential violation condition, where the violation condition is associated with an activity that is a violation of a predetermined standard. The method also includes, responsive to detecting the indicator of the potential violation condition, marking the electronic communication as being associated with a potential violation condition, and presenting the potential violation condition to a user for review. The method also includes receiving a decision from the user, based on the review, on whether the electronic communication is associated with a violation condition, and based on the decision, improving the model for detecting potential violation conditions in other electronic communications.

Claims (41)

1. A computer-implemented method, comprising:

receiving data associated with an electronic communication, the data comprising text data of text content from the body of at least one of a plurality of messages and a plurality of advertisements;

determining semantic meaning of the text content based at least in part on contextual usage of words in the text content;

detecting, from the received data, and using an unsupervised machine learning model, at least one indicator of a potential violation condition, wherein a violation condition is associated with a human activity that is a violation of a predetermined standard,

wherein the detecting of the at least one indicator is based at least in part on the determined semantic meaning and comprises performing entity resolution on the text data, across the at least one of the plurality of messages and plurality of advertisements, to resolve coreferent mentions to entities, wherein the coreferent mentions comprise a plurality of aliases corresponding to the same person and resolving the coreferent mentions comprises determining that the plurality of aliases refer to the same person, and

wherein the at least one indicator is comprised of features defining behavioral patterns of one or more particular individuals, and the behavioral patterns comprise a plurality of actions by the one or more individuals;

responsive to detecting the at least one indicator of the potential violation condition, marking the electronic communication as being associated with a potential violation condition;

presenting the potential violation condition to a reviewing user for review, wherein the reviewing user is the first user or a second user;

receiving a decision from the reviewing user, based on the review, on whether the electronic communication is associated with a violation condition; and

based on the decision, adjusting at least one of feature selection, pattern selection, feature weighting, pattern weighting, and alerting thresholds, to improve the accuracy of the unsupervised machine learning model used for detecting potential violation conditions.

2. The method of claim 1 , wherein the received data further comprises metadata associated with the electronic communication.

3. The method of claim 2 , wherein the metadata comprises an identifier of at least one of a sender or recipient, a time stamp, a domain, and a server.

4. The method of claim 1 , wherein the electronic communication is a communication between humans.

5. The method of claim 1 , wherein the predetermined standard is a legal or ethical standard.

6. The method of claim 1 , wherein the model is configured to generate a prediction that the indicator identifies a violation condition.

7. The method of claim 1 , wherein the at least one indicator comprises a language pattern indicative of a violation condition.

8. The method of claim 1 , wherein the at least one indicator is detected based at least in part on a context of the electronic communication.

9. The method of claim 1 , wherein the at least one indicator is detected based at least in part on at least one of domain, audience, and tone associated with the electronic communication.

10. The method of claim 1 , wherein marking the electronic communication as being associated with a potential violation condition comprises flagging one or more specific portions of the electronic communication or the entire electronic communication as being associated with a potential violation condition.

11. The method of claim 1 , wherein presenting the potential violation condition to the reviewing user for review comprises at least one of generating and sending an alert to the reviewing user.

12. The method of claim 1 , wherein presenting the potential violation condition to the reviewing user for review comprises presenting some or all of the electronic communication to the reviewing user.

13. The method of claim 1 , wherein the decision from the reviewing user comprises a decision to discard the electronic communication from being considered as associated with a potential violation condition, a decision to escalate the electronic communication to a higher authority user for review, or a decision to confirm that the electronic communication is associated with a potential violation condition.

14. The method of claim 13 , wherein the decision comprises an indication of a true positive, false positive, true negative, or false negative in relation to a potential violation condition.

15. The method of claim 13 , wherein the decision is associated with a degree or weighting of the electronic communication as indicating a potential violation condition.

16. The method of claim 1 , wherein improving the accuracy of the model comprises adjusting at least one of feature selection, pattern selection, feature weighting, pattern weighting, and alerting thresholds.

17. The method of claim 1 , wherein the at least one indicator of the potential violation condition is one of a plurality of possible indicators of potential violation conditions, and wherein improving the accuracy of the model comprises adding an indicator to the plurality of possible indicators.

18. The method of claim 17 , wherein improving the accuracy of the model comprises assigning a weighting to the added indicator.

19. The method of claim 17 , wherein the at least one indicator of the potential violation condition is one of a plurality of possible indicators of potential violation conditions, and wherein improving the accuracy of the model comprises selecting one or more particular indicators of the plurality of possible indicators for future runs of the model.

20. The method of claim 1 , wherein the at least one indicator comprises one or more indicators created by the first user.

21. The method of claim 1 , wherein the at least one indicator comprises a combination of features or patterns that is indicative of unauthorized financial activity by the one or more individuals.

22. The method of claim 21 , wherein the unauthorized financial activity comprises at least one of insider trading, bribery, and money laundering.

23. The method of claim 1 , wherein the at least one indicator comprises a combination of features or patterns that is indicative of illegal activities associated with sexual services.

24. The method of claim 1 , wherein the at least one indicator comprises a combination of features or patterns associated with human trafficking.

25. The method of claim 1 , wherein adjusting at least one of feature selection, pattern selection, feature weighting, pattern weighting, and alerting thresholds comprises at least one of:

retaining a particular feature for future runs of the model; adding a new feature;

raising or lowering a weighting of a particular feature; and raising or lowering an alerting threshold.

26. The method of claim 1 , wherein performing the entity resolution on the text data comprises resolving the plurality of aliases based on contextual features associated with the context of usage of the plurality of aliases across the at least one of the plurality of messages and plurality of advertisements.

27. The method of claim 26 , wherein the contextual features associated with the context of usage of the plurality of aliases include contextual features corresponding to geographic locations of the same person across the at least one of the plurality of messages and plurality of advertisements.

28. The method of claim 26 , wherein the contextual features associated with the context of usage include contextual features corresponding to consistency of actions of the same person across the at least one of the plurality of messages and plurality of advertisements.

29. A system, comprising: one or more processors; and at least one memory device storing instructions which, when executed by the one or more processors, cause the system to perform functions that include: receiving data associated with an electronic communication, the data comprising text data of text content from the body of at least one of a plurality of messages and a plurality of advertisements; determining semantic meaning of the text content based at least in part on contextual usage of words in the text content; detecting, from the received data, and using an unsupervised machine learning model, at least one indicator of a potential violation condition, wherein a violation condition is associated with a human activity that is a violation of a predetermined standard, wherein the detecting of the at least one indicator is based at least in part on the determined semantic meaning and comprises performing entity resolution on the text data, across the at least one of the plurality of messages and plurality of advertisements, to resolve coreferent mentions to entities, wherein the coreferent mentions comprise a plurality of aliases corresponding to the same person and resolving the coreferent mentions comprises determining that the plurality of aliases refer to the same person, and wherein the at least one indicator is comprised of features defining behavioral patterns of one or more particular individuals, and the behavioral patterns comprise a plurality of actions by the one or more individuals; responsive to detecting the at least one indicator of the potential violation condition, marking the electronic communication as being associated with a potential violation condition; presenting the potential violation condition to a reviewing user for review, wherein the reviewing user is the first user or a second user; receiving a decision from the reviewing user, based on the review, on whether the electronic communication is associated with a violation condition; and based on the decision, adjusting at least one of feature selection, pattern selection, feature weighting, pattern weighting, and alerting thresholds, to improve the accuracy of the unsupervised machine learning model used for detecting potential violation conditions.

30. A non-transitory computer-readable medium storing instructions which, when executed by one or more processors, cause a computing device to perform functions that comprise: receiving data associated with an electronic communication, the data comprising text data of text content from the body of at least one of a plurality of messages and a plurality of advertisements; determining semantic meaning of the text content based at least in part on contextual usage of words in the text content; detecting, from the received data, and using an unsupervised machine learning model, at least one indicator of a potential violation condition, wherein a violation condition is associated with a human activity that is a violation of a predetermined standard, wherein the detecting of the at least one indicator is based at least in part on the determined semantic meaning and comprises performing entity resolution on the text data, across the at least one of the plurality of message and plurality of advertisements, to resolve coreferent mentions to entities, wherein the coreferent mentions comprise a plurality of aliases corresponding to the same person and resolving the coreferent mentions comprises determining that the plurality of aliases refer to the same person, and Wherein the at least one indicator is comprised of features defining behavioral patterns of one or more particular individuals, and the behavioral patterns comprise a plurality of actions by the one or more individuals; responsive to detecting the at least one indicator of the potential violation condition, marking the electronic communication as being associated with a potential violation condition; presenting the potential violation condition to a reviewing user for review, wherein the reviewing user is the first user or a second user; receiving a decision from the reviewing user, based on the review, on whether the electronic communication is associated with a violation condition; and based on the decision, adjusting at least one of feature selection, pattern selection, feature weighting, pattern weighting, and alerting thresholds, to improve the accuracy of the unsupervised machine learning model used for detecting potential violation conditions.

Assignments (6)
TERMINATION AND RELEASE OF PATENT SECURITY AGREEMENT AT REEL/FRAME NO. 54537/0541 Recorded Feb 22, 2022
From: PNC BANK, NATIONAL ASSOCIATION
To: DIGITAL REASONING SYSTEMS, INC.; MOBILEGUARD, LLC; ACTIANCE, INC.; ENTREDA, INC.
Reel/Frame 059353/0549 →
PATENT SECURITY AGREEMENT Recorded Feb 18, 2022
From: DIGITAL REASONING SYSTEMS, INC.
To: OWL ROCK CAPITAL CORPORATION, AS COLLATERAL AGENT
Reel/Frame 059191/0435 →
SECURITY INTEREST Recorded Dec 3, 2020
From: DIGITAL REASONING SYSTEMS, INC.; MOBILEGUARD, LLC; ACTIANCE, INC.; ENTRADA, INC.
To: PNC BANK, NATIONAL ASSOCIATION
Reel/Frame 054537/0541 →
RELEASE OF SECURITY INTEREST : RECORDED AT REEL/FRAME - 050289/0090 Recorded Nov 23, 2020
From: MIDCAP FINANCIAL TRUST
To: DIGITAL REASONING SYSTEMS, INC.
Reel/Frame 054499/0041 →
SECURITY INTEREST Recorded Sep 6, 2019
From: DIGITAL REASONING SYSTEMS, INC.
To: MIDCAP FINANCIAL TRUST, AS AGENT
Reel/Frame 050289/0090 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 28, 2017
From: WAGSTER, JOHN; METCALF, ROBERT; MASSEY, KEITH ELLIS; GRAHAM, KENNETH LORAN; CANNON, SARAH; JAGGERS, ADAM; BALLURU, VISHNUVARDHAN; DIPIETRO, BILL
To: DIGITAL REASONING SYSTEMS, INC.
Reel/Frame 042832/0920 →