IP Library Granted Patent US 10,419,375
Granted Patent B1
US 10,419,375 · App. 15/181,457 · Granted Sep 17, 2019

Systems and methods for analyzing emotional responses to online interactions

Inventors: Ilya Sokolov (Boston, MA); Keith Newstadt (West Newton, MA)
Assignee: Symantec Corporation
H04L51/12G10L25/63H04L51/32H04L67/22
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,419,375
App. No.
15/181,457
Granted
Sep 17, 2019
Kind
B1
Abstract

The disclosed computer-implemented method for analyzing emotional responses to online interactions may include (1) identifying an online interaction of a user, (2) detecting an emotional response of the user to the online interaction by monitoring one or more emotional indicators of the user during the online interaction and determining, based on an evaluation of the one or more emotional indicators, that the emotional response of the user is outside an expected range, and (3) performing a security action in response to determining that the user's emotional response is outside the expected range. Various other methods, systems, and computer-readable media are also disclosed.

Claims (104)

1. A computer-implemented method for analyzing emotional responses to online interactions, at least a portion of the method being performed by a computing device comprising at least one processor, the method comprising:

identifying an online interaction between a user and a second user, the online interaction comprising an electronic message received by the user;

detecting an emotional response of the user to the online interaction by:

monitoring one or more emotional indicators of the user during the online interaction; and

determining, based on an evaluation of the one or more emotional indicators, that the emotional response of the user to the online interaction:

comprises a negative emotional response of the user to the online interaction; and

is outside an expected range;

evaluating the one or more emotional indicators to assign a category to an emotional relationship between the user and the second user; and

performing a security action to protect the user based on the category assigned to the emotional relationship between the user and the second user and in response to determining that the emotional response of the user to the online interaction:

comprises a negative emotional response of the user to the online interaction; and

is outside the expected range;

wherein the emotional response of the user to the online interaction further comprises a change in a measurement of at least one emotional indicator of the user that occurs in response to the online interaction.

2. The computer-implemented method of claim 1 , wherein determining that the emotional response of the user is outside the expected range comprises:

identifying previous measurements for the one or more emotional indicators of the user to determine the expected range of the one or more emotional indicators for the user; and

determining that the emotional response of the user is outside the expected range for the user.

3. The computer-implemented method of claim 1 , further comprising:

analyzing measurements of the one or more emotional indicators for a group of users to establish the expected range.

4. The computer-implemented method of claim 1 , wherein:

monitoring the one or more emotional indicators of the user during the online interaction comprises accumulating, for a plurality of users, emotional indicator measurements pertaining to the online interaction;

determining, based on the evaluation of the one or more emotional indicators, that the emotional response of the user is outside the expected range comprises determining, based on the evaluation of the emotional indicator measurements, that for at least a subset of the plurality of users the online interaction elicits emotional responses outside the expected range; and

the security action is performed in response to the determination that the online interaction elicits emotional responses outside the expected range for at least the subset of the plurality of users.

5. The computer-implemented method of claim 1 , wherein:

the method further comprises:

accumulating, for a plurality of users, emotional indicator measurements pertaining to the online interaction; and

determining, based on the evaluation of the emotional indicator measurements, that the online interaction elicits emotional responses outside the expected range for the user and within the expected range for a subset of the plurality of users; and

the security action is performed in response to the determination that the online interaction elicits emotional responses outside the expected range for the user and within the expected range for the subset of the plurality of users.

6. The computer-implemented method of claim 1 , wherein:

the method further comprises:

identifying a plurality of online interactions between the user and the second user;

monitoring one or more emotional indicators of the user during the plurality of online interactions; and

evaluating the one or more emotional indicators to assign the category to the emotional relationship between the user and the second user comprises evaluating the one or more emotional indicators of the user during the plurality of online interactions.

7. The computer-implemented method of claim 1 , wherein the electronic message comprises at least one of:

an email message;

a text message;

social network content;

an audio message; and

a video message.

8. The computer-implemented method of claim 1 , wherein the security action comprises at least one of:

blocking a message from a sender;

blocking a social network message;

reporting the online interaction to an administrator of an online service;

reporting the online interaction to a parent of the user;

reporting the online interaction to an account administrator for the user; and

reporting the online interaction to a school administrator for the user.

9. A system for analyzing emotional responses to online interactions, the system comprising:

an identification module, stored in memory, that identifies an online interaction of a user, the online interaction comprising an electronic message received by the user;

an analysis module, stored in memory, that:

detects an emotional response of the user to the online interaction by:

monitoring one or more emotional indicators of the user during the online interaction; and

determining, based on an evaluation of the one or more emotional indicators, that the emotional response of the user to the online interaction:

comprises a negative emotional response of the user to the online interaction; and

is outside an expected range; and

evaluates the one or more emotional indicators to assign a category to an emotional relationship between the user and the second user;

a security module, stored in memory, that performs a security action to protect the user based on the category assigned to the emotional relationship between the user and the second user and in response to determining that the emotional response of the user to the online interaction:

comprises a negative emotional response of the user to the online interaction; and

is outside the expected range; and

at least one physical processor configured to execute the identification module, the analysis module, and the security module;

wherein the emotional response of the user to the online interaction further comprises a change in a measurement of at least one emotional indicator of the user that occurs in response to the online interaction.

10. The system of claim 9 , wherein the analysis module determines that the emotional response of the user is outside the expected range by:

identifying previous measurements for the one or more emotional indicators of the user to determine the expected range of the one or more emotional indicators for the user; and

determining that the emotional response of the user is outside the expected range for the user.

11. The system of claim 9 , wherein the analysis module analyzes measurements of the one or more emotional indicators for a group of users to establish the expected range.

12. The system of claim 9 , wherein:

the analysis module:

monitors the one or more emotional indicators of the user during the online interaction by accumulating, for a plurality of users, emotional indicator measurements pertaining to the online interaction; and

determines, based on the evaluation of the one or more emotional indicators, that the emotional response of the user is outside the expected range by determining, based on the evaluation of the emotional indicator measurements, that for at least a subset of the plurality of users the online interaction elicits emotional responses outside the expected range; and

the security module performs the security action in response to the determination that the online interaction elicits emotional responses outside the expected range for at least the subset of the plurality of users.

13. The system of claim 9 , wherein:

the analysis module:

accumulates, for a plurality of users, emotional indicator measurements pertaining to the online interaction; and

determines, based on the evaluation of the emotional indicator measurements, that the online interaction elicits emotional responses outside the expected range for the user and within the expected range for a subset of the plurality of users; and

the security module performs the security action in response to the determination that the online interaction elicits emotional responses outside the expected range for the user and within the expected range for the subset of the plurality of users.

14. The system of claim 9 , wherein:

the identification module identifies a plurality of online interactions between the user and the second user;

the analysis module:

monitors one or more emotional indicators of the user during the plurality of online interactions; and

evaluates the one or more emotional indicators to assign the category to the emotional relationship between the user and the second user by evaluating the one or more emotional indicators of the user during the plurality of online interactions.

15. The system of claim 9 , wherein the electronic message comprises at least one of:

an email message;

a text message;

social network content;

an audio message; and

a video message.

16. A non-transitory computer-readable medium comprising one or more computer-readable instructions that, when executed by at least one processor of a computing device, cause the computing device to:

identify an online interaction between a user and a second user, the online interaction comprising an electronic message received by the user;

detect an emotional response of the user to the online interaction by:

monitoring one or more emotional indicators of the user during the online interaction; and

determining, based on an evaluation of the one or more emotional indicators, that the emotional response of the user to the online interaction:

comprises a negative emotional response of the user to the online interaction; and

is outside an expected range;

evaluate the one or more emotional indicators to assign a category to an emotional relationship between the user and the second user; and

perform a security action to protect the user based on the category assigned to the emotional relationship between the user and the second user and in response to determining that the emotional response of the user to the online interaction:

comprises a negative emotional response of the user to the online interaction; and

is outside the expected range;

wherein the emotional response of the user to the online interaction further comprises a change in a measurement of at least one emotional indicator of the user that occurs in response to the online interaction.

17. The non-transitory computer-readable medium of claim 16 , wherein the one or more computer-readable instructions cause the computing device to determine that the emotional response of the user is outside the expected range by:

identifying previous measurements for the one or more emotional indicators of the user to determine the expected range of the one or more emotional indicators for the user; and

determining that the emotional response of the user is outside the expected range for the user.

18. The non-transitory computer-readable medium of claim 16 , wherein the one or more computer-readable instructions cause the computing device to analyze measurements of the one or more emotional indicators for a group of users to establish the expected range.

19. The non-transitory computer-readable medium of claim 16 , wherein the one or more computer-readable instructions cause the computing device to:

monitor the one or more emotional indicators of the user during the online interaction by accumulating, for a plurality of users, emotional indicator measurements pertaining to the online interaction;

determine, based on the evaluation of the one or more emotional indicators, that the emotional response of the user is outside the expected range by determining, based on the evaluation of the emotional indicator measurements, that for at least a subset of the plurality of users the online interaction elicits emotional responses outside the expected range; and

perform the security action in response to the determination that the online interaction elicits emotional responses outside the expected range for at least the subset of the plurality of users.

20. The method of claim 1 , wherein the security action comprises monitoring additional emotional indicators for the user.

Assignments (6)
CHANGE OF NAME Recorded Feb 6, 2023
From: NORTONLIFELOCK INC.
To: GEN DIGITAL INC.
Reel/Frame 062714/0605 →
NOTICE OF SUCCESSION OF AGENCY (REEL 050926 / FRAME 0560) Recorded Sep 13, 2022
From: JPMORGAN CHASE BANK, N.A.
To: BANK OF AMERICA, N.A., AS COLLATERAL AGENT
Reel/Frame 061422/0371 →
SECURITY AGREEMENT Recorded Sep 13, 2022
From: NORTONLIFELOCK INC.
To: BANK OF AMERICA, N.A., AS COLLATERAL AGENT
Reel/Frame 062220/0001 →
CHANGE OF NAME Recorded Feb 14, 2020
From: SYMANTEC CORPORATION
To: NORTONLIFELOCK INC.
Reel/Frame 051935/0228 →
SECURITY AGREEMENT Recorded Nov 4, 2019
From: SYMANTEC CORPORATION; BLUE COAT LLC; LIFELOCK, INC,; SYMANTEC OPERATING CORPORATION
To: JPMORGAN, N.A.
Reel/Frame 050926/0560 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 14, 2016
From: SOKOLOV, ILYA; NEWSTADT, KEITH
To: SYMANTEC CORPORATION
Reel/Frame 038903/0762 →
Cited By (1)
US 12,683,920