IP Library Granted Patent US 11,211,095
Granted Patent B1
US 11,211,095 · App. 16/907,096 · Granted Dec 28, 2021

Modifying media content playback based on user mental state

Inventors: Joseph Verbeke (San Francisco, CA); Sven Kratz (Saratoga, CA); Adam Boulanger (Palo Alto, CA); Priya Seshadri (Stamford, CT); Stefan Marti (Oakland, CA)
Assignee: Harman International Industries, Incorporated
G11B27/005H04N21/44213
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,211,095
App. No.
16/907,096
Granted
Dec 28, 2021
Kind
B1
Abstract

Embodiments of the present disclosure sets forth a computer-implemented method for playing back media content comprising determining, based on sensor data, a first mental state associated with a user accessing media content, and modifying playback of media content based on the first mental state.

Claims (50)

1. A computer-implemented method for playing back media content, the method comprising:

acquiring sensor data via one or more sensors that perform measurements related to at least one of a user or an environment of the user;

determining, based on the sensor data, a first mental state metric associated with the user during playback of media content;

determining a first component of the first mental state metric that is caused by the user consuming the media content;

removing the first component of the first mental state metric from the first mental state metric to produce a second component of the first mental state metric; and

modifying playback of the media content based on the second component of the first mental state metric.

2. The computer-implemented method of claim 1 , wherein modifying the playback of the media content comprises:

rewinding and replaying the media content from a point in time at which the first mental state metric satisfies a threshold value.

3. The computer-implemented method of claim 1 , wherein modifying the playback of the media content comprises:

pausing the playback of the media content in response to the second component of the first mental state metric satisfying a threshold value; and

resuming the playback of the media content in response to the second component of the first mental state metric not satisfying the threshold value.

4. The computer-implemented method of claim 3 , further comprising:

storing a portion of the media content from a first time when the second component of the first mental state metric satisfies the threshold value; and

playing a temporally compressed version of the portion of the media content when the playback of the media content is resumed.

5. The computer-implemented method of claim 1 , further comprising:

determining a second mental state metric associated with the environment of the user; and

modifying playback of the media content based on the second mental state metric.

6. The computer-implemented method of claim 1 , wherein the first mental state metric comprises at least one of a cognitive load state metric, an emotional load state metric, or an amount of mind wandering metric.

7. The computer-implemented method of claim 1 , wherein the first mental state metric is determined using a pupillometry-based technique.

8. The computer-implemented method of claim 1 , wherein the first mental state metric is determined using a machine-learning-based technique.

9. The computer-implemented method of claim 1 , wherein the media content comprises one of audio content or video content.

10. One or more non-transitory computer-readable storage media including instructions that, when executed by one or more processors, cause the one or more processors to play back media content, by performing steps of:

determining, based on sensor data, a first mental state metric associated with a user during playback of media content;

determining a first component of the first mental state metric that is caused by the user consuming the media content;

removing the first component of the first mental state metric from the first mental state metric to produce a second component of the first mental state metric; and

modifying playback of the media content based on the second component of the first mental state metric.

11. The one or more non-transitory computer-readable storage media of claim 10 , wherein modifying the playback of the media content comprises rewinding and replaying the media content from a point in time at which the second component of the first mental state metric satisfies a threshold value.

12. The one or more non-transitory computer-readable storage media of claim 11 , wherein the rewinding and replaying of the media content is performed in response to a user request.

13. The one or more non-transitory computer-readable storage media of claim 11 , the steps further comprising determining the threshold value based on historical mental state data associated with the user.

14. The one or more non-transitory computer-readable storage media of claim 10 , wherein modifying the playback of the media content comprises:

pausing the playback of the media content in response to the second component of the first mental state metric satisfying a threshold value; and

resuming the playback of the media content in response to the second component of the first mental state metric not satisfying the threshold value.

15. The one or more non-transitory computer-readable storage media of claim 14 , the steps further comprising:

storing a portion of the media content from a first time when the second component of the first mental state metric satisfies the threshold value; and

playing a temporally compressed version of the portion of the media content when the playback of the media content is resumed.

16. The one or more non-transitory computer-readable storage media of claim 10 , the steps further comprising:

determining a second mental state metric associated with an environment of the user; and

modifying playback of the media content based on the second mental state metric.

17. The one or more non-transitory computer-readable storage media of claim 10 , wherein the first mental state metric comprises at least one of a cognitive load state metric, an emotional load state metric, or an amount of mind wandering metric.

18. The one or more non-transitory computer-readable storage media of claim 10 , wherein the first mental state metric is determined using at least one of a pupillometry-based technique or a machine-learning-based technique.

19. A device, comprising:

a memory that includes instructions; and

a processor that is coupled to the memory and, when executing the instructions:

determines, based on sensor data, a first mental state metric associated with a user during playback of media content;

determines a first component of the first mental state metric that is caused by the user consuming the media content;

removes the first component of the first mental state metric from the first mental state metric to produce a second component of the first mental state metric; and

modifies playback of the media content based on the second component of the first mental state metric.

20. The device of claim 19 , wherein the processor, when executing the instructions, further:

determines a second mental state metric based on data included in the sensor data that is associated with an environment; and

modifies playback of the media content based on the second mental state metric.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Sep 30, 2020
From: VERBEKE, JOSEPH; KRATZ, SVEN; BOULANGER, ADAM; SESHADRI, PRIYA; MARTI, STEFAN
To: HARMAN INTERNATIONAL INDUSTRIES, INCORPORATED
Reel/Frame 053931/0672 →
Cited By (1)
US 12,373,508