IP Library › Granted Patent US 10,950,254
Granted Patent B2
US 10,950,254 · App. 16/170,110 · Granted Mar 16, 2021

Producing comprehensible subtitles and captions for an effective group viewing experience

Inventors: Garfield Vaughn (South Windsor, CT); Moncef Benboubakeur (Brno, CZ); Julija Narodicka (Brno, CZ); Aaron K. Baughman (Silver Spring, MD)
Assignee: INTERNATIONAL BUSINESS MACHINES CORPORATION
G10L21/10G06N3/08G10L15/22G10L15/26G10L15/30H04N21/4884
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,950,254
App. No.
16/170,110
Filed
Oct 25, 2018
Granted
Mar 16, 2021
Kind
B2
Examiner
HE, JIALONG
Art Unit
2659
USPC
704/235
Abstract

One or more processors identify one or more users expected to consume media content having associated subtitle data. A user profile associated with each of the one or more users is received, and one or more features are extracted from each user profile. The one or more features are representative of a characteristic of the user. A group profile is created for the one or more users based upon the extracted features. The subtitle data associated with the media content is received, and one or more portions of the subtitle data are modified based upon the group profile to generate augmented subtitle data. The augmented subtitle data is sent to a display device for being rendered in the display device.

Claims (56)

1. A computer-implemented method, comprising:

identifying, by one or more processors, one or more users expected to consume media content, the media content having associated subtitle data;

receiving a user profile associated with each of the one or more users;

extracting one or more features from each user profile, the one or more features representative of a characteristic of the user;

creating a group profile that is representative of features of the plurality of the one or more users said group profile being based upon the extracted features;

receiving the subtitle data associated with the media content;

automatically modifying one or more portions of the subtitle data based upon the group profile to generate augmented subtitle data; and

sending the augmented subtitle data to a display device for being rendered in the display device;

wherein the modifying includes using a machine learning model that has been trained with training datasets associated with a plurality of one or more other users, the training datasets including a lexicon, a demographic characteristic and/or a somatic state of said one or more other users,

wherein responsive to receiving the subtitle data and values of the features from the group profile, computing a corresponding array of values as input to said machine learning model in order to produce an output array that contains the augmented subtitle data.

2. The computer-implemented method of claim 1 , further comprising:

rendering the augmented subtitle data in the display device.

3. The computer-implemented method of claim 2 , further comprising:

presenting the media content by the display device in synchronization with the rendering of the augmented subtitle data.

4. The computer-implemented method of claim 1 , further comprising:

determining a change in a level of engagement of one of more of the users with the media content; and

further modifying the subtitle data to obtain a higher level of engagement of the one or more users with the media content.

5. The computer-implemented method of claim 4 , further comprising determining the level of engagement based upon an input from one or more sensors within a viewing environment.

6. The computer-implemented method of claim 1 , wherein extracting the one or more features is based upon cognitive processing of the user profile.

7. The computer-implemented method of claim 1 , wherein creating the group profile includes determining a value of a feature that satisfies a best match within a threshold value of the feature among the one or more users.

8. The computer-implemented method of claim 1 , wherein the user profile is based upon one or more of previous content consumed by the user, social media posts by the user, and input from a sensor associated with the user.

9. The computer-implemented method of claim 1 , wherein the one or more features include one or more of a language usage, a parental guide rating, a demographic characteristic, a mood, an experience, a motion, physical health, and a lexicon of the user.

10. The computer-implemented method of claim 1 , further comprising:

identifying a source of the media content.

11. The computer-implemented method of claim 1 , wherein the identifying of the one or more users includes receiving a list of the one or more users.

12. The computer-implemented method of claim 1 , wherein the receiving of the subtitle data associated with the media content is responsive to requesting the subtitled data.

13. A computer usable program product comprising one or more computer-readable storage devices, and program instructions stored on at least one of the one or more storage devices, the stored program instructions comprising:

program instructions to identify, by one or more processors, one or more users expected to consume media content, the media content having associated subtitle data;

program instructions to receive a user profile associated with each of the one or more users;

program instructions to extract one or more features from each user profile, the one or more features representative of a characteristic of the user;

program instructions to create a group profile that is representative of features of the plurality of the one or more users said group profile being based upon the extracted features;

program instructions to receive the subtitle data associated with the media content;

program instructions to automatically modify one or more portions of the subtitle data based upon the group profile to generate augmented subtitle data; and

program instructions to send the augmented subtitle data to a display device for being rendered in the display device,

wherein to automatically modify the one or more portions, the program instructions use a machine learning model that has been trained with training datasets associated with a plurality of one or more other users, the training datasets including a lexicon, a demographic characteristic and/or a somatic state of said one or more other users,

wherein responsive to receiving the subtitle data and values of the features from the group profile, the program instructions compute a corresponding array of values as input to said machine learning model in order to produce an output array that contains the augmented subtitle data.

14. The computer usable program product of claim 13 , further comprising:

program instructions to render the augmented subtitle data in the display device.

15. The computer usable program product of claim 13 , further comprising:

program instructions to present the media content by the display device in synchronization with the rendering of the augmented subtitle data.

16. The computer usable program product of claim 13 , wherein the computer usable code is stored in a computer readable storage device in a data processing system, and wherein the computer usable code is transferred over a network from a remote data processing system.

17. The computer usable program product of claim 13 , wherein the computer usable code is stored in a computer readable storage device in a server data processing system, and wherein the computer usable code is downloaded over a network to a remote data processing system for use in a computer readable storage device associated with the remote data processing system.

18. A computer system comprising one or more processors, one or more computer-readable memories, and one or more computer-readable storage devices, and program instructions stored on at least one of the one or more storage devices for execution by at least one of the one or more processors via at least one of the one or more memories, the stored program instructions comprising:

program instructions to identify, by one or more processors, one or more users expected to consume media content, the media content having associated subtitle data;

program instructions to receive a user profile associated with each of the one or more users;

program instructions to extract one or more features from each user profile, the one or more features representative of a characteristic of the user;

program instructions to create a group profile that is representative of features of the plurality of the one or more users said group profile being based upon the extracted features;

program instructions to receive the subtitle data associated with the media content;

program instructions to automatically modify one or more portions of the subtitle data based upon the group profile to generate augmented subtitle data; and

program instructions to send the augmented subtitle data to a display device for being rendered in the display device,

wherein to automatically modify the one or more portions the program instructions use a machine learning model that has been trained with training datasets associated with a plurality of one or more other users, the training datasets including a lexicon, a demographic characteristic and/or a somatic state of said one or more other users,

wherein responsive to receiving the subtitle data and values of the features from the group profile, the program instructions compute a corresponding array of values as input to said machine learning model in order to produce an output array that contains the augmented subtitle data.

19. The computer system of claim 18 , the stored program instructions further comprising:

program instructions to render the augmented subtitle data in the display device.

20. The computer system of claim 18 , the stored program instructions further comprising:

program instructions to present the media content by the display device in synchronization with the rendering of the augmented subtitle data.

Assignments (2)
CORRECTIVE ASSIGNMENT TO CORRECT THE ASSIGNEE NAME AND ADDRESS PREVIOUSLY RECORDED ON REEL 047305 FRAME 0709. ASSIGNOR(S) HEREBY CONFIRMS THE ASSIGNEE IBM AT 1 NEW ORCHARD RD, SHOULD BE CORRECTED AS INTERNATIONAL BUSINESS MACHINES CORPORATION, NEW ORCHARD RD. Recorded Nov 20, 2018
From: VAUGHN, GARFIELD; BENBOUBAKEUR, MONCEF; NARODICKA, JULIJA; BAUGHMAN, AARON K
To: INTERNATIONAL BUSINESS MACHINES CORPORATION
Reel/Frame 048147/0492 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Oct 25, 2018
From: VAUGHN, GARGIELD; BENBOUBAKEUR, MONCEF; NARODICKA, JULIJA; BAUGHMAN, AARON K
To: IBM
Reel/Frame 047305/0709 →
Continuity (1)
Related Publication 20200135225A1 · Apr 30, 2020
Cited By (3)
US 12,494,293 US 12,633,315 US 12,640,168