IP Library › Granted Patent US 11,653,071
Granted Patent B2
US 11,653,071 · App. 17/483,218 · Granted May 16, 2023

Responsive video content alteration

Inventors: Jacob Thomas Covell (New York, NY); Thomas Jefferson Sandridge (Tampa, FL); Alan Chung (Hopewell Junction, NY); Jeremy R. Fox (Georgetown, TX); Sarbajit K. Rakshit (Kolkata, IN)
Assignee: International Business Machines Corporation
H04N21/8146G06N20/00G06T13/205G06T13/40H04N21/44008H04N21/44213H04N21/4532
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,653,071
App. No.
17/483,218
Granted
May 16, 2023
Kind
B2
Abstract

A first user input is detected from a client device. The first user input is directed at a video content that includes a set of one or more topics, the first user input is from a viewer of the video content. A set of one or more frames in the video content is analyzed based on the first user input. A first topic in the video content is identified based on the set of frames and based on the viewer. The video content, related to the first topic of the set of topics, is altered based on the set of frames and based on the viewer.

Claims (58)

1. A method comprising:

detecting, from a client device, a first user input, the first user input directed at a video content that includes a set of one or more topics, the first user input being received from a viewer of the video content;

analyzing, based on the first user input, a set of one or more frames in the video content;

identifying, based on the set of frames and based on a determined knowledge of the viewer, a first topic in the video content; and

altering, based on the set of frames and based on the determined knowledge, the video content related to the first topic of the set of topics by removing a frame of the set of frames.

2. The method of claim 1 , wherein the altering the video content includes:

generating a virtualized avatar;

compositing an updated version of the video content that includes the virtualized avatar;

determining a first subset of frames of the set of frames that is related to the first topic of the set of topics; and

creating, based on the first subset of frames and based on the first topic, one or more gestures of the virtualized avatar, wherein the gestures are related to a portion of the first subset of frames that visually depict the first topic, and wherein:

a portion comprising the frame visually depicts the first topic; and

the gestures include the virtualized avatar writing the first topic.

3. The method of claim 2 , wherein the generating the virtualized avatar is based on a machine learning technique.

4. The method of claim 3 , wherein the machine learning technique includes a generative adversarial network.

5. The method of claim 2 , wherein the video content also includes an audio narration, and wherein the generating the virtualized avatar includes manipulating a mouth of the virtualized avatar based on the audio narration.

6. The method of claim 1 , wherein the analyzing the set of frames includes performing one or more image analysis operations on the set of frames.

7. The method of claim 6 , wherein the image analysis operations include a machine learning technique.

8. The method of claim 1 , wherein the altering includes inserting additional frames in the video content, wherein additional frames are inserted in the video content near the set of frames that include the first topic.

9. The method of claim 1 , wherein the altering comprises:

determining a first number of frames of the set of frames that corresponds to the first topic;

removing the first set of frames from the video content;

generating a new set of frames that include a condensed explanation of the first topic, wherein the new set of frames is less than the first set of frames.

10. The method of claim 1 , wherein the altering is further based on a user profile of the viewer.

11. The method of claim 10 , wherein the user profile includes a viewing history of the viewer.

12. The method of claim 10 , wherein the user profile includes a proficiency level of various topics of the viewer.

13. The method of claim 1 , wherein the method further comprises:

receiving, from the viewer, a request directed at the video content, and wherein the altering the video content is based on the request.

14. A system, the system comprising:

a memory, the memory containing one or more instructions; and

a processor, the processor communicatively coupled to the memory, the processor, in response to reading the one or more instructions, configured to:

detect, from a client device, a first user input, the first user input directed at a video content that includes a set of one or more topics, the first user input from a viewer of the video content;

analyze, based on the first user input, a set of one or more frames in the video content;

identify, based on the set of frames and based on a determined knowledge of the viewer, a first topic in the video content; and

alter, based on the set of frames and based on the determined knowledge, the video content related to the first topic of the set of topics by removing a frame from the set of frames.

15. The system of claim 14 , wherein the video content is altered by:

generating a virtualized avatar;

compositing an updated version of the video content that includes the virtualized avatar;

determining a first subset of frames of the set of frames that is related to the first topic of the set of topics; and

creating, based on the first subset of frames and based on the first topic, one or more gestures of the virtualized avatar, wherein the gestures are related to a portion of the first subset of frames that visually depict the first topic, and wherein:

a portion comprising the frame visually depicts the first topic; and

the gestures include the virtualized avatar writing the first topic.

16. A computer program product, the computer program product comprising:

one or more computer readable storage media; and

program instructions collectively stored on the one or more computer readable storage media, the program instructions configured to:

detect, from a client device, a first user input, the first user input directed at a video content that includes a set of one or more topics, the first user input from a viewer of the video content;

analyze, based on the first user input, a set of one or more frames in the video content;

identify, based on the set of frames and based on a determined knowledge of the viewer, a first topic in the video content; and

alter, based on the set of frames and based on the determined knowledge, the video content related to the first topic of the set of topics by removing a frame from the set of frames.

17. The computer program product of claim 16 , wherein the video content is altered by:

generating a virtualized avatar;

compositing an updated version of the video content that includes the virtualized avatar;

determining a first subset of frames of the set of frames that is related to the first topic of the set of topics; and

creating, based on the first subset of frames and based on the first topic, one or more gestures of the virtualized avatar, wherein the gestures are related to a portion of the first subset of frames that visually depict the first topic, and wherein:

a portion comprising the frame visually depicts the first topic; and

the gestures include the virtualized avatar writing the first topic.

18. The computer program product of claim 17 , wherein the video content also includes an audio narration, and wherein the generating the virtualized avatar includes manipulating a mouth of the virtualized avatar based on the audio narration.

19. The computer program product of claim 17 , wherein analyzing the set of frames includes performing one or more image analysis operations on the set of frames using a machine learning technique.

20. The computer program product of claim 17 , wherein the altering includes inserting additional frames in the video content.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Sep 23, 2021
From: COVELL, JACOB THOMAS; SANDRIDGE, THOMAS JEFFERSON; CHUNG, ALAN; FOX, JEREMY R.; RAKSHIT, SARBAJIT K.
To: INTERNATIONAL BUSINESS MACHINES CORPORATION
Reel/Frame 057580/0781 →
Continuity (1)
Related Publication 20230091912A1 · Mar 23, 2023