IP Library Patent Application 17303118
Patent Application
App. No. 17/303,118

DYNAMICALLY ENHANCING A VIDEO BY AUTOMATICALLY GENERATING AND ADDING AN OVERLAY WINDOW

Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US None
App. No.
17/303,118
Abstract

A computer-implemented method for enhancing a video is provided. The method may include generating an annotation matrix comprising extracted video content associated with a video. The method may further include generating a viewer feedback matrix comprising extracted and aggregated viewer feedback, wherein the aggregated viewer feedback comprises a plurality of comments and viewer actions associated with the video, and wherein the plurality of comments appears as text that is located separate from a main window for playing the video. The method may further include generating an overlay matrix by merging the viewer feedback matrix and the annotation matrix. The method may further include generating at least one overlay window for overlaying in the main window of the video at the particular point in time during a playing of the video, wherein the at least one overlay window includes textual information generated from the aggregated viewer feedback.

Claims (57)

1 . A computer-implemented method for enhancing a video, comprising:

generating an annotation matrix comprising extracted video content associated with a video;

generating a viewer feedback matrix comprising extracted and aggregated viewer feedback from a plurality of viewers of the video, wherein the aggregated viewer feedback comprises a plurality of comments and viewer actions associated with the video, and wherein the aggregated viewer feedback comprising the plurality of comments appears as text that is located separate from a main window for playing the video;

generating an overlay matrix by merging the viewer feedback matrix and the annotation matrix, wherein the overlay matrix correlates the aggregated viewer feedback with at least one context in the video and corresponding time points; and

generating at least one overlay window for overlaying in the main window of the video at a particular point in time during a playing of the video, wherein the at least one overlay window includes textual information generated from the aggregated viewer feedback, and wherein overlaying in the main window of the video at the particular point in time further comprises displaying the textual information at the particular point in time of the at least one context of the video related to the textual information.

2 . The method of claim 1 , wherein generating the annotation matrix further comprises:

transcribing audio of the video using a speech-to-text algorithm and identifying keywords in the transcribed audio; and

identifying objects rendered at various points in time in the video using an image recognition algorithm and a machine learning model.

3 . The method of claim 1 , wherein generating the viewer feedback matrix further comprises:

generating a keyword map by mapping keywords from each comment from the plurality of comments to a matrix node with timeframe and context information;

parsing the viewer actions and mapping the viewer actions to the keywords; and

weighting the keywords to account for the viewer actions and storing the keyword map in the viewer feedback matrix.

4 . The method of claim 1 , wherein the extracted and aggregated viewer feedback further comprises a frequency of keywords used in the plurality of comments, comment upvotes, comment downvotes, comment referrers, and video scrubbing activity.

5 . The method of claim 1 , wherein generating the viewer feedback matrix further comprises:

prompting a viewer via a chatbot to provide more information in response to a viewer comment missing timeframe or context information.

6 . The method of claim 1 , wherein generating the overlay matrix further comprises:

feeding the annotation matrix and the viewer feedback matrix into a machine learning model to correlate keywords and contexts from both the annotation matrix and the viewer feedback matrix.

7 . The method of claim 1 , further comprising:

adding the generated at least one overlay window to the video and playing the video.

8 . A computer system for enhancing a video, comprising:

one or more processors, one or more computer-readable memories, one or more computer-readable tangible storage devices, and program instructions stored on at least one of the one or more storage devices for execution by at least one of the one or more processors via at least one of the one or more memories, wherein the computer system is capable of performing a method comprising:

generating an annotation matrix comprising extracted video content associated with a video;

generating a viewer feedback matrix comprising extracted and aggregated viewer feedback from a plurality of viewers of the video, wherein the aggregated viewer feedback comprises a plurality of comments and viewer actions associated with the video, and wherein the aggregated viewer feedback comprising the plurality of comments appears as text that is located separate from a main window for playing the video;

generating an overlay matrix by merging the viewer feedback matrix and the annotation matrix, wherein the overlay matrix correlates the aggregated viewer feedback with at least one context in the video and corresponding time points; and

generating at least one overlay window for overlaying in the main window of the video at a particular point in time during a playing of the video, wherein the at least one overlay window includes textual information generated from the aggregated viewer feedback, and wherein overlaying in the main window of the video at the particular point in time further comprises displaying the textual information at the particular point in time of the at least one context of the video related to the textual information.

9 . The computer system of claim 8 , wherein generating the annotation matrix further comprises:

transcribing audio of the video using a speech-to-text algorithm and identifying keywords in the transcribed audio; and

identifying objects rendered at various points in time in the video using an image recognition algorithm and a machine learning model.

10 . The computer system of claim 8 , wherein generating the viewer feedback matrix further comprises:

generating a keyword map by mapping keywords from each comment from the plurality of comments to a matrix node with timeframe and context information;

parsing the viewer actions and mapping the viewer actions to the keywords; and

weighting the keywords to account for the viewer actions and storing the keyword map in the viewer feedback matrix.

11 . The computer system of claim 8 , wherein the extracted and aggregated viewer feedback further comprises a frequency of keywords used in the plurality of comments, comment upvotes, comment downvotes, comment referrers, and video scrubbing activity.

12 . The computer system of claim 8 , wherein generating the viewer feedback matrix further comprises:

prompting a viewer via a chatbot to provide more information in response to a viewer comment missing timeframe or context information.

13 . The computer system of claim 8 , wherein generating the overlay matrix further comprises:

feeding the annotation matrix and the viewer feedback matrix into a machine learning model to correlate keywords and contexts from both the annotation matrix and the viewer feedback matrix.

14 . The computer system of claim 8 , further comprising:

adding the generated at least one overlay window to the video and playing the video.

15 . A computer program product for enhancing a video, comprising:

one or more tangible computer-readable storage devices and program instructions stored on at least one of the one or more tangible computer-readable storage devices, the program instructions executable by a processor, the program instructions comprising:

generating an annotation matrix comprising extracted video content associated with a video;

generating a viewer feedback matrix comprising extracted and aggregated viewer feedback from a plurality of viewers of the video, wherein the aggregated viewer feedback comprises a plurality of comments and viewer actions associated with the video, and wherein the aggregated viewer feedback comprising the plurality of comments appears as text that is located separate from a main window for playing the video;

generating an overlay matrix by merging the viewer feedback matrix and the annotation matrix, wherein the overlay matrix correlates the aggregated viewer feedback with at least one context in the video and corresponding time points; and

generating at least one overlay window for overlaying in the main window of the video at a particular point in time during a playing of the video, wherein the at least one overlay window includes textual information generated from the aggregated viewer feedback, and wherein overlaying in the main window of the video at the particular point in time further comprises displaying the textual information at the particular point in time of the at least one context of the video related to the textual information.

16 . The computer program product of claim 15 , wherein the program instructions to generate the annotation matrix further comprises:

transcribing audio of the video using a speech-to-text algorithm and identifying keywords in the transcribed audio; and

identifying objects rendered at various points in time in the video using an image recognition algorithm and a machine learning model.

17 . The computer program product of claim 15 , wherein generating the viewer feedback matrix further comprises:

generating a keyword map by mapping keywords from each comment from the plurality of comments to a matrix node with timeframe and context information;

parsing the viewer actions and mapping the viewer actions to the keywords; and

weighting the keywords to account for the viewer actions and storing the keyword map in the viewer feedback matrix.

18 . The computer program product of claim 15 , wherein the extracted and aggregated viewer feedback further comprises a frequency of keywords used in the plurality of comments, comment upvotes, comment downvotes, comment referrers, and video scrubbing activity.

19 . The computer program product of claim 15 , wherein generating the viewer feedback matrix further comprises:

prompting a viewer via a chatbot to provide more information in response to a viewer comment missing timeframe or context information.

20 . The computer program product of claim 15 , wherein generating the overlay matrix further comprises:

feeding the annotation matrix and the viewer feedback matrix into a machine learning model to correlate keywords and contexts from both the annotation matrix and the viewer feedback matrix.

Assignments (6)
CORRECTIVE ASSIGNMENT TO CORRECT THE CONVEYING PARTY FROM IBM RESEARCH AND INTELLECTUAL PROPERTY TO INTERNATIONAL BUSINESS MACHINED CORPORATION AND TO CORRECT THE RECEIVING PARTY FROM ZEPHYR BUYER L.P. TO ZEPHYR BUYER, L.P. PREVIOUSLY RECORDED AT REEL: 66795 FRAME: 858. ASSIGNOR(S) HEREBY CONFIRMS THE ASSIGNMENT. Recorded Mar 20, 2024
From: INTERNATIONAL BUSINESS MACHINES CORPORATION
To: ZEPHYR BUYER, L.P.
Reel/Frame 066838/0157 →
CORRECTIVE ASSIGNMENT TO CORRECT THE CONVEYING PARTY NAME PREVIOUSLY RECORDED AT REEL: 66796 FRAME: 188. ASSIGNOR(S) HEREBY CONFIRMS THE ASSIGNMENT. Recorded Mar 20, 2024
From: ZEPHYR BUYER, L.P.
To: THE WEATHER COMPANY, LLC
Reel/Frame 067188/0894 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 13, 2024
From: IBM RESEARCH AND INTELLECTUAL PROPERTY
To: ZEPHYR BUYER L.P.
Reel/Frame 066795/0858 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 13, 2024
From: ZEPHYR BUYER L.P.
To: THE WEATHER COMPANY, LLC
Reel/Frame 066796/0188 →
PATENT SECURITY AGREEMENT Recorded Jan 31, 2024
From: THE WEATHER COMPANY, LLC
To: MIDCAP FINANCIAL TRUST, AS COLLATERAL AGENT
Reel/Frame 066404/0122 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded May 20, 2021
From: LU, CINDY HAN; KOSTICK, MEGAN; BREWER, MICHAEL; TRAN, THAI QUOC
To: INTERNATIONAL BUSINESS MACHINES CORPORATION
Reel/Frame 056304/0298 →