IP Library Granted Patent US 10,834,439
Granted Patent B2
US 10,834,439 · App. 16/067,036 · Granted Nov 10, 2020

Systems and methods for correcting errors in caption text

Inventors: Ajay Kumar Gupta (Andover, MA); Abhijit Satchidanand Savarkar (Andover, MA)
Assignee: Rovi Guides, Inc.
H04N21/23424G06F40/166G06F40/232G06F40/40H04N21/234336H04N21/4884
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,834,439
App. No.
16/067,036
Granted
Nov 10, 2020
Kind
B2
Abstract

Systems and methods are described to address shortcomings in conventional systems by correcting an erroneous term in on-screen caption text for a media asset. In some aspects, the systems and methods identify the erroneous term in a text segment of the on-screen caption text, and identify one or more video frames of the media asset corresponding to the text segment. The systems and methods further identify a contextual term related to the erroneous term from the one or more video frames. By accessing a knowledge graph, the systems and methods identify a candidate correction based on the contextual term and a portion of the text segment. Lastly, the systems and methods replaces the erroneous term with the candidate correction.

Claims (58)

1. A method for correcting an erroneous term in on-screen caption text of a media asset, comprising:

analyzing an audio stream of the media asset to determine a first text segment of the on-screen caption text;

identifying an erroneous term in the first text segment of the on-screen caption text;

extracting one or more video frames from a video stream of the media asset corresponding to the first text segment;

analyzing a first video frame of the one or more video frames to determine a contextual term associated with the erroneous term;

accessing a knowledge graph to identify a candidate correction for the erroneous term based on the contextual term and a portion of the first text segment;

replacing the erroneous term in the first text segment of the closed captioning text with the candidate correction;

identifying the erroneous term in a second text segment of the on-screen caption text;

analyzing a second video frame corresponding to the second text segment to determine a second contextual term associated with the erroneous term;

accessing the knowledge graph to identify an updated candidate correction based on the first contextual term, the second contextual term, the portion of the first text segment and a portion of the second text segment; and

replacing the erroneous term in the second text segment of the on-screen caption text with the updated candidate correction.

2. The method of claim 1 , wherein identifying the erroneous term in the first text segment further comprises performing natural language processing on the first text segment to compare the first text segment against a plurality of grammar rules.

3. The method of claim 1 , wherein the first text segment of the on-screen caption text is time-stamped, and wherein the first video frame is extracted at a position of the media asset corresponding to a position of the erroneous term in the time-stamped first text segment.

4. The method of claim 1 , wherein accessing the knowledge graph to identify the candidate correction based on the contextual term and the portion of the first text segment further comprises:

extracting a keyword from the portion of the first text segment;

searching in the knowledge graph for nodes corresponding to the contextual term and the keyword;

analyzing the nodes for properties associated with the contextual term and the keyword; and

determining at least one other node based on the properties associated with the contextual term and the keyword, wherein the at least one other node corresponds to the candidate correction.

5. The method of claim 1 , further comprising replacing the candidate correction in the first text segment with the updated candidate correction.

6. The method of claim 1 , wherein accessing the knowledge graph to identify the candidate correction for the erroneous term further comprises:

determining a plurality of potential corrections for the erroneous term from the knowledge graph;

assigning a weight to each potential correction of the plurality of potential corrections based on the determining; and

identifying a potential correction associated with a highest weight as the candidate correction.

7. The method of claim 6 , wherein a more recent potential correction of the plurality of potential corrections is assigned a higher weight.

8. The method of claim 6 , further comprising:

determining a phonetic similarity score between a potential candidate correction and the erroneous term based on a phonetic algorithm; and

assigning a higher weight to the potential candidate correction with a higher phonetic similarity score.

9. The method of claim 1 , wherein accessing the knowledge graph to identify the candidate correction based on the contextual term and the portion of the first text segment further comprises updating existing nodes of the knowledge graph.

10. A system for correcting an erroneous term in on-screen caption text of a media asset, comprising:

a memory storing a knowledge graph; and

control circuitry configured to:

analyze an audio stream of the media asset to determine a first text segment of the on-screen caption text;

identify an erroneous term in the first text segment of the on-screen caption text;

extract one or more video frames from a video stream of the media asset corresponding to the first text segment;

analyze a first video frame of the one or more video frames to determine a contextual term associated with the erroneous term;

access a knowledge graph to identify a candidate correction for the erroneous term based on the contextual term and a portion of the first text segment;

replace the erroneous term in the first text segment of the closed captioning text with the candidate correction;

identify the erroneous term in a second text segment of the on-screen caption text;

analyze a second video frame corresponding to the second text segment to determine a second contextual term associated with the erroneous term;

access the knowledge graph to identify an updated candidate correction based on the first contextual term, the second contextual term, the portion of the first text segment and a portion of the second text segment; and

replace the erroneous term in the second text segment of the on-screen caption text with the updated candidate correction.

11. The system of claim 10 , wherein the control circuitry is further configured to identify the erroneous term in the first text segment further comprises performing natural language by processing on the first text segment to compare the first text segment against a plurality of grammar rules.

12. The system of claim 10 , wherein the first text segment of the on-screen caption text is time-stamped, and wherein the first video frame is extracted at a position of the media asset corresponding to a position of the erroneous term in the time-stamped first text segment.

13. The system of claim 10 , wherein the control circuitry is further configured to access the knowledge graph to identify the candidate correction based on the contextual term and the portion of the first text segment by:

extracting a keyword from the portion of the first text segment;

searching in the knowledge graph for nodes corresponding to the contextual term and the keyword;

analyzing the nodes for properties associated with the contextual term and the keyword; and

determining at least one other node based on the properties associated with the contextual term and the keyword, wherein the at least one other node corresponds to the candidate correction.

14. The system of claim 10 , wherein the control circuitry is further configured to replace the candidate correction in the first text segment with the updated candidate correction.

15. The system of claim 10 , wherein the control circuitry is further configured to access the knowledge graph to identify the candidate correction for the erroneous term by:

determining a plurality of potential corrections for the erroneous term from the knowledge graph;

assigning a weight to each potential correction of the plurality of potential corrections based on the determining; and

identifying a potential correction associated with a highest weight as the candidate correction.

16. The system of claim 15 , wherein a more recent potential correction of the plurality of potential corrections is assigned a higher weight.

17. The system of claim 15 , wherein the control circuitry is further configured to:

determine a phonetic similarity score between a potential candidate correction and the erroneous term based on a phonetic algorithm; and

assign a higher weight to the potential candidate correction with a higher phonetic similarity score.

18. The system of claim 10 , wherein the control circuitry is further configured to access the knowledge graph to identify the candidate correction based on the contextual term and the portion of the first text segment by updating existing nodes of the knowledge graph.

Assignments (7)
CHANGE OF NAME Recorded Oct 2, 2024
From: ROVI GUIDES, INC.
To: ADEIA GUIDES INC.
Reel/Frame 069085/0755 →
RELEASE OF SECURITY INTEREST Recorded Jun 5, 2020
From: MORGAN STANLEY SENIOR FUNDING, INC.
To: ROVI SOLUTIONS CORPORATION; ROVI TECHNOLOGIES CORPORATION; ROVI GUIDES, INC.; TIVO SOLUTIONS, INC.; VEVEO, INC.
Reel/Frame 053481/0790 →
RELEASE OF SECURITY INTEREST Recorded Jun 5, 2020
From: HPS INVESTMENT PARTNERS, LLC
To: ROVI SOLUTIONS CORPORATION; ROVI TECHNOLOGIES CORPORATION; ROVI GUIDES, INC.; TIVO SOLUTIONS, INC.; VEVEO, INC.
Reel/Frame 053458/0749 →
SECURITY INTEREST Recorded Jun 1, 2020
From: ROVI SOLUTIONS CORPORATION; ROVI TECHNOLOGIES CORPORATION; ROVI GUIDES, INC.; TIVO SOLUTIONS INC.; VEVEO, INC.; INVENSAS CORPORATION; INVENSAS BONDING TECHNOLOGIES, INC.; TESSERA, INC.; TESSERA ADVANCED TECHNOLOGIES, INC.; DTS, INC.; PHORUS, INC.; IBIQUITY DIGITAL CORPORATION
To: BANK OF AMERICA, N.A.
Reel/Frame 053468/0001 →
PATENT SECURITY AGREEMENT Recorded Nov 25, 2019
From: ROVI SOLUTIONS CORPORATION; ROVI TECHNOLOGIES CORPORATION; ROVI GUIDES, INC.; TIVO SOLUTIONS, INC.; VEVEO, INC.
To: MORGAN STANLEY SENIOR FUNDING, INC., AS COLLATERAL AGENT
Reel/Frame 051110/0006 →
SECURITY INTEREST Recorded Nov 22, 2019
From: ROVI SOLUTIONS CORPORATION; ROVI TECHNOLOGIES CORPORATION; ROVI GUIDES, INC.; TIVO SOLUTIONS, INC.; VEVEO, INC.
To: HPS INVESTMENT PARTNERS, LLC, AS COLLATERAL AGENT
Reel/Frame 051143/0468 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jan 18, 2019
From: GUPTA, AJAY KUMAR; SAVARKAR, ABHIJIT SATCHIDANAND
To: ROVI GUIDES, INC.
Reel/Frame 048055/0809 →
Continuity (1)
Related Publication 20190215545A1 · Jul 11, 2019