IP Library › Granted Patent US 11,122,344
Granted Patent B2
US 11,122,344 · App. 17/022,796 · Granted Sep 14, 2021

System and method for synchronizing metadata with audiovisual content

Inventors: Alan David Young (North Bethesda, MD); Dimitri Felixovich Tarassenko (Peachtree Corners, GA); Roger James Franklin (Dawsonville, GA); Malik Khan (Bethesda, MD)
Assignee: LTN GLOBAL COMMUNICATIONS, INC.
H04N21/8547H04N21/233H04N21/235H04N21/23418H04N21/23614H04N21/242H04N21/26603H04N21/8133
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,122,344
App. No.
17/022,796
Granted
Sep 14, 2021
Kind
B2
Abstract

An example method comprises receiving, at a first digital device, video data, scanning video content of the video data for visual transitions within the video content between consecutive frames of the video data, each transition indicating significant visual transitions relative to other frames of the video data, timestamping each visual transition and create a first set of temporal video fingerprints, identifying items of metadata to be associated with the video data, identifying a location within the video data using the temporal video fingerprints for the identified items of metadata, generating a metadata index identifying each item of metadata and a location for each item of metadata relative to the video data using at least one of the temporal video fingerprints, and transmitting, at the first digital device, the video data, the first set of temporal video fingerprints, and the metadata index to a different digital device.

Claims (44)

1. A method used for transmitting a video stream that contains video content from one or more source nodes to one or more destination nodes, wherein the video content comprises consecutive frames, the method comprising:

receiving the video stream at the one or more source nodes;

scanning the video content for visual transitions within the video content between the consecutive frames, each transition indicating significant visual transitions relative to other frames of the video stream;

retrieving a different time for each visual transition to timestamp each visual transition and create a set of temporal video fingerprints, each of the set of temporal video fingerprints being relative to a different visual transition and retrieved time;

identifying items of metadata to be associated with specific frames of video within the video stream at the one or more source nodes;

identifying a location of a frame within the video stream using the temporal video fingerprints for the identified items of metadata at the one or more source nodes; generating a metadata index identifying each item of metadata and a location for each item of metadata relative to a frame in the video stream using at least one of the temporal video fingerprints;

transmitting the video stream from the one or more source nodes to the one or more destination nodes in-band; and

transmitting the set of temporal video fingerprints and the metadata index from the one or more source nodes to the one or more destination nodes out-of-band.

2. The methods of claim 1 , wherein the video stream and the set of temporal video fingerprints and the metadata index are either transmitted over the same channel or different channels.

3. The methods of claim 1 , wherein the timestamp of each visual transition is used to determine a delay at the one or more destination nodes.

4. The methods of claim 3 , wherein the delay is used to synchronize items of metadata associated with specific frames at the one or more source nodes with items of metadata associated with the same specific frames at the one or more destination nodes.

5. A method used for transmitting a video stream that contains video content from one or more source nodes to one or more destination nodes, wherein the video content comprises consecutive frames, the method comprising:

receiving the video stream at the one or more source nodes;

identifying one or more first metadata at the one or more source nodes;

associating the identified one or more first metadata with one or more specific frames at one or more source nodes;

associating one or more second metadata with the one or more specific frames, wherein a second metadata conveys a timing information relating to a frame associated with a first identified metadata;

transmitting the video stream, the identified first metadata and the second metadata from the one or more source nodes to one or more destination nodes;

receiving the video stream, the first identified metadata and the second metadata at the one or more destination nodes;

determining a delay at the at the one or more destination nodes based the timing information conveyed by the second metadata; and

based on the delay, modifying association of the first identified metadata with the one or more frames of the video stream at the one or more destination nodes.

6. The method of claim 5 , wherein the delay is determined based on a temporal video fingerprint that references the second metadata.

7. The method of claim 5 , wherein the delay is determined based on a marker that contains the second metadata.

8. The method of claim 7 , wherein the marker comprises a watermark or a SCTE mark.

9. The method of claim 5 , wherein the timing information is derived from a GPS clock at a source node.

10. The method of claim 5 , wherein the timing information is compared with a time reference at a destination node to determine the delay.

11. The method of claim 5 , wherein the video stream is transmitted in-band and the identified first metadata is transmitted out-of-band.

12. The methods of claim 11 , wherein the video stream and the identified first metadata are either transmitted over the same channel or different channels.

13. The methods of claim 5 , wherein the delay is used to synchronize metadata associated with specific frames at the one or more source nodes with metadata associated with the same specific frames at the one or more destination nodes.

14. A method used for communicating a video stream that contains video content, wherein the video content comprises frames associated with metadata, the method comprising:

receiving the video stream from a at a node configured to:

scan the video content for transitions between consecutive frames, each transition indicating significant visual transition relative to other frames of the video data; scan the video stream for identified metadata associated with one or more specific frames; and

modify the association of the identified metadata to the same one or more specific frames based on significant visual transitions, wherein the modification synchronizes metadata associated with specific frames at one or more source nodes with metadata associated with the same specific frames at one or more destination nodes.

15. The method of claim 14 , wherein scanning the video content for visual transitions of significance within the video content comprises calculating histograms STREAM=(h 0 , h 1 . . . h n ) for every chroma/luma/color channel where n is the number of possible pixel brightness levels for the particular channel and h i is the count of pixels with value i in each frame of video, calculating a sum of absolute differences (SAD) between histogram vectors of every pair of adjacent frames to generate resulting values, and comparing the resulting values to a threshold to identify at least one visual transition of significance within the video content.

16. A method used for communicating a video stream that contains video content, wherein the video content comprises frames associated with metadata, the method comprising:

receiving the video stream at a node, which is configured to:

scan the video stream for a first set of metadata associated with one or more frames;

scan the video stream for a second set of metadata associated with the one or more frames that are associated with the first set of metadata; and

modify the association of the second set of metadata to the same one or more frames as determined by association of the first set of metadata, wherein the modification synchronizes metadata associated with specific frames at one or more source nodes with metadata associated with the same specific frames at one or more destination nodes.

17. The method of claim 16 , wherein a SCTE marker contains a first metadata in the first set of metadata and a watermark contains a second metadata in the second set of metadata.

18. A method used for communicating a video stream that contain video content, wherein the video content comprises frames associated with metadata, the method comprising:

receiving the video stream at a node, which is configured to:

scan the video stream for a first set of metadata associated with one or more frames; and

insert a second set of metadata associated to the same one or more frames as determined by association of the first set of metadata, wherein the insertion synchronizes metadata associated with specific frames at one or more source nodes with metadata associated with the same specific frames at one or more destination nodes.

19. The methods of claim 18 , wherein a SCTE marker contains a first metadata in the first set of metadata and a watermark contains a second metadata in the second set of metadata.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Sep 20, 2020
From: YOUNG, ALAN DAVID; TARASSENKO, DIMITRI FELIXOVICH; FRANKLIN, ROGER JAMES; KHAN, MALIK
To: LTN GLOBAL COMMUNICATIONS, INC.
Reel/Frame 053824/0210 →
Continuity (5)
Continuation In Part 16546240 · Aug 20, 2019
Continuation 15900665 · Feb 20, 2018
Provisional Application 62568975 · Oct 6, 2017
Provisional Application 62605526 · Aug 17, 2017
Related Publication 20210084389A1 · Mar 18, 2021