IP Library Granted Patent US 9,542,976
Granted Patent B2
US 9,542,976 · App. 14/025,907 · Granted Jan 10, 2017

Synchronizing videos with frame-based metadata using video content

Inventors: Michael Andrew Sipe (Pittsburgh, PA); Michael Christian Nechyba (Pittsburgh, PA)
Assignee: Google Inc.
G11B27/10G11B27/11H04N7/26941H04N21/00H04N21/2343H04N21/2355H04N21/23418H04N21/234309H04N21/234345H04N21/440245H04N21/242
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 9,542,976
App. No.
14/025,907
Granted
Jan 10, 2017
Kind
B2
Abstract

A transformed video and a source video may be synchronized according to implementations disclosed herein to provide tag information to the device receiving the transformed version of the video. A synchronization signal may be computed on the source video and the transformed video using a statistic such as mean pixel intensity. The synchronization signals computed for the transformed video and source video may be compared to determine a transformed video reference point location for the requested tag information. The requested tag information may be provided to the device receiving the transformed version of the video.

Claims (114)

1. A method, comprising:

determining tag information for a plurality of entities in a video, wherein the tag information comprises an identity of each of the plurality of entities and a video entity reference location for each of the plurality of entities;

determining a first synchronization signal for the video using a first statistic and a second statistic, each of the first statistic and the second statistic selected independently from the group consisting of: a mean pixel intensity, a mean color intensity, a mean color tone, a number of pixels of a particular intensity, a number of pixels of a particular color, and a centroid of a highest pixel intensity for a frame;

providing a transformed version of the video;

determining a second synchronization signal for the transformed version of the video using the second statistic;

receiving a request for tag information for at least one of the plurality of entities in the transformed version of the video;

comparing the first synchronization signal to the second synchronization signal to determine a transformed video reference point location for the requested tag information; and

providing the requested tag information at the transformed video reference point location based on the comparison of the first synchronization signal to the second synchronization signal.

2. The method of claim 1 , wherein the tag information further comprises a position of an entity in a frame of the video.

3. The method of claim 1 , wherein the video entity reference location comprises a frame of the video.

4. The method of claim 1 , wherein an entity is selected from the group consisting of: an audio component and a video component.

5. The method of claim 4 , wherein the audio component is selected from the group consisting of: a soundtrack, a voice, a speech, and a sound effect.

6. The method of claim 4 , wherein the video component is selected from the group consisting of: a scene break, a geographic location, a face, a person, an object, a physical object, a text, and a landmark.

7. The method of claim 1 , wherein the first statistic is computed for at least a portion of a video frame.

8. The method of claim 1 , wherein the receiving the request for the tag information includes receiving a query related to a selected portion of a video frame in the transformed version of the video.

9. The method of claim 8 , wherein the receiving the request for the tag information further includes:

determining a corresponding selected portion of a video frame in the video; and

excluding tag information for an entity that is not within the selected portion of the video frame in the transformed version of the video from being provided.

10. A system, comprising:

a database for storing tag information for a plurality of entities in a video, wherein the tag information comprises an identity of each of the plurality of entities and a video entity reference location for each of the plurality of entities; and

a processor communicatively coupled to the database and configured to:

determine tag information for the plurality of entities in the video;

determine a first synchronization signal for the video using a first statistic and a second statistic, each of the first statistic and the second statistic selected independently from the group consisting of: a mean pixel intensity, a mean color intensity, a mean color tone, a number of pixels of a particular intensity, a number of pixels of a particular color, and a centroid of a highest pixel intensity for a frame;

provide a transformed version of the video;

determine a second synchronization signal for the transformed version of the video using the second statistic;

receive a request for tag information for at least one of the plurality of entities in the transformed version of the video;

compare the first synchronization signal to the second synchronization signal to determine a transformed video reference point location for the requested tag information; and

provide the requested tag information at the transformed video reference point location based on the comparison of the first synchronization signal to the second synchronization signal.

11. The system of claim 10 , wherein the tag information further comprises a position of an entity in a frame of the video.

12. The system of claim 10 , wherein the video entity reference location comprises a frame of the video.

13. The system of claim 10 , wherein an entity is selected from the group consisting of: an audio component and a video component.

14. The system of claim 13 , wherein the audio component is selected from the group consisting of: a soundtrack, a voice, a speech, and a sound effect.

15. The system of claim 13 , wherein the video component is selected from the group consisting of: a scene break, a geographic location, a face, a person, an object, a physical object, a text, and a landmark.

16. The system of claim 10 , wherein the first statistic is computed for at least a portion of a video frame.

17. The system of claim 10 , wherein the processor is configured to receive the request for the tag information by receiving a query related to a selected portion of a video frame in the transformed version of the video.

18. The system of claim 17 , wherein the processor is further configured to receive the request for the tag information by:

determining a corresponding selected portion of a video frame in the video; and

excluding tag information for an entity that is not within the selected portion of the video frame in the transformed version of the video from being provided.

19. A method, comprising:

determining tag information for a plurality of entities in a video, wherein the tag information comprises an identity of each of the plurality of entities and a video entity reference location for each of the plurality of entities;

determining a first synchronization signal for the video using a first statistic and a second statistic, each of the first statistic and the second statistic selected independently from the group consisting of: a mean pixel intensity, a mean color intensity, a mean color tone, a number of pixels of a particular intensity, a number of pixels of a particular color, and a centroid of a highest pixel intensity for a frame;

providing a transformed version of the video;

receiving a second synchronization signal for the transformed version of the video using the second statistic;

receiving a request for tag information for at least one of the plurality of entities in the transformed version of the video;

comparing the first synchronization signal to the second synchronization signal to determine a transformed video reference point location for the requested tag information; and

providing the requested tag information at the transformed video reference point location based on the comparison of the first synchronization signal to the second synchronization signal.

20. A system, comprising:

a database for storing tag information for a plurality of entities in a video, wherein the tag information comprises an identity of each of the plurality of entities and a video entity reference location for each of the plurality of entities; and

a processor communicatively coupled to the database and configured to:

determine tag information for the plurality of entities in the video;

determine a first synchronization signal for the video using a first statistic and a second statistic, each of the first statistic and the second statistic selected independently from the group consisting of: a mean pixel intensity, a mean color intensity, a mean color tone, a number of pixels of a particular intensity, a number of pixels of a particular color, and a centroid of a highest pixel intensity for a frame;

provide a transformed version of the video;

receive a second synchronization signal for the transformed version of the video using the second statistic;

receive a request for tag information for at least one of the plurality of entities in the transformed version of the video;

compare the first synchronization signal to the second synchronization signal to determine a transformed video reference point location for the requested tag information; and

provide the requested tag information at the transformed video reference point location based on the comparison of the first synchronization signal to the second synchronization signal.

21. A method, comprising:

receiving a first synchronization signal for a video that is based on a first statistic and a second statistic, each of the first statistic and the second statistic selected independently from the group consisting of: a mean pixel intensity, a mean color intensity, a mean color tone, a number of pixels of a particular intensity, a number of pixels of a particular color, and a centroid of a highest pixel intensity for a frame;

receiving a transformed version of the video;

determining a second synchronization signal for the transformed version of the video using the second statistic;

comparing the first synchronization signal to the second synchronization signal to determine a transformed video reference point location;

requesting tag information for at least one of a plurality of entities in the transformed version of the video based on the transformed video reference point location, wherein the tag information for the plurality of entities comprises an identity of each of the plurality of entities for the transformed video reference point location; and

receiving the tag information for the at least one of the plurality of entities for the transformed video reference point location.

22. The method of claim 21 , further comprising requesting the first synchronization signal.

23. A system, comprising:

a database for storing synchronization signal data; and

a processor communicatively coupled to the database and configured to:

receive a first synchronization signal for a video that is based on a first statistic and a second statistic, each of the first statistic and the second statistic selected independently from the group consisting of: a mean pixel intensity, a mean color intensity, a mean color tone, a number of pixels of a particular intensity, a number of pixels of a particular color, and a centroid of a highest pixel intensity for a frame;

receive a transformed version of the video;

determine a second synchronization signal for the transformed version of the video using the second statistic;

compare the first synchronization signal to the second synchronization signal to determine a transformed video reference point location;

request tag information for at least one of a plurality of entities in the transformed version of the video based on the transformed video reference point location, wherein the tag information for the plurality of entities comprises an identity of each of the plurality of entities for the transformed video reference point location; and

receive the tag information for the at least one of the plurality of entities for the transformed video reference point location.

24. The system of claim 23 , wherein the processor is further configured to request the first synchronization signal.

25. A method, comprising:

determining tag information for a plurality of entities in a video, wherein the tag information comprises an identity of each of the plurality of entities and a video entity reference location for each of the plurality of entities;

determining a first synchronization signal for the video using a statistic, the statistic selected independently from the group consisting of: a mean pixel intensity, a mean color intensity, a mean color tone, a number of pixels of a particular intensity, a number of pixels of a particular color, and a centroid of a highest pixel intensity for a frame;

providing a transformed version of the video;

determining a second synchronization signal for the transformed version of the video using the statistic;

receiving a request for tag information for at least one of the plurality of entities in the transformed version of the video, the request including a query related to a selected portion of a video frame in the transformed version of the video;

determining a corresponding selected portion of a video frame in the video;

comparing the first synchronization signal to the second synchronization signal to determine a transformed video reference point location for the requested tag information; and

providing the requested tag information at the transformed video reference point location based on the comparison of the first synchronization signal to the second synchronization signal, but excluding tag information for an entity that is not within the selected portion of the video frame in the transformed version of the video from being provided.

26. A system, comprising:

a database for storing tag information for a plurality of entities in a video, wherein the tag information comprises an identity of each of the plurality of entities and a video entity reference location for each of the plurality of entities; and

a processor communicatively coupled to the database and configured to:

determine tag information for the plurality of entities in the video;

determine a first synchronization signal for the video using a statistic, the statistic selected independently from the group consisting of: a mean pixel intensity, a mean color intensity, a mean color tone, a number of pixels of a particular intensity, a number of pixels of a particular color, and a centroid of a highest pixel intensity for a frame;

provide a transformed version of the video;

determine a second synchronization signal for the transformed version of the video using the statistic;

receive a request for tag information for at least one of the plurality of entities in the transformed version of the video, the request including a query related to a selected portion of a video frame in the transformed version of the video;

determine a corresponding selected portion of a video frame in the video;

compare the first synchronization signal to the second synchronization signal to determine a transformed video reference point location for the requested tag information; and

provide the requested tag information at the transformed video reference point location based on the comparison of the first synchronization signal to the second synchronization signal, but exclude tag information for an entity that is not within the selected portion of the video frame in the transformed version of the video from being provided.

27. A method, comprising:

determining tag information for a plurality of entities in a video, wherein the tag information comprises an identity of each of the plurality of entities and a video entity reference location for each of the plurality of entities;

determining a first synchronization signal for the video using a statistic, the statistic selected independently from the group consisting of: a mean pixel intensity, a mean color intensity, a mean color tone, a number of pixels of a particular intensity, a number of pixels of a particular color, and a centroid of a highest pixel intensity for a frame;

providing a transformed version of the video;

receiving a second synchronization signal for the transformed version of the video using the statistic;

receiving a request for tag information for at least one of the plurality of entities in the transformed version of the video, the request including a query related to a selected portion of a video frame in the transformed version of the video;

determining a corresponding selected portion of a video frame in the video;

comparing the first synchronization signal to the second synchronization signal to determine a transformed video reference point location for the requested tag information; and

providing the requested tag information at the transformed video reference point location based on the comparison of the first synchronization signal to the second synchronization signal, but excluding tag information for an entity that is not within the selected portion of the video frame in the transformed version of the video from being provided.

28. A system, comprising:

a database for storing tag information for a plurality of entities in a video, wherein the tag information comprises an identity of each of the plurality of entities and a video entity reference location for each of the plurality of entities; and

a processor communicatively coupled to the database and configured to:

determine tag information for the plurality of entities in the video;

determine a first synchronization signal for the video using a statistic, the statistic selected independently from the group consisting of: a mean pixel intensity, a mean color intensity, a mean color tone, a number of pixels of a particular intensity, a number of pixels of a particular color, and a centroid of a highest pixel intensity for a frame;

provide a transformed version of the video;

receive a second synchronization signal for the transformed version of the video using the statistic;

receive a request for tag information for at least one of the plurality of entities in the transformed version of the video, the request including a query related to a selected portion of a video frame in the transformed version of the video;

determine a corresponding selected portion of a video frame in the video;

compare the first synchronization signal to the second synchronization signal to determine a transformed video reference point location for the requested tag information; and

provide the requested tag information at the transformed video reference point location based on the comparison of the first synchronization signal to the second synchronization signal, but exclude tag information for an entity that is not within the selected portion of the video frame in the transformed version of the video from being provided.

Assignments (2)
CHANGE OF NAME Recorded Oct 2, 2017
From: GOOGLE INC.
To: GOOGLE LLC
Reel/Frame 044097/0658 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Sep 13, 2013
From: SIPE, MICHAEL ANDREW; NECHYBA, MICHAEL CHRISTIAN
To: GOOGLE INC.
Reel/Frame 031267/0734 →
Continuity (1)
Related Publication 20150078729A1 · Mar 19, 2015