IP Library › Granted Patent US 9,684,644
Granted Patent B2
US 9,684,644 · App. 13/341,131 · Granted Jun 20, 2017

Annotating video intervals

Inventors: Mihai Badoiu (New York, NY); Shanmugavelayutham Muthukrishnan (New York, NY)
Assignee: Google Inc.
G06F17/241G06F17/3082G06F17/30864G11B27/105G11B27/34G06F17/30412G06F17/30525
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 9,684,644
App. No.
13/341,131
Granted
Jun 20, 2017
Kind
B2
Abstract

A system and method is provided for managing and displaying annotations in online hosted videos. To determine related annotations associated with a video, a plurality of annotations for one or more intervals of the video are collected and organized into a plurality of groups. Each group includes annotations for intervals of the video that are similar to each other. A group having related annotations is identified, and an annotated clip of the video is formed based upon the intervals in the group. This process can be used to determine any number of groups in a given video having related annotations, forming the respective annotated intervals within a single video, and can also identify and organize annotated intervals within a large number of different videos.

Claims (57)

1. A method for managing annotations associated with a video, the method comprising:

receiving from viewers of a video, a plurality of annotations for each of a plurality of intervals of the video, each interval having a start time and an end time, a content of each annotation of a plurality of the annotations including a plurality of words;

forming a plurality of annotation groups by clustering the annotations based on the start times and the end times of the intervals;

determining a first annotation group of the plurality of annotations groups having related annotations based at least in part on a comparison of the content of each annotation to determine that the related annotations have at least one identical or similar word;

forming a first annotated clip of the video based upon the intervals in the first annotation group; and

forming a synoptic annotation for the first annotated clip of the video based on the related annotations in the first annotation group, the synoptic annotation comprising one or more words selected from the related annotations in the first annotation group, the one or more words summarizing the content of the related annotations in the first annotation group.

2. The method of claim 1 , further comprising:

receiving a search query including one or more search terms;

responsive to receiving the search query, determining that the synoptic annotation matches the one or more search terms; and

displaying the identified synoptic annotation and an identifier of the video associated with the identified synoptic annotation.

3. The method of claim 1 , wherein forming the plurality of annotation groups comprises clustering the annotations based on a timestamp associated with each annotation.

4. The method of claim 1 , wherein forming the plurality of annotation groups comprises:

identifying a plurality of scenes in the video; and

associating each annotation group with one or more scenes.

5. The method of claim 1 , wherein forming the plurality of annotation groups comprises:

identifying a plurality of features in the video; and

associating each annotation group with a set of one or more features in the video.

6. The method of claim 1 , further comprising:

assigning a weight to each annotation based on whether the annotation was provided by a unique user; and

determining the first annotation group having related annotations based on the assigned weights of the annotations.

7. The method of claim 1 , wherein forming the plurality of annotation groups comprises determining a predetermined number of annotated clips permitted based on a number of indications of annotated clips being visually distinguished on a timeline corresponding to the video.

8. The method of claim 1 , wherein forming the plurality of annotation groups comprises determining an amount of time between the intervals with which the annotations are associated.

9. The method of claim 1 , further comprising:

displaying an indication of the first annotated clip on a timeline associated with the video.

10. The method of claim 1 , further comprising:

determining a second annotation group having related annotations; and

forming a second annotated clip based on the intervals in the second annotation group; and

ranking the first annotated clip and the second annotated clip based on the annotations in each annotation group.

11. A system for managing annotations associated with a video comprising:

a computer processor; and

a non-transitory computer-readable storage medium storing computer program modules configured to execute on the computer processor, the computer program modules comprising:

an extraction module, adapted to receive from viewers of a video, a plurality of annotations for each of a plurality of intervals of the video, each interval having a start time and an end time, a content of each annotation of a plurality of the annotations including a plurality of words;

a grouping module, adapted to form a plurality of annotation groups by clustering the annotations based on the start times and the end times of the intervals;

an annotation determination module, adapted to:

determine a first annotation group of the plurality of annotations groups having related annotations based at least in part on a comparison of the content of each annotation to determine that the related annotations have at least one identical or similar word; and

form an annotated clip of the video based upon the intervals in the first annotation group; and

a synoptic annotation module, adapted to form a synoptic annotation for the annotated clip of the video based on the related annotations in the first annotation group, the synoptic annotation comprising one or more words selected from the related annotations in the first annotation group, the one or more words summarizing the content of the related annotations in the first annotation group.

12. The system of claim 11 , wherein forming the plurality of annotation groups comprises clustering the annotations based on a timestamp associated with each annotation.

13. The system of claim 11 , wherein forming the plurality of annotation groups comprises:

identifying a plurality of scenes in the video; and

associating each annotation group with one or more scenes.

14. The system of claim 11 , wherein forming the plurality of annotation groups comprises:

identifying a plurality of features in the video; and

associating each annotation group with a set of one or more features in the video.

15. A computer program product having a non-transitory computer readable storage medium and including computer executable code for managing annotations associated with a video, the code adapted to perform steps comprising:

receiving from viewers of a video, a plurality of annotations for each of a plurality of intervals of the video, each interval having a start time and an end time, a content of each annotation of a plurality of the annotations including a plurality of words;

forming a plurality of annotation groups by clustering the annotations based on the start times and the end times of the intervals;

determining a first annotation group of the plurality of annotations groups having related annotations based at least in part on a comparison of the content of each annotation, the related annotations having at least one identical or similar word;

forming a first annotated clip of the video based upon the intervals in the first annotation group; and

forming a synoptic annotation for the first annotated clip of the video based on the related annotations in the first annotation group, the synoptic annotation comprising one or more words selected from the related annotations in the first annotation group, the one or more words summarizing the content of the related annotations in the first annotation group.

16. The computer program product of claim 15 , wherein forming the plurality of groups comprises clustering the annotations based on a timestamp associated with each annotation.

17. The computer program product of claim 15 , wherein forming the plurality of groups comprises:

identifying a plurality of scenes in the video; and

associating each group with one or more scenes.

18. The computer program product of claim 15 , wherein forming the plurality of groups comprises:

identifying a plurality of features in the video; and

associating each group with a set of one or more features in the video.

Assignments (2)
CHANGE OF NAME Recorded Oct 2, 2017
From: GOOGLE INC.
To: GOOGLE LLC
Reel/Frame 044097/0658 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded May 18, 2017
From: BADOIU, MIHAI; MUTHUKRISHNAN, SHANMUGAVELAYUTHAM
To: GOOGLE INC.
Reel/Frame 042431/0091 →
Continuity (2)
Continuation 12033817 · Feb 19, 2008
Related Publication 20120102387A1 · Apr 26, 2012