Granted Patent B2

US 9,489,577 · App. 12/804,518 · Granted Nov 8, 2016

Abstract

Claims (31)

1. A method comprising:

receiving a stream of video content;

generating speech to text for the received video content;

generating passage level annotations from the generated text using natural language processing (NLP);

associating the passage level annotations with a timeline; and

associating imagery with the text to generate thumbnails at periodic time intervals resulting in a database of annotations to imagery and imagery to annotations.

2. The method of claim 1 further comprising using database of annotations to imagery and imagery to annotations to obtain interpretations of other digital images to improve the interpretation derived from the speech.

3. The method of claim 1 wherein the annotations comprise names, concepts, and/or logical substructure.

4. The method of claim 1 wherein associating imagery with the annotated text to generate thumbnails at periodic time intervals comprises applying a machine learning technique.

5. The method of claim 4 wherein the machine learning technique includes one of a neural networks approach or a classification and regression tree (CART) approach to classify an image into an annotation or to classify an annotation into an image.

6. The method of claim 4 wherein the machine learning technique is a principal component analysis (PCA) of facial imagery.

7. The method of claim 4 wherein associating imagery with the annotated text to generate thumbnails at periodic time intervals further comprises clustering the images.

8. The method of claim 1 further comprising generating a master catalog of images for selected annotations.

9. The method of claim 8 further comprising comparing videos to determine whether the videos contain speech.

10. The method of claim 9 wherein comparing videos comprises pixel comparison.

11. An apparatus comprising:

a local computing system linked to a network of interconnected computer systems, the local computing system comprising a processor, a memory and a storage device;

the memory comprising an operating system and a visual similarity process, the visual similarity process comprising:

receiving a stream of video content;

generating speech to text for the received video content;

generating passage level annotations from the generated text using natural language processing (NLP);

associating the passage level annotations with the text time aligned to result in text, annotations and a time stamp; and

associating imagery with the annotated text to generate thumbnails at periodic time intervals resulting in a database of annotations to imagery and imagery to annotations.

12. The apparatus of claim 11 wherein the annotations comprise names, concepts, and/or logical substructure.

13. The apparatus of claim 11 wherein associating imagery with the annotated text to generate thumbnails at periodic time intervals comprises applying a machine learning technique.

14. The apparatus of claim 13 wherein the machine learning technique includes one of a neural networks approach or a classification and regression tree (CART) approach to classify an image into an annotation or to classify an annotation into an image.

15. The apparatus of claim 13 wherein the machine learning technique is a principal component analysis (PCA) of facial imagery.

16. The apparatus of claim 11 wherein associating imagery with the annotated text to generate thumbnails at periodic time intervals further comprises clustering the images.

17. The apparatus of claim 11 wherein the visual similarity process further comprises generating a master catalog of images for selected annotations.

18. The apparatus of claim 17 wherein the visual similarity process further comprises comparing videos to determine whether the videos contain speech.

19. The apparatus of claim 18 wherein comparing videos comprises pixel comparison.

Assignments (6)