Systems and methods for displaying subjects of a video portion of content
Systems and methods are described herein for displaying subjects of a portion of content. Media data of content is analyzed during playback, and a number of action signatures are identified. Each action signature is associated with a particular subject within the content. The action signature is stored, along with a timestamp corresponding to a playback position at which the action signature begins, in association with an identifier of the particular subject. Upon receiving a command, icons representing each of a number of action signatures at or near the current playback position are displayed. Upon receiving user selection of an icon corresponding to a particular signature, a portion of the content corresponding to the action signature is played back.
1. A method for displaying subjects of a portion of audio of content, the method comprising:
receiving at least one video frame from a series of video frames of a content item;
identifying, using audio processing, an audio signature in the content item, wherein the audio signature is associated with the at least one video frame;
determining, using video processing, whether one or more subjects are displayed in the at least one video frame;
based at least in part on the determining whether one or more subjects are displayed in the at least one video frame, associating a subject with the audio signature based at least in part on determining that a mouth of the subject in the at least one video frame is moving;
receiving an input command; and
based on the input command, generating for display an icon of the associated subject representing the audio signature.
2. The method of claim 1 wherein video processing is one of edge detection, facial recognition, or object recognition.
3. The method of claim 1 wherein associating a subject with the audio signature is based at least in part on audio characteristics of the audio signature.
4. The method of claim 3 wherein the audio characteristics of the audio signature include at least either audio frequency or speech pattern.
5. The method of claim 1 wherein:
determining, using video processing, whether one or more subjects are displayed in the at least one video frame indicates no subjects are displayed in the at least one video frame; and
wherein the audio signature is associated with a new subject.
6. The method of claim 1 wherein determining, using video processing, whether one or more subjects are displayed in the at least one video frame indicates no subjects are displayed in the at least one video frame, the method further comprising:
comparing the audio signature to a database of known subjects.
7. The method of claim 1 wherein the at least one video frame and audio of the audio signature occur simultaneously in the content item.
8. A system comprising:
control circuitry configured to:
receive at least one video frame from a series of video frames of a content item;
identify, using audio processing, an audio signature in the content item, wherein the audio signature is associated with the at least one video frame;
determine, using video processing, whether one or more subjects are displayed in the at least one video frame;
based at least in part on the determining whether one or more subjects are displayed in the at least one video frame, associate a subject with the audio signature based on at least in part on determining that a mouth of the subject in the at least one video frame is moving;
receive an input command; and
based on the input command, generate for display an icon of the associated subject representing the audio signature.
9. The system of claim 8 wherein video processing is one of edge detection, facial recognition, or object recognition.
10. The system of claim 8 wherein to associate a subject with the audio signature is based at least in part on audio characteristics of the audio signature.
11. The system of claim 10 wherein the audio characteristics of the audio signature include at least either audio frequency or speech pattern.
12. The system of claim 8 wherein:
to determine, using video processing, whether one or more subjects are displayed in the at least one video frame indicates no subjects are displayed in the at least one video frame; and
wherein the audio signature is associated with a new subject.
13. The system of claim 8 wherein to determine, using video processing, whether one or more subjects are displayed in the at least one video frame indicates no subjects are displayed in the at least one video frame, the control circuitry is further configured to:
compare the audio signature to a database of known subjects.
14. The system of claim 8 wherein the at least one video frame and audio of the audio signature occur simultaneously in the content item.
15. A non-transitory computer-readable medium having instructions encoded thereon that when executed by control circuitry causes the control circuitry to:
receive at least one video frame from a series of video frames of a content item;
identify, using audio processing, an audio signature in the content item, wherein the audio signature is associated with the at least one video frame;
determine, using video processing, whether one or more subjects are displayed in the at least one video frame;
based at least in part on the determining whether one or more subjects are displayed in the at least one video frame, associate a subject with the audio signature based at least in part on determining that a mouth of the subject in the at least one video frame is moving;
receive an input command; and
based on the input command, generate for display an icon of the associated subject representing the audio signature.
16. The non-transitory computer-readable medium of claim 15 wherein video processing is one of edge detection, facial recognition, or object recognition.
17. The non-transitory computer-readable medium of claim 15 wherein to associate a subject with the audio signature is based at least in part on audio characteristics of the audio signature.