Video importance rating based on compressed domain video features
View Patent ↗Systems and methods are provided for identifying important video frames and segments in multimedia content such as a segmented compressed domain video. Video frames in a segment are analyzed to determine intensity, contrast, and motion values for the frames and their segments. The values among frames and segments are compared to identify one or more video segments likely to be important to a viewer. The systems and methods may additionally be augmented with audio data other characteristics associated with the video frames and segments analyzed.
1. A method, comprising:
receiving, at a video frame analyzing device, a segmented video sequence;
identifying a first video frame in said segmented video sequence;
determining an intensity value for the first video frame;
determining a motion value associated with said first video frame;
determining that the motion value corresponds to a fast motion frame, and based on said determination lowering the motion value by a predetermined amount;
determining a contrast value for the first video frame; and
comparing, by the device, the first video frame to a second video frame in the same video sequence, said comparison based on the determined intensity value, the determined motion value, and the determined contrast value.
2. The method of claim 1 , wherein said first video frame is identified as a predictive frame (P-frame), and the motion value is determined by comparing the first video frame to a previous video frame in the segmented video sequence.
3. The method of claim 2 , wherein determining the motion value comprises identifying a plurality of macroblocks in the frame, computing a motion value for each of the plurality of macroblocks, and averaging the plurality of macroblock motion values.
4. The method of claim 1 , wherein said first video frame is identified as an intra frame (I-frame), and said motion value is set to zero.
5. The method of claim 1 , wherein determining the intensity value comprises determining a de-quantized discrete cosine transform coefficient for a plurality of macroblocks in the frame.
6. The method of claim 5 , wherein the intensity value for the first video frame is computed as the average intensity of each of the plurality of macroblocks.
7. The method of claim 1 , wherein determining the contrast value for the first video frame comprises determining a contrast value for a plurality of macroblocks in the frame.
8. The method of claim 1 , further comprising:
determining an audio rating associated with said first video frame, wherein said comparison is further based on the audio rating associated with said first video frame.
9. The method of claim 1 , further comprising:
determining that the contrast value is less than a predetermined threshold, and based on said determination lowering the intensity value by a predetermined amount.
10. A method for identifying a relevant video segment in a segmented video sequence, comprising:
receiving, at a video frame analyzing device, a segmented video sequence;
identifying a first video segment comprising a first plurality of video frames within said segmented video sequence;
determining an intensity value for the first video segment based on intensity values of at least one of said first plurality of video frames;
determining a contrast value for the first video segment based on contrast values of at least one of said first plurality of video frames;
determining a motion value for the first video segment based on motion values of at least one of said first plurality of video frames;
determining a segment relevance rating for the first video segment based on the intensity value, contrast value, and motion value for the first video segment;
comparing, by the device, the segment relevance rating for the first video segment to a corresponding segment relevance rating for a second video segment in the segmented video sequence; and
identifying the first video segment as a relevant video segment based on said comparison.
11. The method of claim 10 , wherein the intensity value and contrast value each are computed from intensity values and contrast values of said first plurality of video frames.
12. The method of claim 10 , wherein the first video segment comprises a plurality of P-frames and at least one I-frame, and determining an intensity value comprises computing the maximum intensity value for the at least one I-frame.
13. The method of claim 11 , wherein the first video segment comprises a plurality of P-frames and no I-frames, and determining an intensity value comprises computing the maximum intensity value among the plurality of P-frames.
14. The method of claim 10 , wherein the segment relevance rating for the first video segment is further based on an audio rating associated with at least one of the first plurality of video frames of the first video segment.
15. The method of claim 10 , wherein the segment relevance rating for the first video segment is not based on the length of the first video segment.
16. A device comprising:
a display screen;
a processor controlling at least some operations of the device;
a memory storing computer executable instructions that, when executed by the processor, cause the device to perform a method for displaying a relevant video segment, said method comprising steps of:
receiving a segmented video sequence;
identifying a first video segment comprising a first plurality of video frames within said segmented video sequence;
determining an intensity value for the first video segment based on intensity values of at least one of said first plurality of video frames;
determining a contrast value for the first video segment based on contrast values of at least one of said first plurality of video frames;
determining a motion value for the first video segment based on motion values of at least one of said first plurality of video frames;
determining a segment relevance rating for the first video segment based on the intensity value, contrast value, and motion value for the first video segment; and
displaying the first video segment to a user of said device based on said segment relevance rating.
17. The device of claim 16 , wherein the intensity value and contrast value each are computed from intensity values and contrast values of said first plurality of video frames.
18. The device of claim 16 , wherein the first video segment comprises a plurality of P-frames and at least one I-frame, and determining an intensity value comprises computing the maximum intensity value for the at least one I-frame.
19. The device of claim 16 , wherein the first video segment comprises a plurality of P-frames and no I-frames, and determining an intensity value comprises computing the maximum intensity value among the plurality of P-frames.
20. The device of claim 16 , wherein the segment relevance rating for the first video segment is further based on an audio rating associated with at least one of the first plurality of video frames of the first video segment.
21. The device of claim 16 , wherein the segment relevance rating for the first video segment is not based on the length of the first video segment.