IP Library Granted Patent US 7,612,832
Granted Patent B2
US 7,612,832 · App. 11/092,389 · Granted Nov 3, 2009

Method and system for video clip compression

Assignee: Microsoft Corporation
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 7,612,832
App. No.
11/092,389
Granted
Nov 3, 2009
Kind
B2
Abstract

In a method for compressing a video clip containing audio content and image content, an image and/or an audio portion of individual video frames of the video clip are analyzed. Next frame scores are calculated for the video frames. Each frame score is based on at least one image attribute of the image of the video frame, and/or an audio attribute of the audio portion of the video frame. Next, key frames are identified that have a frame score that exceeds a threshold frame score. Finally, a compressed video clip is formed in which the images of non-key frames are removed. A system for implementing the method is also disclosed.

Claims (53)

1. A method for compressing a video clip, implemented by a computer comprising a tangible storage medium, the method comprising:

analyzing at least one of an image and an audio portion of individual video frames of the video clip;

calculating frame scores for the individual video frames, each frame score based on at least one of an image attribute of the image of the video frame and an audio attribute of the audio portion of the video frame;

identifying key frames of the video frames having a frame score that exceeds a threshold frame score;

forming a compressed video clip in which the images of non-key video frames are removed, the compressed video clip comprising the images of the key frames, the audio portions of the key frames and the audio portions of the non-key video frames.

2. The method of claim 1 , wherein forming a compressed video clip in which the images of non-key video frames are removed includes synchronizing the images of the identified key frames with the audio portions of the key frames and the non-key video frames.

3. The method of claim 1 , wherein:

each video frame includes a time code period; and

the method further comprises filling in image gaps of the non-key video frames of the compressed video clip with the images of the key frames including extending the time code period for the images of the key frames.

4. The method of claim 1 , wherein:

each frame score is based on a contrast measurement of the image of the video frame, which indicates a relative intensity difference between neighboring pixels of the image of the video frame; and

calculating frame scores for the individual video frames comprises assigning higher frame scores to the video frames whose images have a higher relative intensity difference between neighboring pixels than the video frames whose images have a lower relative intensity difference between neighboring pixels.

5. The method of claim 4 , wherein each frame score is based on a change in pitch of the audio portion from the audio portion of a preceding video frame.

6. The method of claim 1 including modifying at least one of the images of the key frames.

7. A system for compressing a video clip comprising:

a video analyzer having an output of frame scores for video frames of the video clip, each frame score based on at least one of an image attribute of an image of the video frame and an audio attribute of an audio portion of the video frame;

a key frame selector having an output identifying key frames of the video frames having a frame score that exceeds a threshold frame score; and

a video compiler configured to form a compressed video clip in which the images of non-key frames of the video clip are removed, the compressed video clip comprising the images of the key frames, the audio portions of the key frames and the audio portions of the non-key video frames.

8. The system of claim 7 , wherein:

each frame score is based on contrast measurement of the image of the video frame, which indicates a relative intensity difference between neighboring pixels of the image of the video frame; and

the video analyzer assigns higher frame scores for the individual video frames whose images have a higher relative intensity difference between neighboring pixels than the video frames whose images have a lower relative intensity difference between neighboring pixels.

9. The system of claim 8 , wherein each frame score is based on a change in pitch of the audio portion from the audio portion of a preceding video frame.

10. A tangible computer storage medium including instructions executable by a microprocessor of a computing device for compressing a video clip, the instructions comprising:

instructions for analyzing at least one of an image and an audio portion of individual video frames of the video clip;

instructions for calculating frame scores for the individual video frames, each frame score based on at least one of an image attribute of the image of the video frame and an audio attribute of the audio portion of the video frame;

instructions for identifying key frames having a frame score that exceeds a threshold frame score; and

instructions for forming a compressed video clip comprising the images and audio portions of the key frames and the audio portions of the non-key frames, wherein the images of the non-key frames are not included in the compressed video clip.

11. The instructions of claim 10 , wherein:

each frame score is based on a contrast measurement of the image of the video frame, which indicates a relative intensity difference between neighboring pixels of the image of the video frame; and

instructions for calculating frame scores for the individual video frames comprise instructions for assigning higher frame scores to the video frames whose images have a higher relative intensity difference between neighboring pixels than the video frames whose images have a lower relative intensity difference between neighboring pixels.

12. The instructions of claim 11 , wherein each frame score is based on a change in pitch of the audio portion from the audio portion of a preceding video frame.

13. The method of claim 1 , further comprising filling in image gaps of the non-key video frames of the compressed video clip with the images of the key frames.

14. The method of claim 1 , wherein:

each frame score is based on a histogram of the image of the video frame, from which a dynamic range for the image is obtained; and

calculating frame scores for the individual video frames comprises assigning higher frame scores to the video frames whose images have a higher dynamic range than the video frames whose images have a lower dynamic range.

15. The method of claim 1 , wherein:

each frame score is based on a measurement of a magnitude of optical flow of the image of the video frame; and

calculating frame scores for the individual video frames comprises assigning higher frame scores to the video frames whose images have a lower magnitude of optical flow than the video frames whose images have a higher magnitude of optical flow.

16. The system of claim 7 , wherein the video compiler is further configured to fill in image gaps of the non-key video frames of the compressed video clip with the images of the key frames.

17. The system of claim 7 , wherein the video compiler is further configured to extend a time code period for the images of the key frames into the non-key video frames of the compressed video clip.

18. The method of claim 7 , wherein:

each frame score is based on a histogram of the image of the video frame, from which a dynamic range for the image is obtained; and

the video analyzer assigns higher frame scores for the individual video frames whose images have a higher dynamic range than the video frames whose images have a lower dynamic range.

19. The method of claim 7 , wherein:

each frame score is based on a measurement of a magnitude of optical flow of the image of the video frame; and

the video analyzer assigns higher frame scores to the video frames whose images have a lower magnitude of optical flow than the video frames whose images have a higher magnitude of optical flow.

20. The instructions of claim 10 , further comprising instructions for filling in image gaps of the non-key video frames of the compressed video clip with the images of the key frames.

21. The instructions of claim 10 , wherein:

each frame is based on a histogram of the image of the video frame, from which a dynamic range for the image is obtained; and

instructions for calculating frame scores for the individual video frames comprise assigning higher frame scores to the video frames whose images have a higher dynamic range than the video frames whose images have a lower dynamic range.

22. The instructions of claim 10 , wherein:

each frame score is based on a measurement of a magnitude of optical flow of the image of the video frame; and

instructions for calculating frame scores for the individual video frames comprise assigning higher frame scores to the video frames whose images have a lower magnitude of optical flow than the video frames whose images have a higher magnitude of optical flow.

Assignments (2)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Dec 9, 2014
From: MICROSOFT CORPORATION
To: MICROSOFT TECHNOLOGY LICENSING, LLC
Reel/Frame 034543/0001 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded May 13, 2005
From: ZHANG, ZHENGYOU; LIU, ZICHENG
To: MICROSOFT CORPORATION
Reel/Frame 016013/0391 →
Continuity (1)
Related Publication 20060228029A1 · Oct 12, 2006