Granted Patent B2

US 8,913,835 · App. 13/565,911 · Granted Dec 16, 2014

Abstract

Claims (31)

1. A method for identifying a set of key video frames from a video sequence including a time sequence of video frames, each video frame including an array of image pixels having pixel values, comprising:

a) selecting a set of video frames from the video sequence;

b) extracting a feature vector for each video frame in the set of video frames;

c) applying a group sparsity algorithm including a group sparse solver to represent the feature vector for a particular video frame as a group sparse combination of the feature vectors for the other video frames in the set of video frames, each feature vector for the other video frames in the group sparse combination having an associated weighting coefficient, wherein the weighting coefficients for feature vectors corresponding to other video frames that are most similar to the particular video frame are non-zero, and the weighting coefficients for feature vectors corresponding to other video frames that are most dissimilar from the particular video frame are zero;

d) analyzing the weighting coefficients to determine a video frame cluster of temporally-contiguous, similar video frames that includes the particular video frame;

e) repeating steps c)-d) for a plurality of particular video frames to provide a plurality of video frame clusters;

f) selecting a set of key video frames based on the video frame clusters; and

g) storing an indication of the selected key video frames in a processor-accessible memory;

wherein the method is performed, at least in part, using a data processor, and wherein the group sparse solver is invoked once for each group, to reduce computational complexity.

2. The method of claim 1 wherein the extraction of the feature vector for a video frame includes:

forming a frame vector including pixel values from the video frame;

defining a basis function set including a plurality of basis functions, each basis function being a vector having an identical length to the frame vector;

forming products between the frame vector and of the basis functions to determine corresponding feature coefficients;

forming the feature vector by collecting the feature coefficients for each basis function.

3. The method of claim 2 wherein the pixel values used in frame vector are luminance pixel values.

4. The method of claim 2 wherein the frame vector includes pixel values for a subset of the image pixels in the video frame selected according to a predefined sampling pattern.

5. The method of claim 2 wherein the basis functions are vectors of random numbers.

6. The method of claim 2 wherein the basis functions are Fourier transform basis functions, discrete cosine transform basis functions, or wavelet basis functions.

7. The method of claim 1 wherein the determined video frame cluster includes only the particular video frame and video frames that follow the particular video frame in the video sequence.

8. The method of claim 7 wherein the video frames included in the video frame cluster are determined by identifying the next video frame having an insignificant weighting coefficient that is smaller than a predefined threshold.

9. The method of claim 1 wherein the video frame cluster includes the temporally-contiguous video frames where the corresponding feature vectors have weighting coefficients that exceed a predefined threshold.

10. The method of claim 1 further including discarding any video-frame clusters including less than a minimum number of video frames or including more than a maximum number of video frames.

11. The method of claim 1 further including analyzing the video frame clusters to merge groups of video frame clusters into larger video frame clusters.

12. The method of claim 11 wherein the merged video frame clusters are constrained to be temporally-contiguous with each other.

13. The method of claim 11 wherein the video frame clusters to be merged are identified based on analyzing their similarity.

14. The method of claim 11 wherein the video frame clusters are merged using a bipartite graph partitioning algorithm.

15. The method of claim 11 further including specifying a target number of key video frames, and wherein the video frame clusters are merged until a remaining number of merged video frame clusters equals the specified target number of key video frames.

16. The method of claim 15 wherein the target number of key video frames is determined automatically responsive to a number of video frames in the set of video frames.

17. The method of claim 15 wherein the target number of key video frames is user specified.

18. The method of claim 1 wherein a central video frame nearest to the center of a largest temporally-contiguous series of video frames in the video frame cluster is selected to be the key video frame.

19. The method of claim 1 wherein the video frames within a video frame cluster are analyzed to determine corresponding image quality metric values, and wherein the key video frame is selected responsive to the determined image quality values.

Assignments (13)