Key frame selection
View Patent ↗A system and method that processes video to extract a keyframe-based adequate visual representation. The method utilizes a hierarchical processing technique. The first stage in the hierarchy extracts a chromatic difference metric from a pair of video frames. An initial set of frames is chosen based on the chromatic metric and a threshold. A structural difference measurement is extracted from this initial set of frames. A second threshold is used to select key frames from the initial set. The first and second thresholds are user selectable. The output of this process is the visual representation. The method is extensible to any number of metrics and any number of levels.
1. A computerized method of identifying a key frame from a video, the method operating on a processor, comprising:
a) providing a reference frame;
b) providing, via a processor, a current frame different from the reference frame;
c) determining, via a processor, a chromatic difference measure between the reference frame and the current frame;
d) determining, via a processor, a structure difference measure between the reference frame and the current frame based, at least in part, on edges identified in each of the frames;
e) identifying, via a processor, the current frame as a key frame if the chromatic difference measure exceeds a chromatic threshold and the structure difference measure exceeds a structure threshold, otherwise selecting a new current frame; and
f) repeating c), d), and e) until a key frame is identified.
2. The method defined in claim 1 , additionally comprising setting the current frame to be the reference frame if a key frame is identified.
3. The method defined in claim 2 , additionally comprising repeating b)-f) for a new current frame until another key frame is identified or the end of the video is reached.
4. The method defined in claim 1 , wherein the new current frame is selected to be at a predetermined time interval after the current frame.
5. The method defined in claim 4 , wherein the predetermined time interval is user-selectable.
6. The method defined in claim 1 , wherein the value of the chromatic threshold and the value of the structure threshold are each user-selectable.
7. The method defined in claim 1 , wherein determining the structure difference measure is performed only if the chromatic difference measure exceeds the chromatic threshold.
8. A computerized method of identifying a key frame from a video having a plurality of frames, the method comprising:
a) providing a reference frame;
b) providing, via a processor, a current frame different from the reference frame;
c) determining, via a processor, a first difference measure between the reference frame and the current frame;
d) determining, via a processor, a second difference measure between the reference frame and the current frame based, at least in part, on edges identified in each of the frames;
e) identifying, via a processor, the current frame as a key frame if the first difference measure exceeds a first threshold and the second difference measure exceeds a second threshold, otherwise selecting a new current frame; and
f) repeating c), d), and e) until a key frame is identified.
9. The method defined in claim 8 , additionally comprising setting the current frame to be the reference frame if a key frame is identified.
10. The method defined in claim 8 , wherein the first difference measure is orthogonal to the second difference measure.
11. The method defined in claim 9 , additionally comprising the step of repeating b)-f) for a new current frame until another key frame is identified or the end of the video is reached.
12. The method defined in claim 11 , wherein the new current frame is selected to be at a predetermined time interval after the current frame.
13. The method defined in claim 8 , wherein the value of the first threshold and the value of the second threshold are each user-selectable.
14. The method defined in claim 8 , wherein determining the second difference measure is performed only if the first difference measure exceeds the first threshold.
15. The method defined in claim 8 , wherein the second difference measure is computationally more expensive than the first difference measure.
16. The method defined in claim 8 , wherein the second difference measure extracts more information than the first difference measure.
17. The method defined in claim 8 , additionally comprising determining a third difference measure between the reference frame and the current frame, and wherein the identifying identifies the current frame as the key frame if the third difference measure exceeds a third threshold, otherwise selecting a new current frame.
18. A computerized method of identifying a key frame from a video having a plurality of frames, the method comprising:
a) providing a reference frame;
b) providing, via a processor, a current frame different from the reference frame;
c) determining, via a processor, a structure difference measure between the reference frame and the current frame based, at least in part, on edges identified in each of the frames; and
d) identifying, via a processor, the current frame as a key frame if the structure difference measure exceeds a structure threshold, otherwise selecting a new current frame;
e) repeating c) and d) until a key frame is identified.
19. The method defined in claim 18 , additionally comprising setting the current frame to be the reference frame if a key frame is identified.
20. The method defined in claim 19 , additionally comprising repeating b) and e) for a new current frame until another key frame is identified or the end of the video is reached.
21. The method defined in claim 20 , wherein the new current frame is selected to be at a predetermined time interval after the current frame.
22. The method defined in claim 18 , wherein the value of the structure threshold is user selectable.
23. A computerized method of identifying a key frame from a video having a sequence of frames, the method comprising:
a) providing a reference frame;
b) providing, via a processor, a current frame different from the reference frame;
c) determining, via a processor, a chromatic difference measure between the reference frame and the current frame;
d) determining, via a processor, a structure difference measure between the reference frame and the current frame;
e) identifying, via a processor, the current frame as a key frame if the chromatic difference measure exceeds a chromatic threshold and the structure difference measure exceeds a structure threshold, otherwise selecting a new current frame; and
f) repeating c), d), and e) until a key frame is identified.
24. A computerized method of identifying a key frame from a video, comprising:
a) providing a reference frame;
b) providing a current frame different from the reference frame;
c) determining, via a processor, a chromatic difference measure between the reference frame and the current frame;
d) determining, via a processor, if the chromatic difference measure exceeds a chromatic threshold;
e) if the chromatic threshold is exceeded, identifying, via a processor, the current frame as a key frame candidate, otherwise selecting a new current frame and skipping f) and g);
f) determining, via a processor, a structure difference measure between the reference frame and the key frame candidate based, at least in part, on edges identified in each of the frames;
g) identifying, via a processor, the key frame candidate as a key frame if the structure difference measure exceeds a structure threshold, otherwise selecting a new current frame; and
h) repeating c) through g) until a key frame is identified.