Method for determining key video frames
View Patent ↗A method for determining key video frames for a digital video having a time sequence of video frames, comprising using a processor to perform the steps of: analyzing the digital video to determine an importance value as a function of time; forming a warped time representation by temporal relocation of the video frames responsive to the importance value as a function of time; subdividing the warped time digital video sequence representation into a set of equal intervals; selecting a key video frame for each interval by analyzing the video frames within each interval; and storing an indication of the key video frames in a processor-accessible memory.
1. A method comprising:
determining, by a computing device, an importance value for each of a plurality of first video intervals comprising a video frame, wherein a video comprises the plurality of first video intervals in a temporal order;
forming, by the computing device, a warped time representation of the video by modifying a duration of at least one of the first video intervals based at least in part on the importance values;
subdividing, by the computing device, the warped time representation of the video into second video intervals;
selecting, by the computing device, a key video frame for each second video interval; and
storing, by the computing device, an indication of the key video frame for each second video interval.
2. The method of claim 1 , wherein determining the importance value comprises detecting global motion.
3. The method of claim 1 , wherein determining the importance value comprises detecting local motion.
4. The method of claim 1 , wherein determining the importance value comprises classifying the video frame into at least one of a plurality of video frame classifications.
5. The method of claim 4 , wherein the plurality of video frame classifications include zoom, fast pan, interesting, or inactive.
6. The method of claim 4 , wherein each of the plurality of video frame classifications is associated with an importance value.
7. The method of claim 1 , wherein selecting the key video frames comprises determining a selection score for each of a plurality of second video interval frames within each second video interval.
8. The method of claim 7 , wherein determining the selection scores is based at least part on global motion and local motion.
9. The method of claim 1 , wherein selecting the key video frames comprises selecting an independently encoded video frame as the key video frame within each second video interval.
10. The method of claim 1 , wherein storing the indication of the key video frames comprises extracting the key video frames as separate files.
11. The method of claim 1 , wherein storing the indication of the key video frames comprises storing a list of key video frames as metadata associated with the video.
12. The method of claim 1 , wherein storing the indication of the key video frames comprises storing thumbnail images for the key video frames as metadata associated with the video.
13. The method of claim 1 , wherein forming the warped time representation of the video comprises assigning a shorter temporal extent to video frames having importance values indicating lower importance.
14. The method of claim 1 , wherein forming the warped time representation of the video comprises assigning a longer temporal extent to video frames having importance values indicating higher importance.
15. The method of claim 1 , wherein the number of second video intervals is user-specified.
16. The method of claim 1 , further comprising:
ranking, by the computing device, the key video frames based at least in part on a criterion;
forming, by the computing device, key video snippets corresponding to the highest-ranked key video frames;
forming, by the computing device, a video summary by assembling the key video snippets; and
storing, by the computing device, a representation of the video summary.
17. A system comprising:
an image sensor;
an optical system configured to form an image of a scene onto the image sensor;
a data processing system; and
a memory system communicatively connected to the data processing system and storing instructions configured to cause the data processing system to:
capture a video using the image sensor, wherein the video comprises a plurality of first video intervals in a temporal order, wherein each of the plurality of first video intervals comprises a video frame;
determine an importance value for each of the plurality of first video intervals comprising a video frame;
form a warped time representation by modifying a duration of at least one of the first video intervals based at least in part on the importance values without modifying the temporal order of the video frames;
subdivide the warped time representation into intervals;
select a key video frame for each interval; and
store an indication of the key video frames.
18. The method of claim 1 , wherein the temporal order of the plurality of first video intervals is maintained in the warped time representation of the video.