Microvideo system, format, and method of generation
The present disclosure provides systems and methods that use and/or generate image files according to a novel microvideo image format. For example, a microvideo can be a file that contains both a still image and a brief video. The microvideo can include multiple tracks, such as, for example, a separate video track, audio track, and/or one or more metadata tracks. As one example track, the microvideo can include a motion data track that stores motion data that can be used (e.g., at file runtime) to stabilize the video frames. A microvideo generation system included in an image capture device can determine a trimming of the video on-the-fly as the image capture device captures the microvideo.
1. An image capture device comprising:
an image sensor configured to capture raw image data;
an image signal processor configured to process the raw image data to form a plurality of image frames; and
one or more processors to implement a microvideo generation system which is configured to:
generate a microvideo comprising a still image and a video in a same data file, and
perform an image selection algorithm to select the still image, wherein the video is an input to the image selection algorithm to select the still image.
2. The image capture device of claim 1 , wherein the microvideo generation system is configured to, when a duration of the microvideo is less than a minimum duration threshold, store the still image and delete the microvideo.
3. The image capture device of claim 1 , wherein the microvideo generation system is configured to, when a duration of the microvideo is less than a minimum duration threshold, add image frames to at least one of a beginning of the microvideo or an end of the microvideo until the duration of the microvideo is greater than or equal to the minimum duration threshold.
4. The image capture device of claim 1 , wherein the microvideo generation system is configured to:
generate a second microvideo in parallel with generation of the microvideo, the second microvideo comprising a second still image and a second video in a same second data file, and
perform the image selection algorithm to select the second still image, wherein the second video is a second input to the image selection algorithm to select the second still image.
5. The image capture device of claim 4 , wherein the microvideo and the second microvideo have different durations.
6. The image capture device of claim 4 , wherein the second microvideo is generated in parallel with generation of the microvideo in response to the image capture device receiving a user control input to capture an image more than a threshold number of times within a predetermined time period.
7. The image capture device of claim 4 , wherein the still image is displayable at a same or higher resolution than the video.
8. The image capture device of claim 1 , wherein
the microvideo includes a video track and a motion data track, and
the motion data track stores motion data usable to stabilize the microvideo.
9. The image capture device of claim 8 , wherein the motion data comprises a plurality of homography matrices.
10. The image capture device of claim 1 , wherein the microvideo generation system is configured to generate the microvideo by trimming a plurality of image frames that are available for inclusion in the microvideo.
11. The image capture device of claim 1 , wherein the microvideo generation system is configured to generate the microvideo by:
identifying a shutter frame based at least in part on a time associated with a user input;
searching through image frames stored in a buffer to identify a start frame;
identifying an end frame; and
generating the microvideo that includes image frames spanning from the start frame to the end frame.
12. The image capture device of claim 11 , wherein the microvideo generation system is configured to identify the start frame by searching backwards in time through the image frames stored in the buffer.
13. The image capture device of claim 11 , wherein the microvideo generation system is configured to apply one or more criteria to identify one of the start frame or the end frame.
14. The image capture device of claim 13 , wherein the one or more criteria comprise a motion criterion that analyzes an amount of motion between a current frame under evaluation and the shutter frame.
15. The image capture device of claim 14 , wherein the motion criterion analyzes a maximum amount of motion between any of four pairs of frame corners between the current frame under evaluation and the shutter frame.
16. The image capture device of claim 14 , wherein:
the motion criterion compares the amount of motion to a motion threshold;
the motion threshold is dynamically determined based at least in part on a camera motion speed; and
the camera motion speed is determined based at least in part on the amount of motion between the current frame under evaluation and the shutter frame.
17. The image capture device of claim 13 , wherein the one or more criteria comprise at least one of:
a sensitivity criterion that analyzes an amount of change in sensitivity between a current frame and the shutter frame,
a zoom criterion that analyzes an amount of change in zoom level between the current frame and the shutter frame,
a motion blur criterion that analyzes an amount of motion blur associated with the current frame, or
a focus criterion that analyzes auto focus data associated with the current frame.
18. The image capture device of claim 11 , wherein the microvideo generation system begins encoding from the start frame prior to working to identify the end frame.
19. A computer-implemented method, the method comprising:
generating, by a computing system comprising one or more processors, a microvideo comprising a still image and a video in a same data file; and
performing, by the computing system, an image selection algorithm to select the still image, wherein the video is an input to the image selection algorithm to select the still image.
20. One or more non-transitory computer-readable media that store instructions that, when executed by one or more processors of a computing system, cause the computing system to perform operations comprising:
generating a microvideo comprising a still image and a video in a same data file; and
performing an image selection algorithm to select the still image, wherein the video is an input to the image selection algorithm to select the still image.