Microvideo system, format, and method of generation
The present disclosure provides systems and methods that use and/or generate image files according to a novel microvideo image format. For example, a microvideo can be a file that contains both a still image and a brief video. The microvideo can include multiple tracks, such as, for example, a separate video track, audio track, and/or one or more metadata tracks. As one example track, the microvideo can include a motion data track that stores motion data that can be used (e.g., at file runtime) to stabilize the video frames. A microvideo generation system included in an image capture device can determine a trimming of the video on-the-fly as the image capture device captures the microvideo.
1. An image capture device comprising:
an image sensor operable to capture raw image data;
an image signal processor configured to process the raw image data to form a plurality of image frames, wherein the image signal processor comprises one or more hardware blocks;
one or more motion sensing sensors that provide sensor data indicative of motion of the image sensor or the image capture device;
one or more processors; and
a microvideo generation system implemented by the one or more processors, wherein the microvideo generation system is configured to generate a microvideo, the microvideo comprising both a still image and a video in a same data file;
wherein the microvideo generation system performs an image selection algorithm to select the still image; and
wherein the video is an input to the image selection algorithm to select the still image.
2. The image capture device of claim 1 , wherein the still image comprises a JPEG image and the video comprises an MP4 video.
3. The image capture device of claim 1 , wherein the video is within a container associated with the still image.
4. The image capture device of claim 1 , wherein the microvideo includes a video track and a motion data track, wherein the motion data track stores motion data usable to stabilize the microvideo.
5. The image capture device of claim 4 , wherein the motion data comprises a plurality of homography matrices.
6. The image capture device of claim 1 , wherein to generate the microvideo, the microvideo generation system trims a plurality of image frames that are available for inclusion in the microvideo.
7. The image capture device of claim 1 , wherein to generate the microvideo, the microvideo generation system:
identifies a shutter frame based at least in part on a time associated with a user input;
searches through image frames stored in a buffer to identify a start frame;
identifies an end frame; and
generates the microvideo that includes image frames spanning from the start frame to the end frame.
8. The image capture device of claim 7 , wherein to identify the start frame, the microvideo generation system searches backwards in time through the image frames stored in the buffer.
9. The image capture device of claim 7 , wherein to identify one or both of the start frame or the end frame, the microvideo generation system applies one or more criteria.
10. The image capture device of claim 9 , wherein the one or more criteria comprise a motion criterion that analyzes an amount of motion between a current frame under evaluation and the shutter frame.
11. The image capture device of claim 10 , wherein the motion criterion analyzes a maximum amount of motion between any of four pairs of frame corners between the current frame under evaluation and the shutter frame.
12. The image capture device of claim 10 , wherein:
the motion criterion compares the amount of motion to a motion threshold;
the motion threshold is dynamically determined based at least in part on a camera motion speed; and
the camera motion speed is determined based at least in part on the amount of motion between the current frame under evaluation and the shutter frame.
13. The image capture device of claim 7 , wherein the microvideo generation system begins encoding from the starting frame prior to working to identify the ending frame.
14. The image capture device of claim 9 , wherein the one or more criteria comprise a sensitivity criterion that analyzes an amount of change in sensitivity between a current frame under evaluation and the shutter frame.
15. The image capture device of claim 9 , wherein the one or more criteria comprise a zoom criterion that analyzes an amount of change in zoom level between a current frame under evaluation and the shutter frame.
16. The image capture device of claim 9 , wherein the one or more criteria comprise a motion blur criterion that analyzes an amount of motion blur associated with a current frame.
17. The image capture device of claim 9 , wherein the one or more criteria comprise a focus criterion that analyzes auto focus data associated with a current frame.
18. A computer-implemented method, the method comprising:
generating, by a computing system comprising one or more processors, a microvideo, the microvideo comprising both a still image and a video in a same data file;
performing, by the computing system, an image selection algorithm to select the still image; and
inputting, by the computing system, the video to the image selection algorithm to select the still image.
19. The computer-implemented method of claim 18 , wherein the still image comprises a JPEG image and the video comprises an MP4 video.
20. One or more non-transitory computer-readable media that store instructions that, when executed by one or more processors of a computing system, cause the computing system to perform operations comprising:
generating, by a computing system comprising one or more processors, a microvideo, the microvideo comprising both a still image and a video in a same data file;
performing, by the computing system, an image selection algorithm to select the still image; and
inputting, by the computing system, the video to the image selection algorithm to select the still image.