Synthesizing intermediary frames for long exposure images
Techniques for synthesizing intermediary frames for long exposure images are disclosed. Video stream data, comprising a plurality of arriving frames, is received. An indication that the video stream data should be processed into a long exposure image that incorporates one or more synthesized frames is received. A set of arriving frames is used to generate at least one synthesized frame. At least one received frame is blended with the at least one generated synthesized frame to form a long exposure image. The long exposure image is provided as output.
1. A system, comprising:
a processor configured to:
receive, in response to calling a video capture application programming interface (API) of a device, video stream data, wherein the video stream data comprises a plurality of arriving frames captured as video data during a video recording operation on the device;
receive an indication that the received video stream data should be processed into a single long exposure image that incorporates one or more synthesized intermediary frames;
in response to a frame arrival align a current frame with a registration image and blend the aligned current frame with a single image buffer;
use at least two arriving frames included in the plurality of arriving frames to generate at least one synthesized intermediary frame that corresponds to a capture gap between a first arriving frame and a second arriving frame;
blend the single image buffer with the at least one generated synthesized intermediary frame to form the single long exposure image; and
provide the single long exposure image as output; and
a memory communication interface coupled to the processor and configured to provide the processor with instructions.
2. The system of claim 1 , wherein the plurality of arriving frames are obtained from a previously stored video file.
3. The system of claim 1 , wherein the indication is received from a user.
4. The system of claim 3 , wherein the indication is received in response to making a recommendation to the user that the single long exposure image should incorporate the one or more synthesized intermediary frames.
5. The system of claim 1 , wherein the indication is received in response to applying a set of heuristics to a set of image capture environment information.
6. The system of claim 1 , wherein the indication is received in response to applying a trained model to a set of image capture environment information.
7. The system of claim 1 , wherein the indication is received in response to a determination of a scene associated with the video stream data.
8. The system of claim 1 , wherein the indication is received in response to an optical flow analysis.
9. The system of claim 8 , wherein the optical flow analysis is performed using a set of reduced resolution thumbnail images.
10. The system of claim 1 , wherein generating the at least one synthesized intermediary frame includes applying a video smoothing technique.
11. The system of claim 1 , wherein generating the at least one synthesized intermediary frame includes using an optical flow.
12. The system of claim 1 , wherein generating the at least one synthesized intermediary frame includes applying a trained model.
13. The system of claim 12 , wherein the model is trained based at least in part on using a set of other long exposure images that incorporate one or more other synthesized intermediary frames.
14. The system of claim 1 , wherein the processor is further configured to generate a recommendation to a user to employ the one or more synthesized intermediary frames in creating the single long exposure image.
15. A method, comprising:
receiving, in response to calling a video capture application programming interface (API) of a device, video stream data, wherein the video stream data comprises a plurality of arriving frames captured as video data during a video recording operation on the device;
receiving an indication that the received video stream data should be processed into a single long exposure image that incorporates one or more synthesized intermediary frames;
in response to a frame arrival, aligning a current frame with a registration image and blending the aligned current frame with a single image buffer;
using at least two arriving frames included in the plurality of arriving frames to generate at least one synthesized intermediary frame that corresponds to a capture gap between a first arriving frame and a second arriving frame;
blending the single image buffer with the at least one generated synthesized intermediary frame to form the single long exposure image; and
providing the single long exposure image as output.
16. A computer program product embodied in a non-transitory computer readable medium and comprising computer instructions for:
receiving, in response to calling a video capture application programming interface (API) of a device, video stream data, wherein the video stream data comprises a plurality of arriving frames captured as video data during a video recording operation on the device;
receiving an indication that the received video stream data should be processed into a single long exposure image that incorporates one or more synthesized intermediary frames;
in response to a frame arrival, aligning a current frame with a registration image and blending the aligned current frame with a single image buffer;
using at least two arriving frames included in the plurality of arriving frames to generate at least one synthesized intermediary frame that corresponds to a capture gap between a first arriving frame and a second arriving frame;
blending the single image buffer with the at least one generated synthesized intermediary frame to form the single long exposure image; and
providing the single long exposure image as output.
17. The method of claim 16 , wherein the plurality of arriving frames are obtained from a previously stored video file.
18. The method of claim 16 , wherein the indication is received from a user.
19. The method of claim 18 , wherein the indication is received in response to making a recommendation to the user that the single long exposure image should incorporate the one or more synthesized intermediary frames.
20. The method of claim 16 , wherein the indication is received in response to applying a set of heuristics to a set of image capture environment information.
21. The method of claim 16 , wherein the indication is received in response to applying a trained model to a set of image capture environment information.
22. The method of claim 16 , wherein the indication is received in response to a determination of a scene associated with the video stream data.
23. The method of claim 16 , wherein the indication is received in response to an optical flow analysis.
24. The method of claim 23 , wherein the optical flow analysis is performed using a set of reduced resolution thumbnail images.
25. The method of claim 16 , wherein generating the at least one synthesized intermediary frame includes applying a video smoothing technique.
26. The method of claim 16 , wherein generating the at least one synthesized intermediary frame includes using an optical flow.
27. The method of claim 16 , wherein generating the at least one synthesized intermediary frame includes applying a trained model.
28. The method of claim 27 , wherein the model is trained based at least in part on using a set of other long exposure images that incorporate one or more other synthesized intermediary frames.
29. The method of claim 16 , further comprising generating a recommendation to a user to employ the one or more synthesized intermediary frames in creating the single long exposure image.