Mobile device support for capture and synthesis of extreme low-light video
An example embodiment may involve capturing a sequence of images, wherein there are 4 or more images in the sequence of images, and wherein each of the sequence of images has an exposure length of 4-100 seconds; applying a sliding window over the sequence of images as downsampled, wherein at least 4 images are encompassed within the sliding window’, and wherein for each position of the sliding window the applying involves: (i) aligning a set of images within the sliding window, and (ii) merging the set of images as aligned into a video frame; combining video frames generated by way of the sliding window into a video file; and storing, by the mobile device, the video file in memory of the mobile device.
1. A computer-implemented method comprising:
capturing, by a mobile device, a sequence of images, wherein there are 4 or more images in the sequence of images, and wherein each of the sequence of images has an exposure length of 4-100 seconds;
applying, by the mobile device, a sliding window over the sequence of images, wherein at least 4 images are encompassed within the sliding window, and wherein for each position of the sliding window the applying involves: (i) aligning a set of images within the sliding window, and (ii) merging the set of images as aligned into a video frame;
combining, by the mobile device, video frames generated by way of the sliding window into a video file, wherein the combining comprises:
identifying a first video frame and a second video frame that were sequentially generated:
detecting an object that has a first position in the first video frame and a second position in the second video frame, wherein the second position is different than the first position; and
generating an intermediate video frame that is temporally between the first video frame and the second video frame, wherein a position of the object in the intermediate video frame is extrapolated based on the object moving along an arc between the first position in the first video frame and the second position in the second video frame; and
storing, by the mobile device, the video file in memory of the mobile device.
2. The computer-implemented method of claim 1 , further comprising:
prior to applying the sliding window, downsampling, by the mobile device, the sequence of images by at least a factor of 2.
3. The computer-implemented method of claim 2 , wherein the downsampling and applying the sliding window overlap at least in part with the capturing.
4. The computer-implemented method claim 1 , wherein capturing the sequence of images comprises:
measuring a temperature of a camera component of the mobile device;
determining that the temperature exceeds a predetermined threshold; and
in response to determining that the temperature exceeds the predetermined threshold, delaying capture of a subsequent image in the sequence of images.
5. The computer-implemented method of claim 4 , wherein the measuring takes place during or after capture of an nth image in the sequence of images, and wherein the subsequent image is an (n+1) th image in the sequence of images.
6. The computer-implemented method of claim 4 , wherein delaying capture of the subsequent image comprises delaying capture of the subsequent image for a predetermined amount of time.
7. The computer-implemented method of claim 4 , wherein delaying capture of the subsequent image comprises delaying capture of the subsequent image until the temperature no longer exceeds the predetermined threshold.
8. The computer-implemented method of claim 1 , wherein applying the sliding window comprises:
measuring a temperature of a camera component of the mobile device;
determining that the temperature exceeds a predetermined threshold; and
in response to determining that the temperature exceeds the predetermined threshold, delaying movement of the sliding window to a next position.
9. The computer-implemented method of claim 8 , wherein delaying movement of the sliding window to the next position comprises delaying movement of the sliding window for a predetermined amount of time.
10. The computer-implemented method of claim 8 , wherein delaying movement of the sliding window to the next position comprises delaying movement of the sliding window until the temperature no longer exceeds the predetermined threshold.
11. The computer-implemented method of claim 1 , wherein there are at least 10 images in the sequence of images.
12. The computer-implemented method of claim 1 , wherein the exposure length is 12-16 seconds.
13. The computer-implemented method of claim 1 , wherein the object is a celestial object.
14. An article of manufacture including a non-transitory computer-readable medium, having stored thereon program instructions that, upon execution by an image capture device, cause the image capture device to perform operations comprising:
capturing a sequence of images, wherein there are 4 or more images in the sequence of images, and wherein each of the sequence of images has an exposure length of 4-100 seconds;
applying a sliding window over the sequence of images, wherein at least 4 images are encompassed within the sliding window, and wherein for each position of the sliding window the applying involves: (i) aligning a set of images within the sliding window, and (ii) merging the set of images as aligned into a video frame;
combining video frames generated by way of the sliding window into a video file, wherein the combining comprises:
identifying a first video frame and a second video frame that were sequentially generated:
detecting an object that has a first position in the first video frame and a second position in the second video frame, wherein the second position is different than the first position; and
generating an intermediate video frame that is temporally between the first video frame and the second video frame, wherein a position of the object in the intermediate video frame is extrapolated based on the object moving along an arc between the first position in the first video frame and the second position in the second video frame; and
storing the video file in memory of the mobile device.
15. A computer-implemented method comprising:
capturing, by a mobile device, a set of m images, each with an exposure length of 4-100 seconds;
generating, by the mobile device, a sequence of p video frames from the set of m images, wherein each of the p video frames is derived from a respective subset of the set of m images by aligning the respective subset of the m images and merging the respective subset of the m images as aligned into a video frame;
combining, by the mobile device, the p video frames into a video file in order of the sequence, wherein the combining comprises:
identifying a first video frame and a second video frame that were sequentially generated;
detecting an object that has a first position in the first video frame and a second position in the second video frame, wherein the second position is different than the first position: and
generating an intermediate video frame that is temporally between the first video frame and the second video frame, wherein a position of the object in the intermediate video frame is extrapolated based on the object moving along an arc between the first position in the first video frame and the second position in the second video frame; and
storing, by the mobile device, the video file in memory of the mobile device.
16. The computer-implemented method of claim 15 , wherein capturing the set of m images comprises:
measuring a temperature of a camera component of the mobile device;
determining that the temperature exceeds a predetermined threshold; and
in response to determining that the temperature exceeds the predetermined threshold, delaying capture of a subsequent image in the set of m images.
17. The computer-implemented method of claim 16 , wherein the measuring takes place during or after capture of an nth image in the set of m images, and wherein the subsequent image is an (n+1) th image in the set of m images.
18. The computer-implemented method of claim 15 , wherein m is at least 4.
19. The computer-implemented method of claim 15 , wherein the exposure length is 12-16 seconds.
20. The computer-implemented method of claim 15 , wherein the object is a celestial object.