Using interpolation to generate a video from static images
A media application selects, from a collection of images associated with a user account, candidate pairs of images, where each pair includes a first static image and a second static image from the user account. The media application applies a filter to select a particular pair of images from the candidate pairs of images. The media application generates, using an image interpolator, one or more intermediate images based on the particular pair of images. The media application generates a video that includes three or more frames arranged in a sequence, where a first frame of the sequence is the first static image, a last frame of the sequence is the second static image, and each of the one or more intermediate images is a corresponding intermediate frame of the sequence between the first frame and the last frame.
1. A computer-implemented method comprising:
selecting, from a collection of images associated with a user account, candidate pairs of images, wherein each candidate pair includes a first static image of a scene and a second static image of the scene from the user account;
applying a filter to select a particular pair of images from the candidate pairs of images based on the particular pair of images failing to meet a threshold similarity;
generating, using an image interpolator, one or more intermediate images based on the particular pair of images;
providing the first static image as input to a depth machine-learning model, the depth machine-learning model outputting a three-dimensional representation of the scene; and
generating a video that includes three or more frames arranged in a sequence, wherein a first frame of the sequence is the first static image, a last frame of the sequence is the second static image, and each of the one or more intermediate images is a corresponding intermediate frame of the sequence between the first frame and the last frame, wherein the video includes the three-dimensional representation of the scene.
2. The method of claim 1 , wherein the filter further selects the particular pair of images based on the particular pair of images including a subject that a user associated with the user account expressed a preference for including in the video.
3. The method of claim 1 , wherein the video includes at least one camera effect selected from a group of zooming, panning, and combinations thereof.
4. The method of claim 1 , wherein the depth machine-learning model includes a convolutional neural network that outputs a low-resolution version of the three-dimensional representation and iteratively outputs improvements to the low-resolution version of the three-dimensional representation.
5. The method of claim 1 , wherein the depth machine-learning model is trained using training data that includes depth maps.
6. The method of claim 1 , wherein the first static image is in an x-y plane and the three-dimensional representation includes z-axis coordinates of objects in the three-dimensional representation.
7. The method of claim 1 , wherein the image interpolator includes an interpolation machine-learning model that receives the first static image and the second static image as input and that outputs the one or more intermediate images.
8. A system comprising:
one or more processors; and
a memory coupled to the one or more processors, with instructions stored thereon that, when executed by the one or more processors, cause the one or more processors to perform operations comprising:
selecting, from a collection of images associated with a user account, candidate pairs of images, wherein each candidate pair includes a first static image of a scene and a second static image of the scene from the user account;
applying a filter to select a particular pair of images from the candidate pairs of images based on the particular pair of images failing to meet a threshold similarity;
generating, using an image interpolator, one or more intermediate images based on the particular pair of images;
providing the first static image as input to a depth machine-learning model, the depth machine-learning model outputting a three-dimensional representation of the scene; and
generating a video that includes three or more frames arranged in a sequence, wherein a first frame of the sequence is the first static image, a last frame of the sequence is the second static image, and each of the one or more intermediate images is a corresponding intermediate frame of the sequence between the first frame and the last frame, wherein the video includes the three-dimensional representation of the scene.
9. The system of claim 8 , wherein the filter selects the particular pair of images based on the particular pair of images exceeding a motion threshold.
10. The system of claim 8 , wherein the video includes at least one camera effect selected from a group of zooming, panning, and combinations thereof.
11. The system of claim 8 , wherein the depth machine-learning model includes a convolutional neural network that outputs a low-resolution version of the three-dimensional representation and iteratively outputs improvements to the low-resolution version of the three-dimensional representation.
12. The system of claim 8 , wherein the depth machine-learning model is trained using training data that includes depth maps.
13. The system of claim 8 , wherein the first static image is in an x-y plane and the three-dimensional representation includes z-axis coordinates of objects in the three-dimensional representation.
14. The system of claim 8 , wherein the image interpolator includes an interpolation machine-learning model that receives the first static image and the second static image as input and that outputs the one or more intermediate images.
15. A non-transitory computer-readable medium with instructions stored thereon that, when executed by one or more computers, cause the one or more computers to perform operations, the operations comprising:
selecting, from a collection of images associated with a user account, candidate pairs of images, wherein each candidate pair includes a first static image of a scene and a second static image of the scene from the user account;
applying a filter to select a particular pair of images from the candidate pairs of images based on the particular pair of images failing to meet a threshold similarity;
generating, using an image interpolator, one or more intermediate images based on the particular pair of images;
providing the first static image as input to a depth machine-learning model, the depth machine-learning model outputting a three-dimensional representation of the scene; and
generating a video that includes three or more frames arranged in a sequence, wherein a first frame of the sequence is the first static image, a last frame of the sequence is the second static image, and each of the one or more intermediate images is a corresponding intermediate frame of the sequence between the first frame and the last frame, wherein the video includes the three-dimensional representation of the scene.
16. The computer-readable medium of claim 15 , wherein the filter selects the particular pair of images based on the particular pair of images falling below a motion threshold.
17. The computer-readable medium of claim 15 , wherein the video includes at least one camera effect selected from a group of zooming, panning, and combinations thereof.
18. The computer-readable medium of claim 15 , wherein the depth machine-learning model includes a convolutional neural network that outputs a low-resolution version of the three-dimensional representation and iteratively outputs improvements to the low-resolution version of the three-dimensional representation.
19. The computer-readable medium of claim 15 , wherein the depth machine-learning model is trained using training data that includes depth maps.
20. The computer-readable medium of claim 15 , wherein the first static image is in an x-y plane and the three-dimensional representation includes z-axis coordinates of objects in the three-dimensional representation.