Cinematic space-time view synthesis for enhanced viewing experiences in computing environments

1. An apparatus comprising:

memory; and

at least one processor including a graphics processing unit, the at least one processor to:

access first and second images in a first series of images of a first video, the first image immediately preceding the second image in the first series of images, the first and second images corresponding to respective first and second views;

generate first optical flow data based on a change of position of first pixels in the first image relative to corresponding ones of second pixels in the second image;

generate second optical flow data based on a change of position of third pixels in the second image relative to corresponding ones of fourth pixels in the first image;

generate a first warped image based on the first image and the first optical flow data;

generate a second warped image based on the second image and the second optical flow data;

use a neural network to synthesize the first and second warped images into a single intermediate image, the intermediate image corresponding to an intermediate view between the first and second views corresponding to the first and second images respectively; and

generate a second series of images for a second video, the second series of images corresponding to the first series of images, the second series of images having a higher number of images than the first series of images, the second series of images including the first image, the second image, and the intermediate image.

2. The apparatus of claim 1 , wherein the second series of images is to provide a smoother slow-motion effect than the first series of images.

3. The apparatus of claim 1 , wherein the neural network is a first neural network, the at least one processor to use a second neural network to generate the first optical flow data and the second optical flow data.

4. The apparatus of claim 3 , wherein the first and second neural networks are part of a single network, the single network to be trained end-to-end.

5. The apparatus of claim 3 , wherein at least one of the first neural network or the second neural network is a convolutional neural network.

6. The apparatus of claim 5 , wherein the convolutional neural network is to implement at least one of an encoder or a decoder.

7. The apparatus of claim 1 , further including:

at least one storage device;

wireless communication circuitry; and

at least one antenna.

8. The apparatus of claim 1 , further including a display to present the second series of images for the second video.

9. The apparatus of claim 1 , further including at least one camera to generate the first series of images for the first video.

10. At least one storage device comprising instructions that, when executed, cause at least one processor to:

access first and second images in a first series of images of a first video, the first image immediately preceding the second image in the first series of images, the first and second images corresponding to respective first and second views;

generate first optical flow data based on a change of position of first pixels in the first image relative to corresponding ones of second pixels in the second image;

generate second optical flow data based on a change of position of third pixels in the second image relative to corresponding ones of fourth pixels in the first image;

generate a first warped image based on the first image and the first optical flow data;

generate a second warped image based on the second image and the second optical flow data;

execute a neural network to combine the first and second warped images into a single intermediate image, the intermediate image corresponding to an intermediate view between the first and second views corresponding to the first and second images respectively; and

generate a second series of images for a second video, the second series of images corresponding to the first series of images, the second series of images having a higher number of images than the first series of images, the second series of images including the first image, the second image, and the intermediate image.

11. The at least one storage device of claim 10 , wherein the second series of images is to provide a smoother slow-motion effect than the first series of images.

12. The at least one storage device of claim 10 , wherein the neural network is a first neural network, the instructions to cause the at least one processor to execute a second neural network to generate the first optical flow data and the second optical flow data.

13. The at least one storage device of claim 12 , wherein at least one of the first neural network or the second neural network is a convolutional neural network.

14. The at least one storage device of claim 13 , wherein the convolutional neural network is to implement at least one of an encoder or a decoder.

15. The at least one storage device of claim 12 , wherein the first and second neural networks are part of a single network, the single network to be trained end-to-end.

16. A method comprising:

accessing first and second images in a first series of images of a first video, the first image immediately preceding the second image in the first series of images, the first and second images corresponding to respective first and second views;

generating first optical flow data based on a change of position of first pixels in the first image relative to corresponding ones of second pixels in the second image;

generating second optical flow data based on a change of position of third pixels in the second image relative to corresponding ones of fourth pixels in the first image;

generating a first warped image based on the first image and the first optical flow data;

generating a second warped image based on the second image and the second optical flow data;

synthesizing, using a neural network, the first and second warped images into a single intermediate image, the intermediate image corresponding to an intermediate view between the first and second views corresponding to the first and second images respectively; and

generating a second series of images for a second video, the second series of images corresponding to the first series of images, the second series of images having a higher number of images than the first series of images, the second series of images including the first image, the second image, and the intermediate image.

17. The method of claim 16 , wherein the second series of images is to provide a smoother slow-motion effect than the first series of images.

18. The method of claim 16 , wherein the neural network is a first neural network, and generating the first optical flow data and the second optical flow data includes using a second neural network different than the first neural network.

19. The method of claim 18 , wherein at least one of the first neural network or the second neural network is a convolutional neural network.

20. The method of claim 19 , wherein the convolutional neural network is to implement at least one of an encoder or a decoder.

21. The method of claim 18 , wherein the first and second neural networks are part of a single network, the single network to be trained end-to-end.

22. An apparatus comprising:

means for storing information; and

means for executing instructions, the executing means to:

access first and second images in a first series of images of a first video, the first image immediately preceding the second image in the first series of images, the first and second images corresponding to respective first and second views;

generate first optical flow data based on a change of position of first pixels in the first image relative to corresponding ones of second pixels in the second image;

generate second optical flow data based on a change of position of third pixels in the second image relative to corresponding ones of fourth pixels in the first image;

generate a first warped image based on the first image and the first optical flow data;

generate a second warped image based on the second image and the second optical flow data;

use a neural network to combine the first and second warped images into a single intermediate image, the intermediate image corresponding to an intermediate view between the first and second views corresponding to the first and second images respectively; and

generate a second series of images for a second video, the second series of images corresponding to the first series of images, the second series of images having a higher number of images than the first series of images, the second series of images including the first image, the second image, and the intermediate image.

23. The apparatus of claim 22 , wherein the second series of images is to provide a smoother slow-motion effect than the first series of images.

24. The apparatus of claim 22 , wherein the neural network is a first neural network, the executing means to use a second neural network distinct from the first neural network to generate the first optical flow data and the second optical flow data.

25. The apparatus of claim 24 , wherein at least one of the first neural network or the second neural network is a convolutional neural network.

26. The apparatus of claim 25 , wherein the convolutional neural network is to implement at least one of an encoder or a decoder.

27. The apparatus of claim 24 , wherein the first and second neural networks are part of a single network, the single network to be trained end-to-end.