Efficient motion-compensated spatiotemporal sampling

1. A method comprising, by a computing system:

accessing a video comprising a first frame and a second frame;

determining first sampling locations for the first frame;

determining second sampling locations for the second frame by transforming the first sampling locations to the second frame according to an optical flow between the first frame and the second frame;

selecting a subset of the second sampling locations based on a comparison between pixels in the first frame corresponding to the first sampling locations and pixels in the second frame corresponding to the second sampling locations;

defining one or more rejection areas in the second frame based on the subset of the second sampling locations;

determining third sampling locations for the second frame in areas outside of the one or more rejection areas; and

generating a sample of the video based on the pixels in the first frame corresponding to the first sampling locations and pixels in the second frame corresponding to the third sampling locations.

2. The method of claim 1 , wherein determining the first sampling locations for the first frame comprises generating a sample of the first frame using a two-dimensional mask comprising a blue noise property.

3. The method of claim 2 , wherein generating the sample of the first frame using the two-dimensional mask comprises using a threshold percentage of sampling points of the two-dimensional mask.

4. The method of claim 1 , wherein determining the second sampling locations for the second frame by transforming the first sampling locations to the second frame further comprises using a color temporal gradient between the first frame and the second frame.

5. The method of claim 1 , wherein selecting the subset of the second sampling locations comprises:

detecting one or more invalid second sampling locations based on determining pixels in the first frame corresponding to the first sampling locations do not match pixels in the second frame corresponding to the second sampling locations; and

rejecting the one or more invalid second sampling locations.

6. The method of claim 1 , wherein the one or more rejection areas in the second frame are areas of a fixed radius extending from the subset of the second sampling locations.

7. The method of claim 1 , wherein determining third sampling locations for the second frame comprises generating a sample of the second frame in areas outside of the one or more rejection areas using a two-dimensional mask comprising a blue noise property.

8. One or more computer-readable non-transitory storage media embodying software that is operable when executed to:

access a video comprising a first frame and a second frame;

determine first sampling locations for the first frame;

determine second sampling locations for the second frame by transforming the first sampling locations to the second frame according to an optical flow between the first frame and the second frame;

select a subset of the second sampling locations based on a comparison between pixels in the first frame corresponding to the first sampling locations and pixels in the second frame corresponding to the second sampling locations;

define one or more rejection areas in the second frame based on the subset of the second sampling locations;

determine third sampling locations for the second frame in areas outside of the one or more rejection areas; and

generate a sample of the video based on the pixels in the first frame corresponding to the first sampling locations and pixels in the second frame corresponding to the third sampling locations.

9. The media of claim 8 , wherein determining the first sampling locations for the first frame comprises generating a sample of the first frame using a two-dimensional mask comprising a blue noise property.

10. The media of claim 9 , wherein generating the sample of the first frame using the two-dimensional mask comprises using a threshold percentage of sampling points of the two-dimensional mask.

11. The media of claim 8 , wherein determining the second sampling locations for the second frame by transforming the first sampling locations to the second frame further comprises using a color temporal gradient between the first frame and the second frame.

12. The media of claim 8 , wherein selecting the subset of the second sampling locations comprises:

detect one or more invalid second sampling locations based on determining pixels in the first frame corresponding to the first sampling locations do not match pixels in the second frame corresponding to the second sampling locations; and

reject the one or more invalid second sampling locations.

13. The media of claim 8 , wherein the one or more rejection areas in the second frame are areas of a fixed radius extending from the subset of the second sampling locations.

14. The media of claim 8 , wherein determining third sampling locations for the second frame comprises generating a sample of the second frame in areas outside of the one or more rejection areas using a two-dimensional mask comprising a blue noise property.

15. A system comprising:

one or more processors; and

one or more computer-readable non-transitory storage media coupled to one or more of the processors and comprising instructions operable when executed by one or more of the processors to cause the system to:

access a video comprising a first frame and a second frame;

determine first sampling locations for the first frame;

determine second sampling locations for the second frame by transforming the first sampling locations to the second frame according to an optical flow between the first frame and the second frame;

select a subset of the second sampling locations based on a comparison between pixels in the first frame corresponding to the first sampling locations and pixels in the second frame corresponding to the second sampling locations;

define one or more rejection areas in the second frame based on the subset of the second sampling locations;

determine third sampling locations for the second frame in areas outside of the one or more rejection areas; and

generate a sample of the video based on the pixels in the first frame corresponding to the first sampling locations and pixels in the second frame corresponding to the third sampling locations.

16. The system of claim 15 , wherein determining the first sampling locations for the first frame comprises generating a sample of the first frame using a two-dimensional mask comprising a blue noise property.

17. The system of claim 16 , wherein generating the sample of the first frame using the two-dimensional mask comprises using a threshold percentage of sampling points of the two-dimensional mask.

18. The system of claim 15 , wherein determining the second sampling locations for the second frame by transforming the first sampling locations to the second frame further comprises using a color temporal gradient between the first frame and the second frame.

19. The system of claim 15 , wherein selecting the subset of the second sampling locations comprises:

detect one or more invalid second sampling locations based on determining pixels in the first frame corresponding to the first sampling locations do not match pixels in the second frame corresponding to the second sampling locations; and

reject the one or more invalid second sampling locations.

20. The system of claim 15 , wherein the one or more rejection areas in the second frame are areas of a fixed radius extending from the subset of the second sampling locations.