Scale image resolution for motion estimation

1. An apparatus for processing image data, the apparatus comprising:

at least one memory; and

at least one processor coupled to the at least one memory, the at least one processor configured to:

determine, using a higher-power motion estimation pipeline, a dense motion map between a first input image and a first reference image, the dense motion map indicating local motion between pixels in the first input image and pixels in the first reference image;

determine that the local motion indicated by the dense motion map is below a first threshold;

in response to a determination that the local motion indicated by the motion vectors dense motion map is below the first threshold, switch from the higher-power motion estimation pipeline to a lower-power motion estimation pipeline for a second input image and a second reference image, wherein, using the lower-power motion estimation pipeline, the at least one processor is configured to:

determine, using descriptor matching estimation, motion vectors indicating global motion between the second input image and the second reference image;

refrain from determining a dense motion map between the second input image and the second reference image;

refrain from performing image alignment between the second input image and the second reference image; and

warp the motion vectors using a global stabilization matrix to generate a transform matrix without using the dense motion map between the second input image and the second reference image; and

perform image stabilization for the second input image using the transform matrix.

2. The apparatus of claim 1 , wherein the at least one processor is configured to:

determine, based on a third input image and a third reference image, additional motion vectors identifying motion between the third input image and the third reference image;

determine whether the motion indicated by the additional motion vectors is below or above the first threshold;

based on a determination that the motion indicated by the additional motion vectors is above the first threshold, generate an additional dense motion map indicating a local motion between the third input image and the third reference image;

determine an additional transform matrix based on the additional dense motion map; and

adjust the third input image based on the additional transform matrix.

3. The apparatus of claim 2 , wherein, to determine the additional transform matrix based on the additional dense motion map, the at least one processor is configured to warp the additional dense motion map using the global stabilization matrix.

4. The apparatus of claim 2 , wherein the at least one processor is configured to:

prior to generating the additional dense motion map, align the additional motion vectors based on a global motion associated with the apparatus, the additional motion vectors being aligned based on motion information obtained from a motion sensor, the motion sensor comprising at least one of a gyroscope or an inertial measurement unit; and

generate the additional dense motion map based on the aligned additional motion vectors.

5. The apparatus of claim 2 , wherein the at least one processor is configured to determine that the motion indicated by the additional motion vectors is above the first threshold and below a second threshold that is greater than the first threshold.

6. The apparatus of claim 2 , wherein the at least one processor is configured to:

prior to determining the additional motion vectors identifying motion between the third input image and the third reference image, downscale the third input image and the third reference image, wherein the additional motion vectors are determined based on the downscaled third input image and the downscaled third reference image.

7. The apparatus of claim 1 , wherein the at least one processor is configured to:

determine, based on a third input image and a third reference image, additional motion vectors identifying motion between the third input image and the third reference image;

determine whether the motion indicated by the additional motion vectors is above the first threshold and below a second threshold that is greater than the first threshold; and

based on a determination that the motion indicated by the additional motion vectors is above the first threshold and below the second threshold:

downscale the third input image and the third reference image;

generate an additional local motion map indicating a local motion between the downscaled third input image and the downscaled third reference image;

determine an additional transform matrix based on the additional local motion map; and

adjust the third input image based on the additional transform matrix.

8. The apparatus of claim 7 , wherein, to determine the additional transform matrix based on the additional local motion map, the at least one processor is configured to warp the additional local motion map using the global stabilization matrix.

9. The apparatus of claim 7 , wherein the at least one processor is configured to:

prior to generating the additional local motion map, align the additional motion vectors based on a global motion associated with the apparatus, the additional motion vectors being aligned based on motion information obtained from a motion sensor, the motion sensor comprising at least one of a gyroscope and an inertial measurement unit.

10. The apparatus of claim 9 , wherein the at least one processor is configured to generate the additional local motion map based on the aligned additional motion vectors.

11. The apparatus of claim 1 , further comprising a camera configured to capture a video sequence comprising the first input image, the first reference image, the second input image, and the second reference image.

12. The apparatus of claim 1 , wherein the apparatus comprises a mobile device.

13. A method of processing image data, the method comprising:

determining, by a computing device using a higher-power motion estimation pipeline, a dense motion map between a first input image and a first reference image, the dense motion map indicating local motion between pixels in the first input image and pixels in the first reference image;

determining that the local motion indicated by the dense motion map is below a first threshold;

in response to a determination that the local motion indicated by the dense motion map is below the first threshold, switch from the higher-power motion estimation pipeline to a lower-power motion estimation pipeline for a second input image and a second reference image, wherein the lower-power motion estimation pipeline includes:

determining motion vectors indicating global motion between the second input image and the second reference image;

refraining from determining a dense motion map between the second input image and the second reference image;

refraining from performing image alignment between the second input image and the second reference image; and

warping the motion vectors using a global stabilization matrix to generate a transform matrix without using the dense motion map between the second input image and the second reference image; and

performing image stabilization for the second input image using the transform matrix.

14. The method of claim 13 , further comprising:

determining, based on a third input image and a third reference image, additional motion vectors identifying motion between the third input image and the third reference image;

determining whether the motion indicated by the additional motion vectors is below or above the first threshold;

based on a determination that the motion indicated by the additional motion vectors is above the first threshold, generate a motion map indicating a local motion between the third input image and the third reference image;

determining an additional transform matrix based on the motion map; and

adjusting the third input image based on the additional transform matrix.

15. The method of claim 14 , wherein determining the additional transform matrix based on the motion map comprises warping the motion map using the global stabilization matrix.

16. The method of claim 14 , further comprising:

prior to generating the motion map, aligning the additional motion vectors based on a global motion associated with the computing device, the additional motion vectors being aligned based on motion information obtained from a motion sensor, the motion sensor comprising at least one of a gyroscope and an inertial measurement unit; and

generating the motion map based on the aligned additional motion vectors.

17. The method of claim 14 , further comprising determining that the motion indicated by the additional motion vectors is above the first threshold and below a second threshold that is greater than the first threshold.

18. The method of claim 14 , further comprising:

prior to determining the additional motion vectors identifying motion between the third input image and the third reference image, downscale the third input image and the third reference image, wherein the additional motion vectors are determined based on the downscaled third input image and the downscaled third reference image.

19. The method of claim 13 , further comprising:

determining, based on a third input image and a third reference image, additional motion vectors identifying motion between the third input image and the third reference image;

determining whether the motion indicated by the additional motion vectors is above the first threshold and below a second threshold that is greater than the first threshold; and

based on a determination that the motion indicated by the additional motion vectors is above the first threshold and below the second threshold:

downscaling the third input image and the third reference image;

generating a motion map indicating a local motion between the downscaled third input image and the downscaled third reference image;

determining an additional transform matrix based on the motion map; and

adjusting the third input image based on the additional transform matrix.

20. The method of claim 19 , wherein determining the additional transform matrix based on the motion map comprising warping the motion map using the global stabilization matrix.

21. The method of claim 19 , further comprising:

prior to generating the motion map, aligning the additional motion vectors based on a global motion associated with the computing device, the additional motion vectors being aligned based on motion information obtained from a motion sensor, the motion sensor comprising at least one of a gyroscope and an inertial measurement unit.

22. The method of claim 21 , wherein generating the motion map comprises generating the motion map based on the aligned additional motion vectors.

23. A non-transitory computer-readable medium having stored thereon instructions that, when executed by one or more processors, cause the one or more processors to:

determine, using a higher-power motion estimation pipeline, a dense motion map between a first input image and a first reference image, the dense motion map indicating local motion between pixels in the first input image and pixels in the first reference image;

determine that the local motion indicated by the dense motion map is below a first threshold;

in response to a determination that the local motion indicated by the dense motion map is below the first threshold, switch from the higher-power motion estimation pipeline to a lower-power motion estimation pipeline for a second input image and a second reference image, wherein, using the lower-power motion estimation pipeline, the instructions, when executed by the one or more processors, cause the one or more processors to:

determine motion vectors indicating global motion between the second input image and the second reference image;

refrain from determining a dense motion map between the second input image and the second reference image;

refrain from performing image alignment between the second input image and the second reference image; and

warp the motion vectors using a global stabilization matrix to generate a transform matrix without using the dense motion map between the second input image and the second reference image; and

perform image stabilization for the second input image using the transform matrix.

24. The non-transitory computer-readable medium of claim 23 , further comprising instructions that, when executed by the one or more processors, cause the one or more processors to:

determine, based on a third input image and a third reference image, additional motion vectors identifying motion between the third input image and the third reference image;

determine whether the motion indicated by the additional motion vectors is below or above the first threshold;

based on a determination that the motion indicated by the additional motion vectors is above the first threshold, generate a motion map indicating a local motion between the third input image and the third reference image;

determine an additional transform matrix based on the motion map; and

adjust the third input image based on the additional transform matrix.

25. The non-transitory computer-readable medium of claim 24 , wherein, to determine the additional transform matrix based on the motion map, the instructions, when executed by the one or more processors, cause the one or more processors to warp the motion map using the global stabilization matrix.

26. The non-transitory computer-readable medium of claim 24 , further comprising instructions that, when executed by the one or more processors, cause the one or more processors to:

prior to generating the motion map, align the additional motion vectors based on a global motion associated with a computing device comprising the non-transitory computer-readable medium, the additional motion vectors being aligned based on motion information obtained from a motion sensor, the motion sensor comprising at least one of a gyroscope and an inertial measurement unit; and

generate the motion map based on the aligned additional motion vectors.

27. The non-transitory computer-readable medium of claim 24 , further comprising instructions that, when executed by the one or more processors, cause the one or more processors to determine that the motion indicated by the additional motion vectors is above the first threshold and below a second threshold that is greater than the first threshold.

28. The non-transitory computer-readable medium of claim 24 , further comprising instructions that, when executed by the one or more processors, cause the one or more processors to:

prior to determining the additional motion vectors identifying motion between the third input image and the third reference image, downscale the third input image and the third reference image, wherein the additional motion vectors are determined based on the downscaled third input image and the downscaled third reference image.

29. The apparatus of claim 1 , wherein the second reference image is the first input image.

30. The method of claim 13 , wherein the second reference image is the first input image.