Trajectory planning for video stabilization

1. A method comprising:

receiving a sequence of video frames;

low-pass filtering the video frames;

determining whether a subset of the low-pass filtered frames are zero-motion frames;

estimating a smooth trajectory for the zero-motion frames;

removing jitter from the zero-motion frames using the smooth trajectory; and

applying frame warping to the zero-motion frames with the jitter removed.

2. The method of claim 1 , wherein the determining comprises comparing coordinates of a selected low-pass filtered frame to an adjacent low-pass filtered frame in the sequence of video frames.

3. The method of claim 2 , wherein comparing comprises determining a distance between coordinates of the selected low-pass filtered frame and the adjacent low-pass filtered frame and comparing the distance to a threshold, wherein if the distance is less than the threshold then the subset of the low-pass filtered frames are determined to be the zero motion frames.

4. The method of claim 3 , wherein the determining the distance and comparing the distance to the threshold is performed for each frame of the subset and, if the distance is greater than the threshold, then a transition to motion video is determined.

5. The method of claim 1 , wherein the determining comprises:

determining a first trajectory of the subset of low-pass filtered frames;

determining a second zero-motion trajectory; and

comparing the first trajectory to the second trajectory.

6. The method of claim 5 , wherein the comparing comprises comparing coordinates of frames of the first trajectory to corresponding frames of the second trajectory and if the coordinates are within a threshold distance then determining that the subset of low-pass filtered frames are zero-motion frames.

7. The method of claim 6 , wherein the comparing coordinates comprises comparing a mean of a short sequence of coordinates of frames of the first trajectory to a mean of coordinates of corresponding frames of the second trajectory and if the means are within a threshold distance then determining that the subset of low-pass filtered frames are zero-motion frames.

8. The method of claim 3 , wherein the comparing coordinates comprises comparing a variance of a short sequence of coordinates of frames of the first trajectory to a variance of coordinates of corresponding frames of the second trajectory and if the variances are within a threshold distance then determining that the subset of low-pass filtered frames are zero-motion frames.

9. The method of claim 1 , wherein if the subset of low-pass filtered frames are determined to not be zero-motion frames then applying frame warping to the low-pass filtered frames.

10. The method of claim 1 , wherein the smooth trajectory is based on coordinates of a frame that is selected as an anchor point, the method further comprising:

checking for future clamping; and

updating the smooth trajectory based on the clamping check.

11. The method of claim 10 , wherein updating the smooth trajectory comprises selecting a past frame of the output frames and producing a curve between the past frame and the anchor point.

12. An apparatus comprising:

a filter to receive a sequence of video frames and to low-pass filter the video frames;

a zero motion detector to determine whether a subset of the low-pass filtered video frames are zero-motion frames;

trajectory smoothing to estimate a smooth trajectory for the zero-motion frames; and

frame warping to remove jitter from the zero-motion frames using the smooth trajectory.

13. The apparatus of claim 12 , wherein the zero motion detector compares coordinates of a selected low-pass filtered frame to an adjacent low-pass filtered frame in the sequence of video frames.

14. The apparatus of claim 13 , wherein the zero motion detector compares by determining a distance between coordinates of the selected low-pass filtered frame and the adjacent low-pass filtered frame and comparing the distance to a threshold, wherein if the distance is less than the threshold then the subset of the low-pass filtered frames are determined to be zero motion frames.

15. The apparatus of claim 13 , wherein the zero motion detector compares by comparing a mean of a short sequence of coordinates of frames of the subset of frames trajectory to the selected low-pass filtered frame.

16. A non-transitory computer-readable medium having instructions that when operated on cause a computer to perform operations comprising:

receiving a sequence of video frames;

low-pass filtering the video frames;

determining whether a subset of the low-pass filtered video frames are zero-motion frames;

estimating a smooth trajectory for the zero-motion frames;

removing jitter from the zero-motion frames using the smooth trajectory; and

applying frame warping to the zero-motion frames with jitter removed.

17. The medium of claim 16 , wherein if the subset of low-pass filtered video frames are determined to not be zero-motion frames then the operations further comprising applying the frame warping to the low-pass filtered video frames.

18. The medium of claim 16 , wherein the smooth trajectory is based on coordinates of a frame that is selected as an anchor point, the operations further comprising:

checking for future clamping; and

updating the smooth trajectory based on the clamping check.

19. A computing device comprising:

an image sensor to record a sequence of video frames;

a memory to store the sequence of video frames;

a Gaussian filter to low-pass filter the stored sequence of video frames;

a zero motion detector to determine whether a subset of the low-pass filtered frames are zero motion frames;

a trajectory estimator to estimate a smooth trajectory corresponding to the subset of the low-pass filtered frames; and

a trajectory planner to remove jitter from the subset of the low-pass filtered frames using the smooth trajectory if zero motion is detected.

20. The computing device of claim 19 , wherein the smooth trajectory is based on coordinates of a frame that is selected as an anchor point, the trajectory estimator further:

checking for future clamping; and

updating the smooth trajectory based on the clamping check.