Tracking and detection for motion estimation in real time digital video stabilization
A method includes obtaining two consecutive video frames at a global motion estimation function for execution on a processor, wherein the video frames comprise a current video frame and a previous video frame, and estimating motion between the two consecutive video frames by matching a set of feature points common to both video frames. The set of feature points is maintained by tracking a number of feature points in the current video frame, refreshing the feature points if the number of feature points falls below a refresh threshold, and replenishing the feature points if the number of feature points falls below a replenish threshold. Motion filtering may be performed by buffering a homogenous transformation of the global motion estimation, calculating a geometric mean of the buffered motions, and estimating intentional camera trajectory based on the geometric mean.
1. A method comprising:
obtaining two consecutive video frames at a global motion estimation function for execution on a processor, wherein the video frames comprise a current video frame and a previous video frame; and
estimating motion between the two consecutive video frames by matching a set of feature points common to both video frames, wherein the set of feature points is maintained by:
tracking a number of feature points in the current video frame;
refreshing the feature points if the number of feature points falls below a refresh threshold; and
replenishing the feature points if the number of feature points falls below a replenish threshold, the replenishing including retaining features points corresponding to static background and generating new feature points from new areas identified outside of a neighborhood of the feature points.
2. The method of claim 1 wherein refreshing the feature points comprises:
discarding the feature points; and
deriving a new set of feature points from a previous video frame.
3. The method of claim 1 wherein replenishing the feature points further comprises:
identifying the new areas resulting from camera view change.
4. The method of claim 3 wherein the set of feature points has an initial number of feature points, and wherein a number of new feature points are generated corresponding the initial number of feature points minus the replenish threshold.
5. The method of claim 1 wherein the set of feature points has an initial number of feature points, wherein the replenish threshold is less than the initial number of features, and wherein the refresh threshold is less than the replenish threshold.
6. The method of claim 5 wherein the initial number of feature points is a function of a number of pixels in the frames.
7. The method of claim 5 wherein the replenish threshold is about one-half the number of initial feature points.
8. The method of claim 5 wherein the refresh threshold is ⅕ th to ⅙ th of the number of initial feature points.
9. The method of claim 1 wherein each of tracking, refreshing, and replenishing is performed by a state machine.
10. The method of claim 1 and further comprising:
tracking a duration of the feature points being used without replenishing or refreshing;
discarding a portion of the feature points when the duration exceeds a threshold; and
replenishing the feature points with newly tracked feature points.
11. The method of claim 10 wherein the portion of discarded feature points comprises randomly selected feature points.
12. A non-transitory computer readable medium storing computer instructions, that when executed by one or more processors, cause the one or more processors to perform the steps of:
obtaining two consecutive video frames at a global motion estimation function for execution on a processor, wherein the video frames comprise a current video frame and a previous video frame; and
estimating motion between the two consecutive video frames by matching a set of feature points common to both video frames, wherein the set of feature points is maintained by:
tracking a number of feature points in the current video frame;
refreshing the feature points if the number of feature points falls below a refresh threshold; and
replenishing the feature points if the number of feature points falls below a replenish threshold, the replenishing including retaining features points corresponding to static background and generating new feature points from new areas identified outside of a neighborhood of the feature points.
13. The non-transitory computer readable medium of claim 12 wherein refreshing the feature points causes the one or more processors to further perform the steps of:
discarding the feature points; and
deriving a new set of feature points from a previous video frame.
14. The non-transitory computer readable medium of claim 12 wherein replenishing the feature points causes the one or more processors to further perform the step of:
identifying the new areas resulting from camera view change.
15. The non-transitory computer readable medium of claim 12 wherein the set of feature points has an initial number of feature points, wherein the replenish threshold is about one-half the number of initial feature points and the refresh threshold is ⅕ th to ⅙ th of the number of initial feature points.
16. The non-transitory computer readable medium of claim 12 wherein each of tracking, refreshing, and replenishing is performed by a state machine.
17. The non-transitory computer readable medium of claim 12 and causes the one or more processors to further perform the steps of:
tracking a duration of the feature points being used without replenishing or refreshing;
discarding a portion of the feature points when the duration exceeds a threshold, wherein the portion of discarded feature points comprises randomly selected feature points; and
replenishing the feature points with newly tracked feature points.
18. A system comprising:
a non-transitory memory storage comprising instructions; and
one or more processors in communication with the memory, wherein the one or more processors execute the instructions to:
obtain two consecutive video frames at a global motion estimation function for execution on a processor, wherein the video frames comprise a current video frame and a previous video frame; and
estimate motion between the two consecutive video frames by matching a set of feature points common to both video frames, wherein the set of feature points is maintained by:
tracking a number of feature points in the current video frame;
refreshing the feature points if the number of feature points falls below a refresh threshold; and
replenishing the feature points if the number of feature points falls below a replenish threshold, the replenishing including retaining features points corresponding to static background and generating new feature points from new areas identified outside of a neighborhood of the feature points.
19. The system of claim 18 wherein refreshing the feature points causes the one or more processors to further execute the instructions to:
discard the feature points; and
derive a new set of feature points from a previous video frame, and wherein replenishing the feature points comprises:
retaining feature points corresponding to static background;
identifying new areas resulting from camera view change; and
generating new feature points from the new areas, wherein the set of feature points has an initial number of feature points, wherein the replenish threshold is less than the initial number of features, and wherein the refresh threshold is less than the replenish threshold.