Real-time video stabilization for mobile devices based on on-board motion sensing

1. A computer implemented method comprising:

accessing a first image frame from a camera at a first time;

accessing a second image frame from said camera at a subsequent time;

identifying a crop polygon around scene content common to said first image frame and said second image frame;

accessing movement information describing movement of said camera in an interval between said first time and said second time;

using said movement information, warping said crop polygon to remove motion distortions of said second image frame,

wherein said warping comprises defining a virtual camera that remains static when said movement of said camera is below a movement threshold;

defining a padding space around said crop polygon to adjust movement of said virtual camera;

damping movement of said virtual camera when said crop polygon is within an inner region of said padding space; and

accelerating said movement of said virtual camera when said crop polygon extends outside of said inner region of said padding space.

2. The computer implemented method of claim 1 wherein said subsequent time is in the range of two to five frames after said first time.

3. The computer implemented method of claim 1 wherein said movement information describes said movement of said camera at each scan line of said second image frame.

4. The computer implemented method of claim 1 wherein said warping comprises integrating said movement information to describe an orientation of said camera with respect to a global reference frame.

5. The computer implemented method of claim 1 further comprising:

slicing said crop polygon into at least two regions; and

applying a different warping to at least two of said at least two regions of said crop polygon.

6. An electronic system comprising:

an image sensor;

a motion sensing system configured to sense motion of said image sensor;

a processor coupled to said image sensor and to said motion sensing system;

a memory coupled to said processor, wherein said electronic system is configured to:

access a first image frame from a camera at a first time;

access a second image frame from said camera at a subsequent time;

identify a crop polygon around scene content common to said first image frame and said second image frame;

access movement information describing movement of said camera in an interval between said first time and said second time; and

warp said crop polygon, using said movement information, to remove motion distortions of said second image frame,

wherein said electronic system is configured to warp said crop polygon by defining a virtual camera that remains static when said movement of said camera is below a movement threshold,

wherein said electronic system is further configured to define a padding space around said crop polygon to adjust movement of a virtual camera, and

wherein said padding space comprises an inner region, corresponding to damping adjustment of said virtual camera, and an outer region, corresponding to increasing a rate of adjustment of said virtual camera.

7. The electronic system of claim 6 wherein said movement information describes said movement of said camera at each scan line of said second image frame.

8. The electronic system of claim 6 wherein said electronic system is further configured to integrate said movement information to describe an orientation of said camera with respect to a global reference frame.

9. The electronic system of claim 6 wherein said electronic system is further configured to define a virtual camera that remains static when said movement of said camera is relatively small.

10. The electronic system of claim 6 wherein said electronic system is further configured to:

slice said crop polygon into a plurality of regions; and

apply a different warping to at least two of said plurality of regions of said crop polygon.

11. An article of manufacture including a non-transitory computer readable medium having instructions stored thereon that, responsive to execution by an electronic system, cause said electronic system to perform operations comprising:

accessing a first image frame from a camera at a first time;

accessing a second image frame from said camera at a subsequent time;

identifying a crop polygon around scene content common to said first image frame and said second image frame;

accessing movement information describing movement of said camera in an interval between said first time and said second time; and

using said movement information, warping said crop polygon to remove motion distortions of said second image frame,

wherein said warping comprises defining a virtual camera that remains static when said movement of said camera is below a movement threshold,

defining a padding space around said crop polygon to adjust movement of said virtual camera;

damping movement of said virtual camera when said crop polygon is within an inner region of said padding space; and

accelerating said movement of said virtual camera when said crop polygon extends outside of said inner region of said padding space.

12. The article of manufacture of claim 11 wherein said movement information describes said movement of said camera at each scan line of said second image frame.

13. The article of manufacture of claim 11 wherein said warping further comprises integrating said movement information to describe an orientation of said camera with respect to a global reference frame.

14. The article of manufacture of claim 13 wherein said warping further comprises defining a virtual camera that remains static when said movement of said camera is relatively small.