System and method for stabilization of fisheye video imagery

1. A method of stabilizing fisheye video imagery, comprising:

selecting a reference image from a series of fisheye video imagery:

selecting a plurality of landmark points in the reference image;

tracking the plurality of landmark points in a new image from the series of fisheye video imagery;

computing, using a processor, a coordinate system mapping between the new image and the reference image using the tracked landmark points; and

resampling the new image to align with the reference image.

2. The method of claim 1 , wherein the step of computing includes:

transforming fisheye image coordinates of the reference image to a reference world space coordinates using an understanding of a camera lens;

transforming the tracked landmark points in the new image to a new image world space coordinates using the understanding of the camera lens; and

analyzing the plurality of landmark points in the reference world space coordinates and the tracked landmark points in the new image world space coordinates in pairs to define homography between a plane in reference world space and a plane in new image world space, the plurality of landmark points including four or more landmark points.

3. The method of claim 2 , wherein the step of computing further includes:

transforming pixel coordinates in the reference world space to the new image world space using the homography: and

transforming the transformed pixel coordinates to the fisheye image coordinates of the new image.

4. The method of claim 3 , wherein the resampling step includes:

copying data closest to the transformed pixel coordinates in the fisheye image coordinates of the new image to corresponding reference pixel coordinates in the fisheye image coordinates of the reference image.

5. A method of stabilizing fisheye video imagery, comprising:

scanning a reference fisheye image to obtain a pixel coordinate location (u, v); selecting a new fisheye image;

transforming the reference fisheye image coordinate location (u, v) to reference image's world space coordinate (x, y);

transforming the reference image's world space coordinate (x, y) to new image's world space coordinate ({circumflex over (x)}, ŷ);

transforming the new image's world space coordinates ({circumflex over (x)}, ŷ) to new image's fisheye space coordinate (û, {circumflex over (v)}); and

mapping new image data found closest to the new image's fisheye space coordinate (û, {circumflex over (v)}) to the reference fisheye image coordinate location (u, v).

6. The method of claim 5 , wherein the steps are performed for each pixel in the reference fisheye image and a new fisheye image.

7. The method of claim 5 , wherein the step of transforming the reference image's world space coordinate (x, y) to new image's world space coordinate ({circumflex over (x)}, ŷ) is performed using homography computed from tracked points in the reference fisheye image and the new fisheye image.

8. The method of claim 7 , further including:

tracking four or more landmark points in the reference fisheye image to the new fisheye image: and

computing homography that maps the tracked four or more landmark points in the reference fisheye image to the new fisheye image.

9. The method of claim 8 , wherein the tracking step is performed using Lucas-Kanade algorithm.

10. The method of claim 8 , wherein the tracking step is performed using an optical flow algorithm.

11. The method of claim 5 , wherein the step of mapping includes copying new-image data found closest to the new image's fisheye space coordinate (û, {circumflex over (v)}) to the reference fisheye image coordinate location (u, v).

12. A system for stabilizing fisheye video imagery, comprising:

a processor;

a module operable to execute on the processor and further operable to select a reference image and select a plurality of landmark points in the reference image, the module further operable to track the plurality of landmark points to a new image and compute a coordinate system mapping between the new image and the reference image using the tracked landmark points, the module further operable to resample the new image to align with the reference image.

13. The system of claim 12 , wherein the module is operable to compute the coordinate system mapping by:

transforming fisheye image coordinates of the reference image to a reference world space coordinates using an understanding of a camera lens;

transforming the tracked landmark points in the new image to a new image world space coordinates using the understanding of the camera lens; and

analyzing the plurality of landmark points in the reference world space coordinates and the tracked landmark points in the new image world space coordinates in pairs to define homography between a plane in reference world space and a plane in new image world space, the plurality of landmark points including four or more landmark points.

14. The system of claim 13 , wherein the module is operable to compute the coordinate system mapping further by:

transforming pixel coordinates in the reference world space to the new image world space using the homography: and

transforming the transformed pixel coordinates to the fisheye image coordinates of the new image.

15. The system of claim 14 , wherein the module is operable to resample the new image by:

copying data closest to the transformed pixel coordinates in the fisheye image coordinates of the new image to corresponding reference pixel coordinates in the fisheye image coordinates of the reference image.

16. A system for stabilizing fisheye video imagery, comprising:

a processor;

a module operable to scan a reference fisheye image to obtain a pixel coordinate location (u, v), the module further operable to select a new fisheye image, the module further operable to transform the reference fisheye image coordinate location (u, v) to reference image's world space coordinate (x, y), the module further operable to transform the reference image's world space coordinate (x, y) to new image's world space coordinate ({circumflex over (x)}, ŷ), the module further operable to transform the new image's world space coordinates ({circumflex over (x)}, ŷ) to new image's fisheye space coordinate (û, {circumflex over (v)}) and map data found closest to the new image's fisheye space coordinate (û, {circumflex over (v)}) to the reference fisheye image coordinate location (u, v).

17. The system of claim 16 , wherein the steps are performed for each pixel in the reference fisheye image and a new fisheye image.

18. The system of claim 16 , wherein the module is operable to transform the reference image's world space coordinate (x, y) to new image's world space coordinate ({circumflex over (x)}, ŷ) using homography computed from tracked points in the reference fisheye image and the new fisheye image.

19. The system of claim 18 , wherein the module is further operable to track four or more landmark points in the reference fisheye image to the new fisheye image; and

compute homography that maps the tracked four or more landmark points in the reference fisheye image to the new fisheye image.

20. The system of claim 16 , wherein the module is operable to map by copying data found closest to the new image's fisheye space coordinate (û, {circumflex over (v)}) to the reference fisheye image coordinate location (u, v).

21. A non-transitory program storage device readable by a machine, tangibly embodying a program of instructions executable by the machine to perform a method of stabilizing fisheye video imagery, comprising:

selecting a reference image;

selecting a plurality of landmark points in the reference image;

tracking the plurality of landmark points in a new image;

computing a coordinate system mapping between the new image and the reference image using the tracked landmark points; and

resampling the new image to align with the reference image.

22. The program storage device of claim 21 , wherein the step of computing includes:

transforming fisheye image coordinates of the reference image to a reference world space coordinates using an understanding of a camera lens;

transforming the tracked landmark points in the new image to a new image world space coordinates using the understanding of the camera lens; and

analyzing the plurality of landmark points in the reference world space coordinates and the tracked landmark points in the new image world space coordinates in pairs to define homography between a plane in reference world space and a plane in new image world space, the plurality of landmark points including four or more landmark points.

23. The program storage device of claim 22 , wherein the step of computing further includes:

transforming pixel coordinates in the reference world space to the new image world space using the homography; and

transforming the transformed pixel coordinates to the fisheye image coordinates of the new image.

24. The program storage device of claim 23 , wherein the resampling step includes:

copying data closest to the transformed pixel coordinates in the fisheye image coordinates of the new image to corresponding reference pixel coordinates in the fisheye image coordinates of the reference image.

25. A computer readable storage medium storing a program of instructions executable by a machine to perform a method of stabilizing fisheye video imagery, comprising:

scanning a reference fisheye image to obtain a pixel coordinate location (u, v);

selecting a new fisheye image;

transforming the reference fisheye image coordinate location (u, v) to reference image's world space coordinate (x, y);

transforming the reference image's world space coordinate (x, y) to new image's world space coordinate ({circumflex over (x)}, ŷ);

transforming the new image's world space coordinates ({circumflex over (x)}, ŷ) to new image's fisheye space coordinate (û, {circumflex over (v)}); and

mapping new image data found closest to the new image's fisheye space coordinate (û, {circumflex over (v)}) to the reference fisheye image coordinate location (u, v).

26. The computer readable storage medium of claim 25 , wherein the steps are performed for each pixel in the reference fisheye image and a new fisheye image.

27. The computer readable storage medium of claim 25 , wherein the step of transforming the reference image's world space coordinate (x, y) to new image's world space coordinate ({circumflex over (x)}, ŷ) is performed using homography computed from tracked points in the reference fisheye image and the new fisheye image.

28. The computer readable storage medium of claim 27 , further including:

tracking four or more landmark points in the reference fisheye image to the new fisheye image; and

computing homography that maps the tracked four or more landmark points in the reference fisheye image to the new fisheye image.

29. The computer readable storage medium of claim 28 , wherein the tracking step is performed using Lucas-Kanade algorithm.

30. The computer readable storage medium of claim 28 , wherein the tracking step is performed using an optical flow algorithm.

31. The computer readable storage medium of claim 25 , wherein the step of mapping includes copying new image data found closest to the new image's fisheye space coordinate (û, {circumflex over (v)}) to the reference fisheye image coordinate location (u, v).