System and method for distorted camera image correction

1. A method for processing a sequence of images, comprising:

obtaining, from an optical sensor, at least two images of the sequence of images,

determining an image warping function that at least partially compensates a distortion in at least one of the two images,

applying the determined image warping function to the at least one image, at least partially compensating the distortion by modifying the at least one image, and

calculating and outputting an optical flow as a displacement vector field from the modified at least one image and another image of the at least two images,

wherein the optical flow is corrected by subtracting shifts introduced due to the warping function application.

2. The method according to claim 1 , further comprising a step of detecting whether one of the at least two images includes a distortion.

3. The method according to claim 1 , wherein the optical flow is calculated by matching patches of one image to the other.

4. The method according to claim 1 , wherein the warping function is determined from assumptions on the scene that is recorded by the optical sensor and the knowledge about movement of a visual system.

5. The method according to claim 1 , wherein the warping function is determined from motion information supplied by a sensor of a visual system and indicative of a motion the distortion results from.

6. The method according to claim 1 , wherein the distortion comprises a motion distortion or a lens distortion.

7. The method according to claim 1 , wherein more than one image warping function is applied to the at least one image including the distortion.

8. The method according to claim 1 , wherein the image warping function is applied to one of the at least two images.

9. The method according to claim 1 , wherein the method is performed more than once, each time with a different image warping function on the same images and the optical flow results are combined into one optical flow.

10. The method according to claim 1 , wherein in addition to the calculation of the optical flow from the images with application of the image warping function, calculating a second optical flow from the images without application of the image warping function and combining both optical flow results.

11. The method according to claim 1 , wherein the image warping function is determined from a previous optical flow calculation.

12. The method according to claim 1 , wherein the image warping function is a predetermined and fixed image warping function, an affine warping function, or a homography.

13. The method according to claim 1 , wherein objects and/or moving objects in the images are detected based on the optical flow.

14. A sensor system comprising at least an optical sensor configured to supply a sequence of images, the system comprising:

means for obtaining from the optical sensor, at least two images of the sequence of images,

means for determining an image warping function that at least partially compensates a distortion in at least one of the two images,

means for applying the determined image warping function to the at least one image, and for at least partially compensating the distortion by modifying the at least one image, and

means for calculating and outputting, an optical flow as a displacement vector field from the modified at least one image and another image of the at least two images,

wherein the optical flow is corrected by subtracting shifts introduced due to the warping function application.

15. The system of claim 14 , wherein the means for applying is configured to calculate and to apply, in case the at least one image is distorted, the at least one image warping function to the image including the distortion, the image warping function being calculated to compensate the distortion and/or comprising a motion sensor indicating motion of the optical sensor.

16. A sensor system comprising at least one optical sensor configured to obtain a sequence of images, the sensor system comprising:

an obtaining device for obtaining, from the optical sensor, at least two images of the sequence of images;

a determining device configured to determine an image warping function that at least partially compensates a distortion in at least one of the two images;

an applying device configured to apply the determined image warping function to the at least one image, and configured to at least partially compensate the distortion by modifying the at least one image; and

a calculating and outputting device configured to calculate and output an optical flow as a displacement vector field from the at least one modified image and another image of the at least two images,

wherein the optical flow is corrected by subtracting shifts introduced due to the warping function application.

17. A land, air, sea or space vehicle equipped with the sensor system according to claim 14 .

18. The vehicle according to claim 17 , wherein the vehicle comprises a robot or a motorcycle, a scooter, other 2-wheeled vehicle, a passenger car or a lawn mower.

19. A vehicle driver assistance system including the sensor system according to claim 14 .

20. A computer program product embodied on a non-transitory computer-readable medium, said computer program product comprising code that, when executed on a computer, controls the computer to perform the method according to claim 1 .

21. The method according to claim 9 , wherein the one optical flow comprises a single displacement vector field.