Imaging system and method with ego motion detection
An imaging system and method for a vehicle is provided, and includes an imager configured to image a scene external and forward of the vehicle and to generate image data corresponding to the acquired images. A controller is configured to receive the image data and analyze an optical flow between successive image frames to compute a relative motion between the imager and the imaged scene, wherein the optical flow includes a pattern of apparent motion of objects of interest in the successive image frames.
1. An imaging system of a vehicle, comprising:
an imager configured to acquire images of a scene external and forward of the vehicle and to generate image data corresponding to the acquired images; and
a controller configured to:
detect a plurality of objects of interest in the image data, wherein each object of interest comprises one of a moving object and a stationary object;
analyze an optical flow between successive image frames, the optical flow including a pattern of apparent motion of the objects of interest caused by a relative motion between the imager and the scene;
compute a vertical position value based on a change in vertical position for each object of interest appearing in the successive image frames;
compute a vertical motion value based on a change in vertical position for only the objects of interest appearing in the successive image frames as having a common apparent motion in a vertical direction;
determine the relative motion in the vertical direction based on a relationship between the vertical position value and the vertical motion value;
determine the relative motion in a horizontal direction based on a vehicle heading; and
based on the relative motion determined in the vertical and horizontal directions, correct for apparent motion and generate a control signal provided to a vehicle device.
2. The imaging system of claim 1 , wherein the controller computes the vertical position value by determining an average vertical position of all objects of interest appearing in each of the successive image frames and taking the difference between the average vertical positions.
3. The imaging system of claim 2 , wherein the controller computes the vertical motion value by averaging a sum of the changes in vertical position for only the objects of interest appearing in the successive image frames as having the common apparent motion in the vertical direction.
4. The imaging system of claim 3 , wherein the controller is configured to compute a weighted average between the vertical position value and the vertical motion value, wherein the weighted average indicates the relative motion between the imager and the scene in the vertical direction and is used by the controller to correct apparent motion in the vertical direction.
5. The imaging system of claim 1 , wherein the controller is configured to compute the vehicle heading based on a yaw signal of the vehicle.
6. The imaging system of claim 5 , wherein the controller is configured to take a time average of the vehicle heading, wherein the time average of the vehicle heading indicates the relative motion between the imager and the scene in the horizontal direction and is used by the controller to correct apparent motion in the horizontal direction.
7. The imaging system of claim 1 , integrated in a rearview mirror assembly.
8. An imaging method of a vehicle, comprising the steps of:
using an imager to acquire images of a scene external and forward of the controlled vehicle and generating image data corresponding to the acquired images;
providing a controller for:
detecting a plurality of objects of interest in the image data, wherein each object of interest comprises one of a moving object and a stationary object;
analyzing an optical flow between successive image frames, the optical flow including a pattern of apparent motion of the objects of interest caused by a relative motion between the imager and the scene;
computing a vertical position value based on a change in vertical position for each object of interest appearing in the successive image frames;
computing a vertical motion value based on a change in vertical position for only the objects of interest appearing in the successive image frames as having a common apparent motion in a vertical direction;
determining the relative motion in the vertical direction based on a relationship between the vertical position value and the vertical motion value;
determining the relative motion in a horizontal direction based on a vehicle heading; and
based on the relative motion determined in the vertical and horizontal directions, correcting for apparent motion and generating a control signal provided to a vehicle device.
9. The imaging method of claim 8 , wherein the step of computing the vertical position value comprises determining an average vertical position of all objects of interest appearing in each of the successive image frames and taking the difference between the average vertical positions.
10. The imaging method of claim 9 , wherein the step of computing the vertical motion value comprises averaging a sum of the changes in vertical position for only the objects of interest appearing in the successive image frames as having the common apparent motion in the vertical direction.
11. The imaging method of claim 10 , wherein the step of determining the relative motion in the vertical direction further comprises computing a weighted average between the vertical position value and the vertical motion value, wherein the weighted average indicates the relative motion between the imager and the scene in the vertical direction and is used by the controller to correct apparent motion in the vertical direction.
12. The imaging method of claim 8 , wherein the step of determining the relative motion in the horizontal direction comprises computing the vehicle heading based on a yaw signal of the vehicle.
13. The imaging method of claim 12 , wherein the step of determining the relative motion in the horizontal direction further comprises taking a time average of the vehicle heading, wherein the time average of the vehicle heading indicates the relative motion between the imager and the scene in the horizontal direction and is used by the controller to correct apparent motion in the horizontal direction.
14. A non-transitory computer-readable medium having stored thereon software instructions that, when executed by a processor, comprise the steps of:
using an imager to acquire images of a scene external and forward of the controlled vehicle and generating image data corresponding to the acquired images;
detecting a plurality of objects of interest in the image data, wherein each object of interest comprises one of a moving object and a stationary object;
analyzing an optical flow between successive image frames, the optical flow including a pattern of apparent motion of the objects of interest caused by a relative motion between the imager and the scene;
computing a vertical position value based on a change in vertical position for each object of interest appearing in the successive image frames;
computing a vertical motion value based on a change in vertical position for only the objects of interest appearing in the successive image frames as having a common apparent motion in a vertical direction;
determining the relative motion in the vertical direction based on a relationship between the vertical position value and the vertical motion value;
determining the relative motion in a horizontal direction based on a vehicle heading; and
based on the relative motion determined in the vertical and horizontal directions, correcting for apparent motion and generating a control signal provided to a vehicle device.
15. The non-transitory computer-readable medium of claim 14 , wherein the step of computing the vertical position value comprises determining an average vertical position of all objects of interest appearing in each of the successive image frames and taking the difference between the average vertical positions.
16. The non-transitory computer-readable medium of claim 15 , wherein the step of computing the vertical motion value comprises averaging a sum of the changes in vertical position for only the objects of interest appearing to have in the successive image frames as having the common apparent motion in the vertical direction.
17. The non-transitory computer-readable medium of claim 16 , wherein the step of determining the relative motion in the vertical direction further comprises computing a weighted average between the vertical position value and the vertical motion value, wherein the weighted average indicates the relative motion between the imager and the scene in the vertical direction and is used by the controller to correct for apparent motion in the vertical direction.
18. The non-transitory computer-readable medium of claim 14 , wherein the step of determining the relative motion in the horizontal direction comprises computing the vehicle heading based on a yaw signal of the vehicle.
19. The non-transitory computer-readable medium of claim 18 , wherein the step of determining the relative motion in the horizontal direction further comprises taking a time average of the vehicle heading, wherein the time average of the vehicle heading indicates the relative motion between the imager and the scene in the horizontal direction and is used by the controller to correct for apparent motion in the horizontal direction.