Method and device for the estimation of car ego-motion from surround view images
A method and device for determining an ego-motion of a vehicle are disclosed. Respective sequences of consecutive images are obtained from a front view camera, a left side view camera, a right side view camera and a rear view camera and merged. A virtual projection of the images to a ground plane is provided using an affine projection. An optical flow is determined from the sequence of projected images, an ego-motion of the vehicle is determined from the optical flow and the ego-motion is used to predict a kinematic state of the car.
1. A method for determining an ego-motion of a vehicle, the method comprising:
recording a first sequence of consecutive images of a front view camera, a second sequence of consecutive images of a left side view camera, a third sequence of consecutive images of a right side view camera, and a fourth sequence of consecutive images of a rear view camera;
merging the first sequence of consecutive images, the second sequence of consecutive images, the third sequence of consecutive images, and the fourth sequence of consecutive images to obtain a sequence of merged images;
providing a virtual projection of the images of the sequence of merged images to a ground plane using an affine projection, thereby obtaining a sequence of projected images;
determining an optical flow, based on the sequence of projected images, the optical flow comprising motion vectors of target objects in the surroundings of the vehicle;
determining an ego-motion of the vehicle based on the optical flow; and
predicting a kinematic state of the vehicle based on the ego-motion;
wherein the determination of the ego-motion comprises deriving a current position vector of a target object and a current velocity relative to the target object from a previous position of the target object, a previous velocity relative to the target object and an angular velocity with respect to a rotation around an instantaneous center of curvature.
2. The method of claim 1 , wherein the determination of the ego-motion comprises:
applying a RANSAC procedure to the motion vectors.
3. The method of claim 1 , further comprising detecting image regions that correspond to objects that are not at a ground level and masking out/disregarding the detected image regions.
4. A method for determining an ego-motion of a vehicle, the method comprising:
recording a first sequence of consecutive images of a front view camera, a second sequence of consecutive images of a left side view camera, a third sequence of consecutive images of a right side view camera, and a fourth sequence of consecutive images of a rear view camera;
merging the first sequence of consecutive images, the second sequence of consecutive images, the third sequence of consecutive images, and the fourth sequence of consecutive images to obtain a sequence of merged images;
providing a virtual projection of the images of the sequence of merged images to a ground plane using an affine projection, thereby obtaining a sequence of projected images;
determining an optical flow, based on the sequence of projected images, the optical flow comprising motion vectors of target objects in the surroundings of the vehicle;
determining an ego-motion of the vehicle based on the optical flow;
deriving an angular velocity of the vehicle around an instantaneous center of curvature from a wheel speed and a steering angle using an Ackermann steering model;
merging the determined ego-motion and the angular velocity of the vehicle in an incremental pose update; and
predicting a kinematic state of the vehicle based on the ego-motion.
5. The method of claim 4 , wherein the determination of the ego-motion comprises:
applying a RANSAC procedure to the motion vectors.
6. The method of claim 4 , further comprising detecting image regions that correspond to objects that are not at a ground level and masking out/disregarding the detected image regions.
7. A method for determining an ego-motion of a vehicle, the method comprising:
recording a first sequence of consecutive images of a front view camera, a second sequence of consecutive images of a left side view camera, a third sequence of consecutive images of a right side view camera, and a fourth sequence of consecutive images of a rear view camera;
merging the first sequence of consecutive images, the second sequence of consecutive images, the third sequence of consecutive images, and the fourth sequence of consecutive images to obtain a sequence of merged images;
providing a virtual projection of the images of the sequence of merged images to a ground plane using an affine projection, thereby obtaining a sequence of projected images;
determining an optical flow, based on the sequence of projected images, the optical flow comprising motion vectors of target objects in the surroundings of the vehicle;
determining an ego-motion of the vehicle by
deriving vector of ego-motion from the motion vectors of the optical flow, and
applying a prediction filter to the vector of ego-motion for predicting a future position of the vehicle; and
predicting a kinematic state of the vehicle based on the ego-motion.
8. The method of claim 7 , wherein an input to the prediction filter is derived from one or more angular velocities.
9. The method of claim 7 , further comprising detecting image regions that correspond to objects that are not at a ground level and masking out/disregarding the detected image regions.