System and method for egomotion estimation

1. A method for determining egomotion of a system comprising a first camera and a second camera, the method comprising:

determining a first dense disparity map that matches features between a first and second image, wherein the first and second images were acquired at a first time by the first and second camera, respectively;

determining a second dense disparity map that matches features between a third and fourth image, wherein the third and fourth images were acquired at a second time by the first and second camera, respectively;

determining a first optic flow map that matches features between the first and third image;

determining a second optic flow map that matches features between the second and fourth image;

identifying odometry features shared between the first, second, third, and fourth image based on the first and second dense disparity map and the first and second optic flow map; and

determining the egomotion of the system based on the odometry features.

2. The method of claim 1 , wherein the odometry features are identified using a cyclic consistency criterion comprising matching a candidate feature from the first image to a feature in the fourth image along each of a first and second path determined from the first and second dense disparity maps and the first and second optic flow maps.

3. The method of claim 2 , wherein the first path comprises a first optic flow vector from the first optic flow map and a first disparity vector from the second disparity map; wherein the second path comprises a second disparity vector from the first disparity map and a second optic flow vector from the second optic flow map; and wherein the first and second optic flow vectors and the first and second disparity vectors are associated with the candidate feature.

4. The method of claim 1 , further comprising sparsifying the odometry features, wherein the egomotion is determined using the sparsified odometry features.

5. The method of claim 4 , wherein sparsifying the odometry features comprises:

segmenting at least one of the first, second, third, or fourth image into transitory segments and permanent segments; and

excluding features associated with transitory segments from the odometry features.

6. The method of claim 5 , further comprising estimating an egomotion of objects in the transitory segments based on the egomotion of the system.

7. The method of claim 4 , wherein sparsifying the odometry features comprises:

receiving an IMU dataset; and

for each of the odometry features:

predicting a feature position in the third image based on the corresponding feature position in the first image and the IMU dataset;

comparing the predicted feature position to an actual feature position determined based on an optic flow vector associated with the feature; and

when the actual and predicted feature positions are different, removing said odometry feature from the odometry features.

8. The method of claim 1 , further comprising:

determining system yaw and roll based on accelerometer measurements;

determining linear acceleration based on the accelerometer measurements; and

determining the egomotion using a visual-inertial odometry solver based on the system yaw, roll, and linear acceleration.

9. The method of claim 8 , wherein the system yaw and roll are determined based on accelerometer measurements using an extrinsic calibration of an accelerometer pose.

10. The method of claim 1 , wherein the determined egomotion is consistent with a motion constraint of the system.

11. The method of claim 1 , further comprising determining a pixel hash for each pixel in the first and second image; wherein determining the first disparity map comprises iteratively:

determining a set of candidate correspondence vectors for each pixel in the first image based on correspondence vectors of neighboring pixels, wherein each correspondence vector identifies a corresponding pixel in the second image that is paired with the respective pixel; and

selecting a candidate correspondence vector from the set as the correspondence vector of said each pixel based on a cost value determined between the pixel hashes of said each pixel and the corresponding pixels.

12. The method of claim 1 , wherein each odometry feature is a pixel.

13. A non-transitory computer-readable storage medium storing instructions that, when executed by a processor, cause the processor to perform a method comprising:

determining a first dense disparity map matching features between a first and second image, acquired at a first time, by a first and second camera, respectively;

determining a second dense disparity map matching features between a third and fourth image, acquired at a second time, by the first and second camera, respectively;

determining a first optic flow map matching features between the first and third image;

determining a second optic flow map matching features between the second and fourth image;

identifying odometry features shared between the first, second, third, and fourth image based on the first and second dense disparity map and the first and second optic flow map; and

determining the egomotion of a system, comprising the first and second cameras, based on the odometry features.

14. The computer readable storage medium of claim 13 , wherein the odometry features are identified using a cyclic consistency criterion comprising matching a candidate feature from the first image to a feature in the fourth image along each of a first and second path.

15. The computer readable storage medium of claim 14 , wherein the first path comprises a first optic flow vector from the first optic flow map and a first disparity vector from the second disparity map; wherein the second path comprises a second disparity vector from the first disparity map and a second optic flow vector from the second optic flow map; and wherein the first and second optic flow vectors and the first and second disparity vectors are associated with the candidate feature.

16. The computer readable storage medium of claim 13 , further storing computer program instructions for sparsifying the odometry features, wherein the egomotion is determined using the sparsified odometry features.

17. The computer readable storage medium of claim 16 , wherein sparsifying the odometry features comprises:

segmenting at least one of the first, second, third, or fourth image into transitory segments and permanent segments; and

excluding features associated with transitory segments of the segmented image from the odometry features.

18. The computer readable storage medium of claim 16 , wherein sparsifying the odometry features comprises:

receiving an IMU dataset; and

for each of the odometry features:

predicting a feature position in the third image based on the corresponding feature position in the first image and the IMU dataset;

comparing the predicted feature position to an actual feature position determined based on an optic flow vector associated with the feature; and

when the actual and predicted feature positions are different, removing said odometry feature from the odometry features.

19. The computer readable storage medium of claim 13 , further storing computer program instructions for:

determining system yaw and roll based on accelerometer measurements;

determining system yaw, roll, and pitch based on gyroscope measurements;

determining linear acceleration based on the accelerometer measurements; and

determining the egomotion using a visual-inertial odometry measurements based on the system yaw and roll from the accelerometer and gyroscope measurements, the pitch, and the linear acceleration.

20. The computer readable storage medium of claim 19 , wherein the system yaw and roll are determined based on accelerometer measurements using an extrinsic calibration of an accelerometer pose.