System and method for multi-camera visual odometry
A visual odometry system and method for a fixed or known calibration of an arbitrary number of cameras in monocular configuration is provided. Images collected from each of the cameras in this distributed aperture system have negligible or absolutely no overlap. The relative pose and configuration of the cameras with respect to each other are assumed to be known and provide a means for determining the three-dimensional poses of all the cameras constrained in any given single camera pose. The cameras may be arranged in different configurations for different applications and are made suitable for mounting on a vehicle or person undergoing general motion. A complete parallel architecture is provided in conjunction with the implementation of the visual odometry method, so that real-time processing can be achieved on a multi-CPU system.
1. A method for tracking a three-dimensional pose based on video captured in a distributed aperture having a multi-camera configuration, comprising the steps of:
generating a plurality of pose hypotheses in each camera of said multi-camera configuration;
transferring said plurality of pose hypotheses generated in each camera to all cameras in said multi-camera configuration;
scoring each of said plurality of pose hypotheses generated in and transferred to each camera;
determining a cumulative score for each of said plurality of pose hypotheses generated in and transferred to each camera;
discarding least scoring pose hypotheses in each camera; and
executing a pose refinement on a best scoring pose hypothesis remaining in each camera.
2. The method of claim 1 , wherein said plurality of pose hypotheses generated in each camera are based on randomly selected point correspondences from a plurality of point correspondences generated in response to features detected and matched in video frames.
3. The method of claim 1 , wherein transferring said plurality of pose hypotheses provides each camera with all pose hypotheses generated in all cameras of said multi-camera configuration.
4. The method of claim 1 , wherein said cumulative score is determined for a particular pose hypothesis by combining the scores from each camera associated with said pose hypothesis.
5. The method of claim 1 , wherein discarding of hypotheses further comprises discarding a least scoring half of pose hypotheses in each camera.
6. The method of claim 5 , wherein discarding said least scoring half of pose hypotheses is reiterated until there is a single best pose hypothesis remaining in each camera.
7. The method of claim 1 , wherein executing said pose refinement further comprises:
transferring said best scoring pose hypothesis in each camera to all cameras in said multi-camera configuration;
polishing said best scoring pose hypotheses in each camera;
scoring each of said polished best scoring pose hypotheses in each camera;
determining a cumulative score for each of said polished best scoring pose hypothesis;
selecting one of said polished best scoring pose hypothesis having a best said cumulative score from each camera; and
storing said polished best scoring pose hypothesis with said best said cumulative score in a camera of said multi-camera configuration where said polished best scoring pose hypothesis with said best said cumulative score originated.
8. The method of claim 7 , further comprising selecting one of said polished best scoring pose hypothesis with said best said cumulative score stored in corresponding said originating camera, wherein said selection is made from said camera having said polished best scoring pose hypothesis stored therein with a highest said cumulative score compared to all of said best said cumulative scores associated with each of said cameras.
9. The method of claim 8 , further comprising applying said highest cumulative scoring pose hypothesis to entire said distributed aperture having said multi-camera configuration.
10. The method of claim 1 , further comprising stitching poses using a scale factor obtained from each camera and evaluated in all cameras, wherein said scale factor determined to have a best scoring cumulative score is used for entire said distributed aperture having said multi-camera configuration.
11. A distributed aperture system for tracking a three-dimensional pose based on video captured using a multi-camera configuration, comprising:
means for generating a plurality of pose hypotheses in each camera of said multi-camera configuration;
means for transferring said plurality of pose hypotheses generated in each camera to all cameras in said multi-camera configuration;
means for scoring each of said plurality of pose hypotheses generated in and transferred to each camera;
means for determining a cumulative score for each of said plurality of pose hypotheses generated in and transferred to each camera;
means for discarding least scoring pose hypotheses in each camera; and
means for executing a pose refinement on a best scoring pose hypothesis remaining in each camera.
12. A non-transitory computer readable medium having stored thereon a plurality of instructions, the plurality of instructions including instructions which, when executed by a processor, cause the processor to perform the steps of a method for tracking a three-dimensional pose based on video captured in a distributed aperture having a multi-camera configuration, comprising the steps of:
generating a plurality of pose hypotheses in each camera of said multi-camera configuration;
transferring said plurality of pose hypotheses generated in each camera to all cameras in said multi-camera configuration;
scoring each of said plurality of pose hypotheses generated in and transferred to each camera;
determining a cumulative score for each of said plurality of pose hypotheses generated and transferred to each camera;
discarding least scoring pose hypotheses in each camera; and
executing a pose refinement on a best scoring pose hypothesis remaining in each camera.