Automatic vision sensor orientation
Implementations are described herein are directed to reconciling disparate orientations of multiple vision sensors deployed on a mobile robot (or other mobile vehicle) by altering orientations of the vision sensors or digital images they generate. In various implementations, this reconciliation may be performed with little or no ground truth knowledge of movement of the robot. Techniques described herein also avoid the use of visual indicia of known dimensions and/or other conventional tools for determining vision sensor orientations. Instead, techniques described herein allow vision sensor orientations to be determined and/or reconciled using less resources, and are more scalable than conventional techniques.
1. A method implemented using one or more processors, comprising:
obtaining a first sequence of digital images captured by a first vision sensor integral with a robot while the robot moves along a trajectory;
obtaining a second sequence of digital images captured by a second vision sensor integral with the robot while the robot moves along the trajectory;
analyzing two or more digital images of the first sequence to identify a first gradient field associated with the first sequence of digital images;
analyzing two or more digital images of the second sequence to identify a second gradient field associated with the second sequence of digital images;
reconciling any difference between the first and second gradient fields by altering an orientation of:
one of the first and second vision sensors, or
one or more digital images of the first or second sequence of digital images.
2. The method of claim 1 , wherein the altering comprises rotating one or more digital images of the first or second sequence of digital images so that the first and second gradient fields are aligned.
3. The method of claim 1 , wherein the altering comprises rotating the first or second vision sensor so that so that the first and second gradient fields of subsequent digital images captured by the first and second sensors are aligned.
4. The method of claim 1 , wherein the first vision sensor comprises an RGB camera.
5. The method of claim 4 , wherein the second vision sensor comprises an RGB camera.
6. The method of claim 4 , wherein the second vision sensor comprises a 2.5D camera.
7. The method of claim 1 , further comprising identifying the first vision sensors as having a preferred orientation, and the altering comprises:
rotating the second vision sensor so that so that the second gradient field of subsequent digital images captured by the second vision sensor are aligned with the first gradient field; or
rotating subsequent digital images captured by the second vision sensor to be aligned with the first gradient field.
8. The method of claim 7 , wherein the first vision sensor is identified as having the preferred orientation based on the first gradient field being aligned with the trajectory of the robot.
9. The method of claim 7 , wherein the first vision sensor is identified as having the preferred orientation based on odometry or global positioning system (GPS) data.
10. A robot comprising first and second vision sensors, one or more processors, and memory storing instructions that, in response to execution of the instructions by the one or more processors, cause the one or more processors to:
obtain a first sequence of digital images captured by the first vision sensor while the robot moves along a trajectory;
obtain a second sequence of digital images captured by the second vision sensor while the robot moves along the trajectory;
analyze two or more digital images of the first sequence to identify a first gradient field associated with the first sequence of digital images;
analyze two or more digital images of the second sequence to identify a second gradient field associated with the second sequence of digital images;
reconcile any difference between the first and second gradient fields by altering an orientation of:
one of the first and second vision sensors, or
one or more digital images of the first or second sequence of digital images.
11. The robot of claim 10 , wherein the altering comprises rotating one or more digital images of the first or second sequence of digital images so that the first and second gradient fields are aligned.
12. The robot of claim 10 , wherein the altering comprises rotating the first or second vision sensor so that so that the first and second gradient fields of subsequent digital images captured by the first and second sensors are aligned.
13. The robot of claim 10 , wherein the first vision sensor comprises an RGB camera.
14. The robot of claim 13 , wherein the second vision sensor comprises an RGB camera.
15. The robot of claim 13 , wherein the second vision sensor comprises a 2.5D camera.
16. The robot of claim 10 , further comprising instructions to:
identify the first vision sensors as having a preferred orientation; and
rotate the second vision sensor so that so that the second gradient field of subsequent digital images captured by the second vision sensor are aligned with the first gradient field; or
rotate subsequent digital images captured by the second vision sensor to be aligned with the first gradient field.
17. The robot of claim 16 , wherein the first vision sensor is identified as having the preferred orientation based on the first gradient field being aligned with the trajectory of the robot.
18. The method of claim 16 , wherein the first vision sensor is identified as having the preferred orientation based on odometry or global positioning system (GPS) data.
19. At least one non-transitory computer-readable medium comprising instructions that, in response to execution of the instructions by one or more processors, cause the one or more processors to perform the following operations:
obtaining a first sequence of digital images captured by a first vision sensor integral with a robot while the robot moves along a trajectory;
obtaining a second sequence of digital images captured by a second vision sensor integral with the robot while the robot moves along the trajectory;
analyzing two or more digital images of the first sequence to identify a first gradient field associated with the first sequence of digital images;
analyzing two or more digital images of the second sequence to identify a second gradient field associated with the second sequence of digital images;
reconciling any difference between the first and second gradient fields by altering an orientation of:
one of the first and second vision sensors, or
one or more digital images of the first or second sequence of digital images.
20. The at least one non-transitory computer-readable medium of claim 19 , further comprising instructions for identifying the first vision sensors as having a preferred orientation, and the altering comprises:
rotating the second vision sensor so that so that the second gradient field of subsequent digital images captured by the second vision sensor are aligned with the first gradient field; or
rotating subsequent digital images captured by the second vision sensor to be aligned with the first gradient field.