Depth-sensing computer vision system
In various embodiments, systems and methods for acquiring depth images utilize an architecture suited to safety-rated applications, and may include more than sensor (such as time-of-flight sensors) operating along different optical paths and a comparison module for ensuring proper sensor operation. Error metrics may be associated with pixel-level depth values for purposes of allowing safe control based on imperfectly known depths.
1. An image-processing system comprising:
first and second 3D sensors each for generating an output array of pixelwise values indicative of distances to objects within a field of view of the sensor, the fields of view of the first and second 3D sensors overlapping along separate optical paths, the objects including a robot and a person;
first and second depth-compute engines, executable by at least one processor, for processing successive resulting output arrays originating from, respectively, the first and second 3D sensors, into pixelwise arrays of depth values; and
a comparison unit, executable by at least one processor, for detecting pixelwise differences in depth between corresponding processed resulting output arrays originating substantially simultaneously from the first and second 3D sensors; and
a control processor configured to (i) computationally generate a 3D safety envelope surrounding the robot, (ii) control an operating speed of the robot based at least in part on a detected distance between the 3D safety envelope and the person, and (iii) adjust the detected distance based on the detected pixelwise differences in depth.
2. The system of claim 1 , wherein the detected distance is adjusted by expanding or contracting the 3D safety envelope.
3. The system of claim 1 , wherein the detected distance is adjusted by expanding or contracting space represented as being occupied by the person.
4. The system of claim 1 , wherein the depth-compute engines operate in a pipelined fashion so as to begin processing a new resulting output array prior to completing processing of a previous resulting output array.
5. The system of claim 1 , wherein the 3D sensors are time-of-flight (ToF) sensors.
6. The system of claim 1 , further comprising at least one temperature sensor, the processor being responsive to at least one temperature sensor and further adjusting the detected distance based thereon.
7. The system of claim 1 , further comprising at least one humidity sensor, the processor being responsive to the at least one temperature sensor and further adjusting the detected distance based thereon.
8. The system of claim 1 , wherein larger detected pixelwise differences in depth result in larger downward adjustment of the detected distance.
9. A method of controlling a robot in a 3D workspace space, the method comprising the steps of:
disposing first and second 3D sensors in or proximate to the workspace;
causing each of the sensors to generate an output array of pixelwise values indicative of distances to objects in the 3D space and within a field of view of the sensor, the fields of view of the first and second 3D sensors overlapping along separate optical paths, the objects including the robot and a person;
computationally processing successive resulting output arrays originating from, respectively, the first and second 3D sensors, into pixelwise arrays of depth values;
detecting pixelwise differences in depth between corresponding processed resulting output arrays originating substantially simultaneously from the first and second 3D sensors;
computationally generating a 3D safety envelope surrounding the robot;
controlling an operating speed of the robot based at least in part on a detected distance between the 3D safety envelope and the person; and
adjusting the detected distance based on the detected pixelwise differences in depth.
10. The method of claim 9 , wherein the detected distance is adjusted by expanding or contracting the 3D safety envelope.
11. The method of claim 9 , wherein the detected distance is adjusted by expanding or contracting space represented as being occupied by the person.
12. The method of claim 9 , wherein the 3D sensors are time-of-flight (ToF) sensors.
13. The method of claim 9 , further comprising adjusting the detected distance based on a sensed temperature.
14. The method of claim 9 , further comprising adjusting the detected distance based on a sensed humidity.
15. The method of claim 9 , wherein larger detected pixelwise differences in depth result in larger downward adjustment of the detected distance.