Integration of optical area monitoring with industrial machine control
An industrial safety system is provided that integrates optical safety monitoring with machine control. The safety system includes an imaging sensor device supporting pixel array processing functions that allow time-of-flight (TOF) analysis to be performed on selected portions of the pixel array, while two-dimensional imaging analysis is performed on the remaining portions of the array, reducing processing load and response time relative to performing TOF analysis for all pixels of the array. The portion of the pixel array designated for TOF analysis can be pre-defined through configuration of the imaging sensor device, or can be dynamically selected based on object detection and classification by the two-dimensional imaging analysis. The imaging sensor device can also implement a number of safety and redundancy functions to achieve a high degree of safety integrity.
1. An optical safety system, comprising:
a memory that stores executable instructions; and
a processor, communicatively coupled to the memory, that executes or facilitates execution of the executable instructions to at least:
capture image data representing an industrial area;
classify a first object represented in the image data as industrial equipment and a second object represented in the image data as a human based on a first result of a first type of analysis performed on a first subset of pixels of the image data and a second result of a second type of analysis performed on a second subset of the pixels;
correlate the first result and the second result to yield correlated information relating to a relationship between the human and the industrial equipment; and
control an output signal that controls the industrial equipment based on the correlated information.
2. The optical safety system of claim 1 , wherein the first type of analysis comprises a two-dimensional (2D) analysis and the second type of analysis comprises a three-dimensional (3D) analysis.
3. The optical safety system of claim 2 , wherein the processor further executes or facilitates execution of the executable instructions to at least, for one or more pixel arrays of the image data, group pixels of the one or more pixel arrays to yield the second subset of the pixels.
4. The optical safety system of claim 3 , wherein the processor further executes or facilitates execution of the executable instructions to at least identify the second subset of the pixels based on a configuration profile that defines one or more portions of the one or more pixel arrays on which the 3D analysis is to be performed, and wherein the second subset of the pixels comprises one of a single contiguous group of pixels or multiple non-contiguous groups of pixels.
5. The optical safety system of claim 3 , wherein the processor further executes or facilitates execution of the executable instructions to at least select the second subset of the pixels based on classifications of the first object and the second object and locations of the first object and the second object determined based on the 2D analysis, and wherein the second subset of the pixels comprises one of a single contiguous group of pixels or multiple non-contiguous groups of pixels.
6. The optical safety system of claim 3 , wherein the processor further executes or facilitates execution of the executable instructions to at least:
generate three-dimensional image information for the industrial area based on aggregation of pixel-level distance information collected by one or more imaging sensor devices for the second subset of the pixels; and
perform the 3D analysis on the three-dimensional image information.
7. The optical safety system of claim 2 , wherein the 2D analysis comprises at least one of object recognition, object classification, determination of an x-y location of at least one of the first object or the second object, correlation of pixel groups within the image data determined to belong to a common object, human recognition, or facial recognition.
8. The optical safety system of claim 2 , wherein the correlated information comprises at least one of a three-dimensional location of the second object relative to the first object, a three-dimensional velocity of the second object relative to the first object, a three-dimensional acceleration of the second object relative to the first object, or a three-dimensional trajectory of the second object relative to the first object.
9. The optical safety system of claim 6 , wherein the processor further executes or facilitates execution of the executable instructions to at least:
estimate the first object and the second object as respective three-dimensional geometric objects within a three-dimensional space represented by the three-dimensional image data, wherein the geometric objects respectively encapsulate the first object and the second object,
determine a minimum distance between the geometric objects to yield an estimated distance between the first object and the second object; and
control the output signal based at least in part on the minimum distance.
10. A method for controlling industrial equipment, comprising:
recording, by a safety system comprising at least one processor, image data representing an industrial area;
classifying, by the safety system, a first object recorded in the image data as an industrial machine and a second object recorded in the image data as a human based on a first result of a first type of analysis performed on a first subset of pixels of the image data and a second result of a second type of analysis performed on a second subset of the pixels;
correlating, by the safety system, the first result and the second result to yield correlated information defining a relationship between the human and the industrial machine; and
controlling, by the safety system, an output signal that controls the industrial machine based on the correlated information.
11. The method of claim 10 , further comprising:
performing, as the first type of analysis, a two-dimensional (2D) analysis on the first subset of the pixels; and
performing, as the second type of analysis, a three-dimensional (3D) analysis on the second subset of the pixels.
12. The method of claim 10 , wherein the controlling comprises at least one of initiating a removal of power from the industrial machine or initiating a change of an operating mode of the industrial machine.
13. The method of claim 10 , further comprising, for one or more pixel arrays of the image data, grouping pixels of the one or more pixel arrays to yield the second subset of the pixels.
14. The method of claim 11 , further comprising identifying, by the safety system, the second subset of the pixels based on configuration profile data that defines one or more portions of the one or more pixel arrays on which the 3D analysis is to be performed, wherein the second subset of the pixels comprises one of a single contiguous group of pixels or multiple non-contiguous groups of pixels.
15. The method of claim 11 , further comprising:
determining, by the safety system, respective locations of the first object and the second object based on the first result of the 2D analysis; and
selecting, by the safety system, the second subset of the pixels based on the classifying and locations.
16. The method of claim 11 , further comprising:
generating, by the safety system, three-dimensional image information for the industrial area based on aggregation of pixel-level distance information collected by one or more imaging sensor devices for the second subset of the pixels; and
performing, by the safety system, the 3D analysis on the three-dimensional image information.
17. The method of claim 11 , further comprising performing, by the safety system as the 2D analysis, at least one of object recognition, object classification, determination of an x-y location of at least one of the first object or the second object, correlation of pixel groups within the image data determined to belong to a common object, human recognition, or facial recognition.
18. The method of claim 11 , wherein the correlating comprises determining, as the correlated information, at least one of a three-dimensional location of the second object relative to the first object, a three-dimensional velocity of the second object relative to the first object, a three-dimensional acceleration of the second object relative to the first object, or a three-dimensional trajectory for the second object relative to the first object.
19. A non-transitory computer-readable medium having stored thereon instructions that, in response to execution, cause a system comprising a processor to perform operations, the operations comprising:
collecting image data representing an industrial area;
classifying a first object recorded in the image data as industrial equipment and a second object recorded in the image data as a human based on a first result of a first type of analysis performed on a first subset of pixels of the image data and a second result of a second type of analysis performed on a second subset of the pixels;
generate correlated information based on one or more correlations between the first result and the second result, the correlated information defining a relationship between the human and the industrial equipment; and
controlling an output signal that controls the industrial equipment based on the correlated information.
20. The non-transitory computer-readable medium of claim 19 , the operations further comprising:
performing, as the first type of analysis, a two-dimensional analysis; and
performing, as the second type of analysis, a three-dimensional analysis.