Apparatus and method for detecting worker using thermal imaging based on artificial intelligence
In an embodiment, an apparatus may include an image receiver configured to receive a thermal image from a thermal imaging camera, a worker detector configured to detect a worker from the received thermal image by using an artificial intelligence model, a hazard detector configured to detect a hazard based on whether the detected worker has entered a preset hazardous area in the thermal image, and a hazard controller configured to, in response to the hazard being detected, send a hazard notification and stop operation of a work machine in the hazardous area, where a position of the detected worker can be estimated in pixel units.
1 . An apparatus comprising:
one or more processors; and
one or more memory devices storing computer-readable instructions that, when executed by the one or more processors, enable the one or more processors to:
receive a thermal image from a thermal imaging camera;
detect a human being from the received thermal image by using an artificial intelligence model, wherein a position of the detected human being is estimated in pixel units, including to generate a temperature map from the thermal image through an equation of
T
(
x
,
y
)
=
(
I
(
x
,
y
)
T
p
4
I
p
)
1
4
-
273.15
wherein Tp is 310.35K, Ip is an average pixel value of a worker area in the thermal image, I(x, y) is a pixel value at a pixel position (x, y) in the thermal image, and T(x, y) is a temperature value calculated at a pixel position (x, y);
detect a hazard based on whether the detected human being has entered a preset hazardous area in the thermal image; and
based on the hazard being detected, send a hazard notification and stop operation of a work machine in the hazardous area.
2 . The apparatus of claim 1 , wherein the instructions further enable the one or more processors to receive a color image corresponding to the thermal image.
3 . The apparatus of claim 2 , wherein the instructions further enable the one or more processors to:
extract multi-scale features from the color image by using a backbone;
extract enhanced single-level features based on the multi-scale features through an encoder; and
generate a mask feature in a mask branch, and generate an instance feature in an instance branch based on an input feature through a decoder by using the enhanced single-level feature as the input feature.
4 . The apparatus of claim 3 , wherein the instructions further enable the one or more processors to identify an area having a temperature range of top 40% among temperature ranges in the temperature map and determine the identified area as the worker area, and generate a temperature mask by selecting the worker area.
5 . The apparatus of claim 4 , wherein the instructions further enable the one or more processors to generate a mask kernel by inputting a result value according to an element-wise multiplication of the temperature mask and the input feature of the instance branch to an instance activation map.
6 . The apparatus of claim 5 , wherein the instructions further enable the one or more processors to generate a segmentation mask configured to indicate the human being through an element-wise multiplication of the mask kernel and the mask feature.
7 . The apparatus of claim 6 , wherein the instructions further enable the one or more processors to:
determine that the hazard is detected based on a human being mask indicating the human being among the segmentation mask has entered the hazardous area, and
determine that the human being mask has entered the hazardous area based on at least 10% of an area of the human being mask overlapping the hazardous area.
8 . The apparatus of claim 3 , wherein the instructions further enable the one or more processors to, based on the thermal image including a temperature value in pixel units, identify an area in the thermal image whose average temperature is 28° C. to 38° C. and generate a temperature mask.
9 . The apparatus of claim 8 , wherein the instructions further enable the one or more processors to:
generate a mask kernel by inputting a result value according to an element-wise multiplication of the temperature mask and the input feature of the instance branch to an instance activation map, and
generate a segmentation mask configured to indicate the human being through an element-wise multiplication of the mask kernel and the mask feature.
10 . The apparatus of claim 1 , wherein the instructions further enable the one or more processors to set the hazardous area in the thermal image through a deep learning-based hazardous area recommendation model.
11 . A method for detecting a human being using thermal imaging, the method comprising:
receiving a thermal image from a thermal imaging camera;
detecting the human being from the received thermal image by using an artificial intelligence model;
estimating in pixel units the detected human being in the thermal image;
detecting a hazard based on whether the human being has entered a preset hazardous area in the thermal image;
sending a hazard notification and stopping operation of a work machine in the hazardous area based on the hazard being detected,
wherein the detecting the human being further comprises:
in case the thermal image includes a temperature in pixel units, identifying an area whose average temperature is 28° C. to 38° C. and selecting the identified area as a worker area, and
in case the thermal image does not include the temperature in pixel units, generating a temperature map through an equation of
T
(
x
,
y
)
=
(
I
(
x
,
y
)
T
p
4
I
p
)
1
4
-
273.15
,
identifying an area having a temperature range of top 40% among temperature ranges in the temperature map, and selecting the identified area as the worker area,
wherein Tp is 310.35K, Ip is an average pixel value of the worker area in the thermal image, (x, y) is a pixel value at a pixel position (x, y) in the thermal image, and T(x, y) is a temperature value calculated at a pixel position (x, y); and
generating a temperature mask in the area selected as the worker area.
12 . The method of claim 11 , wherein the detecting the human being comprises:
extracting multi-scale features from a color image corresponding to the thermal image by using a backbone; and
extracting enhanced single-level features based on the multi-scale features through an encoder.
13 . The method of claim 12 , wherein the detecting the human being further comprises generating a mask feature and an instance feature through a decoder by using the enhanced single-level feature as an input feature,
wherein the mask feature is generated in a mask branch of the decoder, and
wherein the instance feature is generated in an instance branch of the decoder based on the input feature.
14 . The method of claim 13 , wherein the detecting the human being further comprises matching a resolution of the temperature mask with a resolution of the input feature of the instance branch.
15 . The method of claim 14 , wherein the detecting the human being further comprises generating the instance feature including a mask kernel by inputting a result value according to an element-wise multiplication of the temperature mask and the input feature of the instance branch to an instance activation map.
16 . The method of claim 15 , wherein the detecting the human being further comprises generating a segmentation mask configured to indicate the human being worker through an element-wise multiplication of the mask kernel and the mask feature.
17 . The method of claim 16 , further comprising setting the hazardous area in the thermal image through a deep learning-based hazardous area recommendation model.
18 . The method of claim 17 , wherein the detecting the hazard comprises determining that a human being mask has entered the hazardous area based on at least 10% of an area occupied by the human being mask indicating the human being in the segmentation mask overlaps the hazardous area.
19 . A method for detecting a human being using thermal imaging, the method comprising:
receiving a thermal image from a thermal imaging camera;
detecting the human being from the received thermal image by using an artificial intelligence model, wherein detecting the human being further comprises:
in case the thermal image includes a temperature in pixel units, identifying an area whose average temperature is 28° C. to 38° C. and selecting the identified area as a worker area, and
in case the thermal image does not include the temperature in pixel units, generating a temperature map through an equation of
T
(
x
,
y
)
=
(
I
(
x
,
y
)
T
p
4
I
p
)
1
4
-
273.15
,
identifying an area having a temperature range of top 40% among temperature ranges in the temperature map, and selecting the identified area as the worker area,
wherein Tp is 310.35K, Ip is an average pixel value of the worker area in the thermal image, (x, y) is a pixel value at a pixel position (x, y) in the thermal image, and T(x, y) is a temperature value calculated at a pixel position (x, y);
generating a temperature mask in the area selected as the worker area;
estimating in pixel units the detected human being in the thermal image;
detecting a hazard based on whether the human being has entered a preset hazardous area in the thermal image; and
sending a hazard notification based on the hazard being detected.
20 . The method of claim 19 , further comprising stopping operation of a work machine in the hazardous area based on the hazard being detected.