IP Library › Granted Patent US 12,169,954
Granted Patent B2
US 12,169,954 · App. 15/947,157 · Granted Dec 17, 2024

Neural network combining visible and thermal images for inferring environmental data of an area of a building

Inventor: Francois Gervais (Lachine, CA)
Assignee: DISTECH CONTROLS INC.
G06V10/143F24F11/63G05B13/027G06V10/764G06V10/82G06V20/10H04N5/33
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,169,954
App. No.
15/947,157
Granted
Dec 17, 2024
Kind
B2
Abstract

Method and computing device for inferring via a neural network environmental data of an area of a building based on visible and thermal images of the area. A predictive model generated by a neural network training engine is stored by the computing device. The computing device determines a visible image of an area based on data received from at least one visible imaging camera. The computing device determines a thermal image of the area based on data received from at least one thermal imaging device. The computing device executes a neural network inference engine, using the predictive model for inferring environmental data based on the visible image and the thermal image. The inferred environmental data comprise geometric characteristic(s) of the area, an occupancy of the area, a human activity in the area, temperature value(s) for the area, and luminosity value(s) for the area.

Claims (28)

1. A computing device for inferring via a neural network environmental data of an area inside a building based on visible and thermal images of the area, the computing device comprising:

memory for storing a predictive model of the neural network generated by a neural network training engine; and

a processing unit for:

determining a visible image of an area inside a building based on data received from at least one visible imaging camera;

determining a thermal image of the area inside the building based on data received from at least one thermal imaging device; and

executing a neural network inference engine using the predictive model of the neural network for inferring outputs based on inputs, the outputs comprising a number of persons currently present in the area and at least one of the following: at least one current temperature value for the area, and at least one current luminosity value for the area, the inputs comprising the visible image and the thermal image.

2. The computing device of claim 1 , wherein the outputs further comprise at least one of the following: at least one geometric characteristic of the area, and a human activity in the area.

3. The computing device of claim 1 , wherein the outputs further comprise a human activity in the area, the human activity in the area comprising at least one of the following: a type of the area and a posture of persons present in the area.

4. The computing device of claim 1 , wherein the processing unit further generates one or more commands for controlling a controlled appliance based at least on the inferred outputs of the neural network inference engine.

5. The computing device of claim 1 , wherein the predictive model of the neural network comprises weights used by the neural network inference engine.

6. A method for inferring via a neural network environmental data of an area inside a building based on visible and thermal images of the area, the method comprising:

storing a predictive model of the neural network generated by a neural network training engine in a memory of a computing device;

determining by a processing unit of the computing device a visible image of an area inside a building based on data received from at least one visible imaging camera;

determining by the processing unit a thermal image of the area inside the building based on data received from at least one thermal imaging device; and

executing by the processing unit a neural network inference engine using the predictive model of the neural network for inferring outputs based on inputs, the outputs comprising a number of persons currently present in the area and at least of the following: at least one current temperature value for the area, and at least one current luminosity value for the area, the inputs comprising the visible image and the thermal image.

7. The method of claim 6 , wherein the outputs further comprise at least one of the following: at least one geometric characteristic of the area, and a human activity in the area.

8. The method of claim 6 , wherein the outputs further comprise a human activity in the area, the human activity in the area comprising at least one of the following: a type of the area and a posture of persons present in the area.

9. The method of claim 6 , wherein the processing unit further generates one or more commands for controlling a controlled appliance based at least on the inferred outputs of the neural network inference engine.

10. The method of claim 6 , wherein the predictive model of the neural network comprises weights used by the neural network inference engine.

11. A non-transitory computer program product comprising instructions executable by a processing unit of a computing device, the execution of the instructions by the processing unit providing for inferring via a neural network environmental data of an area inside a building based on visible and thermal images of the area by:

storing a predictive model of the neural network generated by a neural network training engine in a memory of the computing device;

determining by the processing unit of the computing device a visible image of an area inside a building based on data received from at least one visible imaging camera;

determining by the processing unit a thermal image of the area inside the building based on data received from at least one thermal imaging device; and

executing by the processing unit a neural network inference engine using the predictive model of the neural network for inferring outputs based on inputs, the outputs comprising a number of persons currently present in the area and at least one of the following: at least one current temperature value for the area, and at least one current luminosity value for the area, the inputs comprising the visible image and the thermal image.

12. The computer program product of claim 11 , wherein the outputs further comprise at least one of the following: at least one geometric characteristic of the area, and a human activity in the area.

13. The computer program product of claim 11 , wherein the outputs further comprise a human activity in the area, the human activity in the area comprising at least one of the following: a type of the area and a posture of persons present in the area.

14. The computer program product of claim 11 , wherein the processing unit further generates one or more commands for controlling a controlled appliance based at least on the inferred outputs of the neural network inference engine.

15. The computer program product of claim 11 , wherein the predictive model of the neural network comprises weights used by the neural network inference engine.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded May 21, 2018
From: GERVAIS, FRANCOIS
To: DISTECH CONTROLS INC.
Reel/Frame 045862/0155 →
Continuity (1)
Related Publication 20190310589A1 · Oct 10, 2019