Method and computing device using a neural network to localize an overlap between two thermal images respectively generated by two infrared sensors
A computing device stores a predictive model generated by a neural network training engine. The computing device receives first and second two-dimensional (2D) thermal images comprising temperature measurements from respective first and second infrared (IR) sensors. The first and second images have the same size. An image capturing visual field of the second IR sensor partially overlaps with an image capturing visual field of the first IR sensor. The computing device executes a neural network using a predictive model for generating outputs based on inputs. The inputs comprise the temperature measurements of the first and second images. The outputs comprise horizontal and vertical shifts defining a translation of the second image with respect to the first image. An overlapping area in the first image, having a rectangular shape and overlapping with the second image, is determined using the horizontal and vertical shifts.
1. A method using a neural network to localize an overlap between two thermal images respectively generated by two infrared sensors, the method comprising:
storing in a memory of a computing device a predictive model generated by a neural network training engine;
receiving by a processing unit of the computing device via a communication interface of the computing device a first two-dimensional (2D) thermal image comprising temperature measurements from a first infrared (IR) sensor, the first 2D thermal image having M columns and N rows, M and N being integers;
receiving by the processing unit via the communication interface a second 2D thermal image comprising temperature measurements from a second IR sensor, the second 2D thermal image also having M columns and N rows, an image capturing visual field of the second IR sensor partially overlapping with an image capturing visual field of the first IR sensor; and
executing by the processing unit a neural network inference engine, the neural network inference engine implementing a neural network using the predictive model for generating outputs based on inputs, the inputs comprising the temperature measurements of the first 2D thermal image and the temperature measurements of the second 2D thermal image, the outputs comprising a horizontal shift S H and a vertical shift S V , the horizontal and vertical shifts being integers, the horizontal and vertical shifts defining a translation of the second 2D thermal image with respect to the first 2D thermal image.
2. The method of claim 1 , further comprising determining by the processing unit an overlapping area in the first 2D thermal image using the horizontal shift S H and the vertical shift S V , the overlapping area in the first 2D thermal image having a rectangular shape and overlapping with the second 2D thermal image.
3. The method of claim 2 , further comprising combining by the processing unit the first 2D thermal image with the second 2D thermal image, taking into consideration the overlapping area.
4. The method of claim 2 , wherein the horizontal shift S H and the vertical shift S V are positive integers, and the overlapping area extends from column S H +1 to column M and from row S V +1 to row N.
5. The method of claim 2 , wherein the horizontal shift S H is a positive integer and the vertical shift S V is a negative integer, and the overlapping area extends from column S H +1 to column M and from row 1 to row N+S V .
6. The method of claim 2 , wherein the horizontal shift S H is a negative integer and the vertical shift S V is a positive integer, and the overlapping area extends from column 1 to column M+S H and from row S V +1 to row N.
7. The method of claim 2 , wherein the horizontal shift S H and the vertical shift S V are negative integers, and the overlapping area extends from column 1 to column M+S H and from row 1 to row N+S V .
8. The method of claim 1 , wherein the neural network comprises a flattening layer, followed by a plurality of fully connected layers.
9. The method of claim 1 , wherein the neural network comprises a convolutional layer, optionally followed by a pooling layer, followed by a flattening layer, followed by a plurality of fully connected layers.
10. The method of claim 1 , wherein a series of I consecutive first 2D thermal images are received from the first IR sensor, I being an integer greater or equal than 2, a series of I consecutive second 2D thermal images are received from the second IR sensor, the inputs of the neural network include the series of I consecutive first 2D thermal images and the series of I consecutive second 2D thermal images, and the horizontal and vertical shifts define a translation of the last of the series of I consecutive second 2D thermal image with respect to the last of the series of I consecutive first 2D thermal image.