Automatic co-registration of thermal and visible image pairs

1. A method comprising:

receiving a pair of images;

defining a parametric deformation model providing an image deformation for each of a set of deformation parameters;

defining a loss function based, at least in part, on at least one directional gradient of each of a processed pair of images obtained from the received pair of images by a process comprising applying the parametric deformation model, wherein the loss function is minimized when the pair of images are aligned; and

performing a multi-scale search to determine an optimal deformation parameter that minimizes the loss function;

wherein the optimal deformation parameter defines an alignment of the pair of images.

2. The method of claim 1 , wherein the pair of images comprises a visible spectrum image and an infrared image.

3. The method of claim 2 , wherein the process further comprises resizing the visible spectrum image to match the infrared image;

wherein the loss function is based, at least in part, on comparing the directional gradients of the processed pair of images at each of a plurality of pixels.

4. The method of claim 1 , wherein the process further comprises applying at least one lens distortion correction model to each of the pair of images.

5. The method of claim 1 , wherein the process further comprises normalizing a dynamic range of each of the pair of images; and

wherein the loss function is based, at least in part, on comparing the directional gradients of the processed pair of images in each of a plurality of directions at each of the plurality of pixels.

6. The method of claim 1 , wherein the process further comprises resizing the pair of images to a current processing scale of the multi-scale search.

7. The method of claim 6 , wherein the process further comprises, after resizing the pair of images to a current processing scale of the multi-scale search, applying adaptive histogram equalization to the pair of images to generate equalized images.

8. The method of claim 7 , wherein the process further comprises applying Gaussian Blur to the equalized images.

9. The method of claim 1 , wherein the loss function is based, at least in part, on determining a sum of squares of differences between absolute values of the directional gradients of the processed pair of images in each of a plurality of directions at each of a plurality of pixels.

10. The method of claim 1 , wherein:

each deformation parameter p is defined by p=[δx,δy,s,ϕ], where s is a scaling factor that controls image scaling, ϕ controls image rotation in degrees, and δx and δy control the image translation (in pixels) in horizontal and vertical directions, respectively; and

for each deformation parameter, the loss function is based, at least in part, on a penalty term on the deformation parameter.

11. A system comprising:

a processor; and

a local memory operable to store a plurality of machine readable instructions which when executed by the processor are operable to cause the system to perform steps comprising:

receiving a pair of images;

defining a parametric deformation model providing an image deformation for each of a set of deformation parameters;

defining a loss function based, at least in part, on at least one directional gradient of each of a processed pair of images obtained from the received pair of images by a process comprising applying the parametric deformation model, wherein the loss function is minimized when the pair of images are aligned; and

performing a multi-scale search to determine an optimal deformation parameter that minimizes the loss function;

wherein the optimal deformation parameter defines an alignment of the pair of images.

12. The system of claim 11 , wherein the pair of images comprises a visible spectrum image and an infrared image; and

wherein the system further comprises a plurality of image capture components, including a visible spectrum sensor operable to capture the visible spectrum image, and an infrared sensor operable to capture the infrared image.

13. The system of claim 12 , further comprising optical components arranged to pass an image of a scene to the image capture components; and

wherein the process further comprises applying at least one lens distortion correction model to each of the pair of images, the lens distortion correction model comprising a model of the optical components.

14. The system of claim 11 , wherein:

the loss function is based, at least in part, on comparing the directional gradients of the processed pair of images in each of a plurality of directions; and

the process further comprises:

normalizing a dynamic range of each of the pair of images.

15. The system of claim 11 , wherein:

the loss function is based, at least in part, on comparing the directional gradients of the processed pair of images at each of a plurality of pixels; and

the process further comprises:

resizing the pair of images to a current processing scale of the multi-scale search.

16. The system of claim 15 , wherein the process further comprises, after resizing the pair of images to a corresponding processing scale of the multi-scale search, applying adaptive histogram equalization to the pair of images to generate equalized images.

17. The system of claim 16 , wherein the process further comprises applying Gaussian Blur to the equalized images.

18. The system of claim 11 , wherein the loss function is based, at least in part, on determining a sum of squares of differences between absolute values of the directional gradients of the processed pair of images in each of a plurality of directions at each of a plurality of pixels.

19. The system of claim 11 , wherein each deformation parameter p is defined by p=[δx,δy,s,ϕ], where s is a scaling factor that controls image scaling, ϕ controls image rotation in degrees, and δx and δy control the image translation (in pixels) in horizontal and vertical directions, respectively; and

for each deformation parameter, the loss function is based, at least in part, on a penalty term on the deformation parameter.

20. An unmanned aerial system comprising the system of claim 11 .