Image alignment apparatus and image alignment method of using the same

1. An image alignment apparatus comprising a hardware processor to implement:

a first conversion function estimation unit configured to estimate a first conversion function based on feature point information, the feature point information being extracted from a first image, which is captured by using a first image sensor, and a second image, which is captured by using a second image sensor that is different from the first image sensor;

a second conversion function estimation unit configured to estimate a second conversion function based on a similarity between a first motion vector in the first image and a second motion vector in the second image; and

an alignment unit configured to align the first image with the second image by using the first or second conversion function.

2. The image alignment apparatus of claim 1 , further comprising a conversion function selection unit implemented by the hardware processor, the conversion function selection unit configured to, according to whether the first conversion function is estimated, select the first conversion function or the second conversion function as a final conversion function.

3. The image alignment apparatus of claim 1 , wherein, if the first conversion function is not estimated, or periodically, the second conversion function selection unit estimates the second conversion function.

4. The image alignment apparatus of claim 1 , wherein the first conversion function selection unit comprises:

a feature point detection unit configured to detect feature points in the first image and the second image, respectively;

a feature point selection unit configured to select corresponding feature points from among the detected feature points in the first image and the second image; and

a first estimation unit for estimating the first conversion function based on the selected corresponding feature points.

5. The image alignment apparatus of claim 4 , wherein the feature point selection unit comprises:

a patch image obtaining unit configured to obtain patch images having the feature points in the first image and the second image at centers of the patch images, respectively;

a candidate selection unit, with regard to the feature point in the first image, configured to select candidate feature points in the second image, which correspond to the feature point in the first image;

a similarity determination unit configured to determine a similarity between the patch image of the feature point in the first image and the patch images of the candidate feature points in the second image; and

a corresponding feature point selection unit, based on a result of the determining of the similarity, configured to select the corresponding feature points from among the feature point in the first image and the candidate feature points in the second image.

6. The image alignment apparatus of claim 1 , wherein the second conversion function estimation unit comprises:

a motion detection unit configured to detect motions in the first image and the second image, respectively;

a motion vector selection unit configured to select corresponding motion vectors from among motion vectors that are detected from the detected motions in the first image and the second image; and

a second estimation unit configured to estimate the second conversion function based on the selected corresponding motion vectors.

7. The image alignment apparatus of claim 6 , wherein the motion vector selection unit comprises:

a labeling unit configured to label each of the detected motions that are detected from the first image and the second image;

a motion vector detection unit configured to detect the motion vectors with regard to the labeled motions in the first image and the second image;

a similarity determination unit configured to determine a similarity between the detected motion vectors in the first image and the second image; and

a corresponding motion vector selection unit, based on a result of the determining of the similarity, configured to select corresponding motion vectors between the first image and the second image.

8. The image alignment apparatus of claim 7 , wherein, with regard to the motion vector in the first image, the similarity determination unit is configured to select candidate motion vectors in the second image, and determine a similarity between the motion vector in the first image and the candidate motion vectors in the second image.

9. The image alignment apparatus of claim 8 , wherein the similarity determination unit is configured to determine the similarity based on a magnitude and a direction of the detected motion vectors.

10. An image alignment method comprising:

estimating a first conversion function based on feature point information, the feature point information being extracted from a first image, which is captured by using a first image sensor, and a second image, which is captured by using a second image sensor that is different from the first image sensor;

estimating a second conversion function based on a similarity between a first motion vector in the first image and a second motion vector in the second image; and

aligning the first image with the second image by using the first or second conversion function.

11. The image alignment apparatus of claim 10 , further comprising, according to whether the first conversion function is estimated, selecting the first conversion function or the second conversion function as a final conversion function.

12. The image alignment apparatus of claim 11 , wherein the estimating of the second conversion function is performed, if the first conversion function is not estimated, or periodically.

13. The image alignment apparatus of claim 11 , wherein, the estimating of the first conversion function comprises:

detecting feature points in the first image and the second image, respectively;

selecting corresponding feature points from among the detected feature points in the first image and the second image; and

estimating the first conversion function based on the selected corresponding feature points.

14. The image alignment apparatus of claim 13 , wherein the selecting of the corresponding feature points comprises:

obtaining patch images having the feature points in the first image and the second image at the centers of the patch images, respectively;

with regard to the feature point in the first image, selecting candidate feature points in the second image, which correspond to the feature point in the first image;

determining a similarity between the patch image of the feature point in the first image and the patch images of the candidate feature points in the second image; and

based on a result of the determining of the similarity, selecting the corresponding feature points from among the feature point in the first image and the candidate feature points in the second image.

15. The image alignment apparatus of claim 11 , wherein the estimating of the second conversion function comprises:

detecting motions in the first image and the second image, respectively;

selecting corresponding motion vectors from among motion vectors that are detected from the detected motions in the first image and the second image; and

estimating the second conversion function based on the selected corresponding motion vectors.

16. The image alignment apparatus of claim 15 , wherein the selecting of the corresponding motion vectors comprises:

labeling each of the detected motions that are detected from the first image and the second image;

detecting the motion vectors with regard to the labeled motions in the first image and the second image;

determining a similarity between the detected motion vectors in the first image and the second image; and

based on a result of the determining of the similarity, selecting corresponding motion vectors between the first image and the second image.

17. The image alignment apparatus of claim 16 , wherein the determining of the similarity comprises, with regard to the motion vector in the first image, selecting candidate motion vectors in the second image, and determining a similarity between the motion vector in the first image and the candidate motion vectors in the second image.

18. The image alignment apparatus of claim 16 , wherein the determining of the similarity comprises determining the similarity based on a magnitude and a direction of the detected motion vectors.

19. The image alignment apparatus of claim 1 , wherein the first image sensor comprises a visible-ray camera and the second image sensor comprises at least one of an infrared-ray camera and a thermal camera.