Real-time composite image comparator
View Patent ↗An apparatus and method for preparing a composite image from a video data stream and for identifying changed features in two composite images. Frames from the video data stream are transformed to a projected 2D image, aligned with adjacent frames and overlapping areas are averaged to provide a higher apparent resolution. The composite image can be stored in real-time. As a second composite image is prepared of the same location at a later time, portions of the second image can be compared to corresponding portions of the stored image after the intensities of the images are equalized. Image areas whose absolute difference exceeds a threshold are again intensity equalized. Areas that are again above threshold can be flagged for further scrutiny, either by a human or by a machine that performs object recognition. In this way, composite video images of a scene can be prepared and compared in real-time.
1. A computerized method for preparing a first composite image and a second composite image of a first scene comprising: a. using a computer, providing a first series of images from the first scene, at least two images in the first series of images containing an overlapping area in the first scene; b. using the computer, aligning the at least two images in the first series of images; c. using the computer, averaging the intensity of the images in the area of overlap of the at least two images in the first series of images; and d. using the computer, forming the first composite image from the non-overlapping areas of the at least two images in the first series of images and the averaged overlapping area of the at least two images in the first series of images; e. using the computer, storing the first composite image; f. using the computer, preparing the second composite image of the first scene including: i. providing a second series of images from the first scene, at least two images in the second series of images containing an overlapping area in the first scene, ii. aligning the at least two images in the second series of images, iii. averaging the area of overlap of the at least two images in the second series of images, and iv. forming the second composite image from the non-overlapping areas of the at least two images in the second series of images and the averaged overlapping areas of the at least two images in the second series of images; g. using the computer, adjusting a given area of the first composite image to approximately equal the average intensity of a given area of the second composite image, the given area of the first composite image and the given area of the second composite image corresponding to the same location in the first scene; h. using the computer, calculating the absolute difference in intensity between the given area of the first composite image after adjustment and the given area of the second composite image and identifying a selected portion of the given area of the first composite image where the absolute difference is above a threshold; and adjusting the average intensity of the selected portion of the given area of the first composite image to approximately equal the average intensity of the corresponding portion of the given area of the second composite image and marking the selected portion where the absolute difference between the selected portion and the corresponding area in the second composite image is above the threshold.
2. A method according to claim 1 wherein providing a first series of images includes capturing a video sequence with a video camera.
3. A method according to claim 1 wherein the first scene is a terrain scene.
4. A method according to claim 1 wherein providing the first series of images includes transforming the first series of images using the computer.
5. A method according to claim 4 wherein transforming the first series of images includes projecting the first series of images onto a two-dimensional surface.
6. A method according to claim 1 wherein aligning the at least two images in the first series of images includes correlating portions of the at least two images in the first series of images.
7. A method according to claim 1 wherein adjusting a given area of the first composite image begins executing before preparing the second composite image of the first scene completes.
8. A method according to claim 1 wherein edges are removed from the first composite image and from the second composite image before calculating the absolute difference in intensity between the given area of the first composite image after adjustment and the given area of the second composite image.
9. A computer program product for use on a computer system for preparing a first composite image and a second composite image of a first scene, the computer program product comprising a non-transitory computer readable medium encoded with computer readable program code, the computer readable program code including: a. program code for providing a first series of images from the first scene, at least two images in the first series of images containing an overlapping area in the first scene; b. program code for aligning the at least two images in the first series of images of the first scene; c. program code for averaging the intensity of the image in the area of overlap of the at least two images in the first series of images of the first scene; d. program code for forming the first composite image from the non- overlapping areas of the at least two images in the first series of images and the averaged overlapping area of the at least two images in the first series of images; e. program code for storing the first composite image; f. program code for preparing the second composite image of the first scene including: i. providing a second series of images from the first scene, at least two images in the second series of images containing an overlapping area in the first scene, ii. aligning the at least two images in the second series of images, iii. averaging the area of overlap of the at least two images in the second series of images, and iv. forming the second composite image from the non-overlapping areas of the at least two images in the second series of images and the averaged overlapping areas of the at least two images in the second series of images; g. program code for adjusting a given area of the first composite image to approximately equal the average intensity of a given area of the second composite image, the given area of the first composite image and the given area of the second composite image corresponding to the same location in the first scene; h. program code for calculating the absolute difference in intensity between the given area of the first composite image after adjustment and the given area of the second composite image and identifying a selected portion of the given area of the first composite image where the absolute difference is above a threshold; and i. program code for adjusting the average intensity of the selected portion of the given area of the first composite image to approximately equal the average intensity of the corresponding portion of the given area of the second composite image and marking the selected portion where the absolute difference between the selected portion and the corresponding area in the second composite image is above the threshold.
10. A computer program product according to claim 9 , wherein providing a first series of images includes capturing a video sequence comprising the first series of images with a video camera.
11. A computer program product according to claim 9 , wherein the first scene is a terrain scene.
12. A computer program product according to claim 9 , wherein program code for providing the first series of images includes program code for transforming the first series of images.
13. A computer program product according to claim 12 , wherein program code for transforming the first series of images includes program code for projecting the first series of images onto a two-dimensional surface.
14. A computer program product according to claim 9 , wherein program code for aligning the at least two images in the first series of images of the first scene includes program code for correlating portions of the at least two images in the first series of images of the first scene.
15. A computer program product according to claim 9 wherein program code for adjusting a given area of the first composite image is configured to begin executing before program code for preparing the second composite image of the first scene completes.
16. A computer program product according to claim 9 wherein the program code is configured so that edges are removed from the first composite image and from the second composite image before calculating the absolute difference in intensity between the given area of the first composite image after adjustment and the given area of the second composite image.