Image processing apparatus
Multi-resolution images of a reference image and a target image are generated. Then, whole-range matching is performed on an image of a lower resolution to detect a two-dimensional displacement between the images. Block matching is performed on an image of a higher resolution to detect a displacement at each feature point. The accuracy of motion data is increased by correcting the motion data with an image of a higher resolution by using the previously calculated motion data of the lowest resolution through higher resolutions as an initial value.
1. An image processing apparatus comprising:
image data input means for inputting captured multiple images;
detection image generation means for generating, based on the input images, smaller images by scaling down a certain area of the images;
multi-resolution image data storage means for storing multi-resolution image data of the smaller images up to a predetermined resolution level such that each of them is identifiable between a reference image and a target image;
feature point set extraction means for extracting one or more feature point coordinates based on an image corner or edge of the multi-resolution image data of the reference image; and
motion data calculation means for calculating motion data between the reference image and the target image by using the multi-resolution image data,
wherein the motion data calculating means has:
a whole-range matching section for performing, with regard to each of the reference images and target images from a lowest resolution L to a predetermined resolution M (M≦L), a matching process on all image coordinates in a predetermined range of the reference images and calculating motion data based on a first energy function whose arguments are the coordinates on the reference images and the target images; and
a motion data calculating section for, with regard to image data of a resolution level higher than the predetermined resolution M, performing a block matching process based on a second energy function by setting input motion data as an initial value and handling the feature point coordinates on the reference images, which are extracted by the feature point set extraction means, as arguments thereof, and for calculating motion data between the reference images and target images by using reliability defined based on energy of the block matching process or strength of feature points and correlation defined based on motion data among the feature points of the reference images or coordinates based on the motion data; and
motion data is calculated by a process, wherein the process comprises (a) calculating motion data for image data of a resolution M−1, which is a higher resolution than the resolution M, with using motion data for image data of the resolution M, which is calculated by the whole-range matching section, as initial value of motion data to be input to the motion data calculating section, (b) calculating motion data for image data of a much higher resolution with using the motion data as initial value information of motion data to be input to the motion data calculating section, and (c) repeating calculating the image data of higher resolution for predetermined number of times or until a predetermined condition is satisfied.
2. The image processing apparatus according to claim 1 , wherein the motion data calculated by the motion data calculating section is an affine parameter.
3. An image processing apparatus comprising:
image data input means for inputting captured multiple images;
detection image generation means for generating, based on the input images, smaller images by scaling down a certain area of the images;
multi-resolution data generation means for calculating multi-resolution image data up to a predetermined resolution level based on images generated by the detection image generation means;
frame transforming data generation means having:
a motion data detection section for calculating motion data from a reference frame based on the reference frame and an input image;
a frame transforming data combining section for calculating a first frame transforming data based on the motion data and reference frame transforming data;
a frame transforming data standardization section for standardizing the first frame transforming data to calculate a second frame transforming data;
a frame transforming data regularizing section for determining whether a frame transformed by the second frame transforming data is within the captured frame or not and regularizing the second frame transforming data to make the frame transformed by the second frame transforming data fit within the captured frame; and
an update section for updating the reference frame and the reference frame transforming data by using the frame transforming data calculated by the frame transforming data regularizing section.
4. The image processing apparatus according to claim 3 , further comprising:
synthesis means for determining whether coordinate points of four corners of a correction frame rectangle transformed by the frame transforming data are within a captured frame rectangle or not, calculating an optimum frame transforming data to fit the correction frame rectangle within the captured frame rectangle when any of the coordinate points thereof is outside of the captured frame rectangle, and synthesizing the correction frame.