Image processing apparatus and image processing method

1. An image processing apparatus including a processor coupled to a memory, comprising:

a pixel value calculating unit configured to calculate estimated pixel values of respective pixels of a provisional image composed of “m” pixels by interpolation based on pixel values of respective pixels in a reference frame having “n” pixels (m>n);

a segmentation unit configured to segment an interior of the reference frame into (1) an edge region and a texture region, into (2) the edge region and a flat region, into (3) the texture region and the flat region, or into (4) the edge region, the texture region, and the flat region depending on the pixel values of respective pixels in the reference frame;

a position calculating unit configured to set the respective pixels in the reference frame as target pixels one by one in sequence and to calculate corresponding positions on the provisional image, of the respective target pixels, in decimal accuracy, which is finer than pixel intervals, based on information on the segmented edge region, texture region, or flat region including the target pixels;

a converting unit configured to modify the respective estimated pixel values so that a finite difference from the pixel values of the target pixels to the provisionally estimated pixel values of the target pixels, which are obtained from the estimated pixel values of the provisional image for pixels around the corresponding positions of the target pixels, becomes smaller, and to obtain modified pixel values; and

an output unit configured to output an image composed of “m” pixels having the modified pixel values.

2. The apparatus according to claim 1 , wherein the position calculating unit is configured to calculate the corresponding position using pixel values of pixels of a plurality of frames before and after the reference frame in time sequence, in addition to the pixel values of the respective pixels in the reference frame, when the segmented regions in which the target pixels are included is the edge region.

3. The apparatus according to claim 1 , wherein the position calculating unit is configured to calculate the corresponding positions only from the pixel values of the respective pixels in the reference frame, when the segmented region in which the target pixels are included is the edge region.

4. The apparatus according to claim 1 , wherein the position calculating unit is configured to calculate the corresponding positions only from the pixel values of the respective pixels in the reference frame, when the segmented region in which the target pixels are included is the texture region.

5. The apparatus according to claim 1 , wherein the respective estimated pixel values are determined as the modified pixel values when the segmented region in which the target pixels are included is the flat region.

6. The apparatus according to claim 1 ,

wherein the position calculating unit is configured to calculate the corresponding positions utilizing self congruity in the reference frame.

7. An image processing method, comprising:

calculating estimated pixel values of respective pixels of a provisional image composed of m pixels by interpolation based on pixel values of the respective pixels in a reference frame having n pixels (m>n);

segmenting an interior of the reference frame into (1) an edge region and a texture region, into (2) the edge region and a flat region, into (3) the texture region and the flat region, or into (4) the edge region, the texture region, and the flat region depending on the pixel values of the respective pixels in the reference frame;

setting the respective pixels in the reference frame as target pixels one by one in sequence and calculating corresponding positions on the provisional image corresponding to the respective target pixels, in decimal accuracy, which is finer than pixel intervals, based on information on the segmented edge region, texture region, or flat region including the target pixels;

modifying the respective estimated pixel values so that a finite difference between the provisionally estimated pixel values of the target pixels obtained from the estimated pixel values of the provisional image for pixels around the corresponding positions of the target pixels and the pixel values of the target pixels becomes smaller, and obtaining modified pixel values; and

outputting an image composed of m pixels having the modified pixel values.

8. A non-transitory computer readable storage medium on which a program for image processing is stored, the program including instructions which, when executed by a processor, cause the processor to execute a method comprising:

calculating estimated pixel values of respective pixels of a provisional image composed of m pixels by interpolation based on the pixel values of the respective pixels in a reference frame having n pixels (m>n);

segmenting an interior of the reference frame into (1) an edge region and a texture region, into (2) the edge region and a flat region, into (3) the texture region and the flat region, or into (4) the edge region, the texture region, and the flat region depending on the pixel values of the respective pixels in the reference frame;

setting the respective pixels in the reference frame as target pixels one by one in sequence and calculating corresponding positions on the provisional image corresponding to the respective target pixels, in decimal accuracy, which is finer than pixel intervals, based on information on the segmented edge region, texture region, or flat region including the target pixels;

modifying the respective estimated pixel values so that a finite difference between the provisionally estimated pixel values of the target pixels obtained from the estimated pixel values of the provisional image for pixels around the corresponding positions of the target pixels and the pixel values of the target pixels becomes smaller, and obtaining modified pixel values; and

outputting an image composed of m pixels having the modified pixel values.