Image processing device for improving details of an image, and operation method of the same

1. An image processing apparatus comprising:

a memory storing one or more instructions; and

a processor configured to execute the one or more instructions stored in the memory,

wherein the processor is configured to, by using one or more convolution neural networks,

extract target features by performing a convolution operation between features of target regions having same locations in a plurality of input images and a first kernel set,

extract peripheral features by performing a convolution operation between features of peripheral regions located around the target regions in the plurality of input images and a second kernel set, and

determine a feature of a region in an output image, corresponding to the target regions, based on the target features and the peripheral features, and

wherein the plurality of input images comprise a first input image and a second input image,

wherein the target regions comprise a first target region and a second target region having same locations in the first input image and the second input image, respectively, and

wherein the peripheral regions comprise a first peripheral region located around the first target region in the first input image and a second peripheral region located around the second target region in the second input image.

2. The image processing apparatus of claim 1 , wherein the processor is further configured to

extract the first peripheral region in a range where a distance from the first target region is within a preset distance, from the first input image, and

extract the second peripheral region in a range where a distance from the second target region is within the preset distance, from the second input image.

3. The image processing apparatus of claim 1 , wherein the processor is further configured to

determine a first feature similarity between each of a plurality of regions included in the first input image and the first target region, and determine the first peripheral region based on the first feature similarity, and

determine a second feature similarity between each of a plurality of regions included in the second input image and the second target region, and determine the second peripheral region based on the second feature similarity.

4. The image processing apparatus of claim 3 , wherein the first peripheral region is a region having a feature most similar to a feature of the first target region from among the plurality of regions included in the first input image, and

the second peripheral region is a region having a feature most similar to a feature of the second target region from among the plurality of regions included in the second input image.

5. The image processing apparatus of claim 1 , wherein the processor is further configured to

determine a first weight that is applied to the first peripheral region, based on a distance between the first target region and the first peripheral region,

determine a second weight that is applied to the second peripheral region, based on a distance between the second target region and the second peripheral region, and

extract the peripheral features by applying the first weight and the second weight.

6. The image processing apparatus of claim 5 , wherein the processor is further configured to determine the first weight as a larger value as the first target region and the first peripheral region are closer to each other, and determine the second weight as the larger value as the second target region and the second peripheral region are closer to each other.

7. The image processing apparatus of claim 1 , wherein the processor is further configured to

determine a first weight that is applied to the first peripheral region, based on a first similarity between a feature of the first target region and a feature of the first peripheral region,

determine a second weight that is applied to the second peripheral region, based on a second similarity between a feature of the second target region and a feature of the second peripheral region, and

extract the peripheral features by applying the first weight and the second weight.

8. The image processing apparatus of claim 7 , wherein the processor is further configured to determine the first weight as a larger value as the first similarity is larger, and determine the second weight as the larger value as the second similarity is larger.

9. An operation method of an image processing apparatus, the operation method comprising:

extracting target features by performing a convolution operation between features of target regions having same locations in a plurality of input images and a first kernel set;

extracting peripheral features by performing a convolution operation between features of peripheral regions located around the target regions in the plurality of input images and a second kernel set; and

determining a feature of a region in an output image, corresponding to the target regions, based on the target features and the peripheral features, and

wherein the plurality of input images comprise a first input image and a second input image,

wherein the target regions comprise a first target region and a second target region having same locations in the first input image and the second input image, respectively, and

wherein the peripheral regions comprise a first peripheral region located around the first target region in the first input image and a second peripheral region located around the second target region in the second input image.

10. The operation method of claim 9 , wherein the extracting of the peripheral features comprises:

extracting the first peripheral region in a range where a distance from the first target region is within a preset distance, from the first input image; and

extracting the second peripheral region in a range where a distance from the second target region is within the preset distance, from the second input image.

11. The operation method of claim 9 , wherein the extracting of the peripheral features comprises:

determining a first feature similarity between each of a plurality of regions included in the first input image and the first target region, and determining the first peripheral region based on the first feature similarity; and

determining a second feature similarity between each of a plurality of regions included in the second input image and the second target region, and determining the second peripheral region based on the second feature similarity.

12. The operation method of claim 11 , wherein the determining of the first peripheral region comprises determining, as the first peripheral region, a region having a feature most similar to a feature of the first target region from among the plurality of regions included in the first input image, and

the determining of the second peripheral region comprises determining, as the second peripheral region, a region having a feature most similar to a feature of the second target region from among the plurality of regions included in the second input image.

13. The operation method of claim 9 , wherein the extracting of the peripheral features comprises:

determining a first weight that is applied to the first peripheral region, based on a distance between the first target region and the first peripheral region;

determining a second weight that is applied to the second peripheral region, based on a distance between the second target region and the second peripheral region; and

extracting the peripheral features by applying the first weight and the second weight.