Method and apparatus for processing array image
An image processing method includes: based on an imaging device generating an input array image of a multi-view through an array lens, determining a first setting parameter indicating whether to use the multi-view or a single view selected from the multi-view; determining a second setting parameter indicating whether a priority is assigned to sensitivity enhancement based on pixel binning or to resolution enhancement based on pixel interpolation; determining an image processing mode based on the first setting parameter and the second setting parameter; and generating an output image by performing image processing on the input array image based on a processing procedure that is based on the determined image processing mode.
1. An image processing method comprising:
based on an imaging device generating an input array image of a multi-view through an array lens, determining a first setting parameter indicating whether to use the multi-view or a single view selected from the multi-view;
determining a second setting parameter indicating whether a priority is assigned to sensitivity enhancement based on pixel binning or to resolution enhancement based on pixel interpolation;
determining an image processing mode based on the first setting parameter and the second setting parameter; and
generating an output image by performing image processing on the input array image based on a processing procedure that is based on the determined image processing mode.
2. The image processing method of claim 1 , wherein the determined image processing mode comprises:
a first mode that is determined when the first setting parameter indicates to use the multi-view and the second setting parameter indicates the priority is assigned to the resolution enhancement,
a second mode that is determined when the first setting parameter indicates to use the multi-view and the second setting parameter indicates the priority is assigned to the sensitivity enhancement,
a fourth mode that is determined when the first setting parameter indicates to use the single view and the second setting parameter indicates the priority is assigned to the resolution enhancement, or
a fifth mode that is determined when the first setting parameter indicates to use the single view and the second setting parameter indicates the priority is assigned to the sensitivity enhancement.
3. The image processing method of claim 2 ,
wherein the processing procedure that is based on the first mode comprises sub stages, and
wherein the processing procedure that is based on the second mode, the fourth mode, or the fifth mode comprises only some of the sub stages.
4. The image processing method of claim 3 , wherein the sub stages comprise:
a first sub stage for performing demosaicing on raw data,
a second sub stage for performing upsampling for resolution enhancement,
a third sub stage for generating matching information of the multi-view, and
a fourth sub stage for generating a synthesized image using the matching information.
5. The image processing method of claim 4 ,
wherein the processing procedure that is based on the second mode does not include the first sub stage and the second sub stage,
wherein the processing procedure that is based on the fourth mode does not include the third sub stage and the fourth sub stage, and
wherein the processing procedure that is based on the fifth mode does not include the second sub stage, the third sub stage, and the fourth sub stage.
6. The image processing method of claim 1 , further comprising:
determining a third setting parameter indicating whether to use hardware logic-based binning processing,
wherein the determining the image processing mode is further based on the third setting parameter.
7. The image processing method of claim 6 , wherein the determined image processing mode comprises:
a first mode that is determined when the first setting parameter indicates to use the multi-view, the third setting parameter indicates to not use the hardware logic-based binning processing, and the second setting parameter indicates the priority is assigned to the resolution enhancement,
a second mode that is determined when the first setting parameter indicates to use the multi-view, the third setting parameter indicates to not use the hardware logic-based binning processing, and the second setting parameter indicates the priority is assigned to the sensitivity enhancement,
a third mode that is determined when the first setting parameter indicates to use the multi-view and the third setting parameter indicates to use the hardware logic-based binning processing,
a fourth mode that is determined when the first setting parameter indicates to use the single view, the third setting parameter indicates to not use the hardware logic-based binning processing, and the second setting parameter indicates the priority is assigned to the resolution enhancement,
a fifth mode that is determined when the first setting parameter indicates to use the single view, the third setting parameter indicates to not use the hardware logic-based binning processing, and the second setting parameter indicates the priority is assigned to the sensitivity enhancement, and
a sixth mode that is determined when the first setting parameter indicates to use the single view and the third setting parameter indicates to use the hardware logic-based binning processing.
8. The image processing method of claim 6 , wherein the third setting parameter indicates to use the hardware logic-based binning processing in at least one of a low-illuminance environment or a video capturing mode.
9. The image processing method of claim 6 , wherein the hardware logic-based binning processing comprises:
determining initial corresponding pixels between sub images that respectively correspond to different views of the input array image,
extracting a refinement target from the initial corresponding pixels based on an intensity difference between the initial corresponding pixels,
determining a new corresponding pixel for the refinement target by using a calibration parameter, and
performing binning on final corresponding pixels through a weighted average of the final corresponding pixels that are based on the new corresponding pixel.
10. The image processing method of claim 1 , wherein the first setting parameter indicates to use the single view in response to a demand for low power.
11. A non-transitory computer-readable storage medium storing instructions that, when executed by a processor, cause the processor to perform an image processing method comprising:
Based on an imaging device generating an input array image of a multi-view through an array lens, determining a first setting parameter indicating whether to use the multi-view or a single view selected from the multi-view;
determining a second setting parameter indicating whether a priority is assigned to sensitivity enhancement based on pixel binning or to resolution enhancement based on pixel interpolation;
determining an image processing mode based on the first setting parameter and the second setting parameter; and
generating an output image by performing image processing on the input array image based on a processing procedure that is based on the determined image processing mode.
12. An image processing apparatus comprising:
a processor; and
a memory configured to store instructions executable by the processor,
wherein, in response to the instructions being executed by the processor, the processor is configured to:
based on an imaging device generating an input array image of a multi-view through an array lens, determine a first setting parameter indicating whether to use the multi-view or a single view selected from the multi-view,
determine a second setting parameter indicating whether a priority is assigned to sensitivity enhancement based on pixel binning or to resolution enhancement based on pixel interpolation,
determine an image processing mode based on the first setting parameter and the second setting parameter, and
generate an output image by performing image processing on the input array image based on a processing procedure that is based on the determined image processing mode.
13. The image processing apparatus of claim 12 , wherein the determined image processing mode comprises:
a first mode that is determined when the first setting parameter indicates to use the multi-view and the second setting parameter indicates the priority is assigned to the resolution enhancement,
a second mode that is determined when the first setting parameter indicates to use the multi-view and the second setting parameter indicates the priority is assigned to the sensitivity enhancement,
a fourth mode that is determined when the first setting parameter indicates to use the single view and the second setting parameter indicates the priority is assigned to the resolution enhancement, or
a fifth mode that is determined when the first setting parameter indicates to use the single view and the second setting parameter indicates the priority is assigned to the sensitivity enhancement.
14. The image processing apparatus of claim 13 ,
wherein the processing procedure that is based on the first mode comprises sub stages, and
wherein the processing procedure that is based on the second mode, the fourth mode, or the fifth mode comprises only some of the sub stages.
15. The image processing apparatus of claim 12 , wherein the processor is further configured to:
determine a third setting parameter indicating whether to use hardware logic-based binning processing,
wherein the processor being configured to determine the image processing mode is further based on the third setting parameter.
16. The image processing apparatus of claim 15 , wherein the determined image processing mode comprises:
a first mode that is determined when the first setting parameter indicates to use the multi-view, the third setting parameter indicates to not use the hardware logic-based binning processing, and the second setting parameter indicates the priority is assigned to the resolution enhancement,
a second mode that is determined when the first setting parameter indicates to use the multi-view, the third setting parameter indicates to not use the hardware logic-based binning processing, and the second setting parameter indicates the priority is assigned to the sensitivity enhancement,
a third mode that is determined when the first setting parameter indicates to use the multi-view and the third setting parameter indicates to use the hardware logic-based binning processing,
a fourth mode that is determined when the first setting parameter indicates to use the single view, the third setting parameter indicates to not use the hardware logic-based binning processing, and the second setting parameter indicates the priority is assigned to the resolution enhancement,
a fifth mode that is determined when the first setting parameter indicates to use the single view, the third setting parameter indicates to not use the hardware logic-based binning processing, and the second setting parameter indicates the priority is assigned to the sensitivity enhancement, and
a sixth mode that is determined when the first setting parameter indicates to use the single view and the third setting parameter indicates to use the hardware logic-based binning processing.
17. The image processing apparatus of claim 15 , wherein the third setting parameter indicates to use the hardware logic-based binning processing in at least one of a low-illuminance environment or a video capturing mode.
18. The image processing apparatus of claim 15 , wherein the hardware logic-based binning processing comprises the processor being configured to:
determine initial corresponding pixels between sub images respectively that correspond to different views of the input array image,
extract a refinement target from the initial corresponding pixels based on an intensity difference between the initial corresponding pixels,
determine a new corresponding pixel for the refinement target by using a calibration parameter, and
perform binning on final corresponding pixels through a weighted average of the final corresponding pixels that are based on the new corresponding pixel.
19. The image processing apparatus of claim 12 , wherein the first setting parameter indicates to use the single view in response to a demand for low power.
20. An electronic device comprising:
an imaging device configured to generate an input array image of a multi-view through an array lens, the input array image comprising sub images corresponding to the multi-view; and
a processor configured to:
determine a first setting parameter indicating whether to use the multi-view or a single view selected from the multi-view,
determine a second setting parameter indicating whether a priority is assigned to sensitivity enhancement based on pixel binning or to resolution enhancement based on pixel interpolation,
determine an image processing mode based on the first setting parameter and the second setting parameter, and
generate an output image by performing image processing on the input array image based on a processing procedure that is based on the determined image processing mode.