Image processing apparatus and image processing method
The present disclosure relates to an image processing apparatus, an image processing method, and a program therefor with which slow shutter photographing can be performed with ease. An image gradient extraction unit extracts image gradient components in an object movement direction from a long-exposure image out of input images. An initial label map generation unit generates an initial label map based on a gradient extraction result from the image gradient extraction unit. A foreground extraction unit extracts a foreground from the input images based on the label map from the initial label map generation unit or label map update unit to generate a synthesis mask. The present disclosure is applicable to, for example, an image pickup apparatus including an image processing function.
1. An image processing apparatus, comprising:
a display screen; and
circuitry configured to:
control the display screen to display a frame and a long-exposure image of a plurality of consecutively-photographed images;
determine a movement direction of an object in the long-exposure image based on the displayed frame;
extract image gradient information of the long-exposure image, wherein the image gradient information indicates image gradient components in the movement direction of the object in the long-exposure image;
generate a first synthesis mask based on the extracted image gradient information, wherein the plurality of consecutively-photographed images have different exposure times;
synthesize the plurality of consecutively-photographed images based on the generated first synthesis mask;
generate an output image based on the synthesis of the plurality of the consecutively-photographed images; and
control the display screen to display the output image.
2. The image processing apparatus according to claim 1 , wherein the circuitry is further configured to:
generate an initial label map of the first synthesis mask based on a gradient intensity of the long-exposure image, wherein the gradient intensity is in the movement direction of the object;
extract a first foreground based on the initial label map; and
generate the first synthesis mask based on the extraction of the first foreground.
3. The image processing apparatus according to claim 2 , wherein the circuitry is further configured to:
update the initial label map based on at least one of a representative color or an update area, wherein each of the representative color and the update area corresponds to the first foreground of the first synthesis mask;
extract a second foreground based on the updated initial label map; and
generate a second synthesis mask based on the extraction of the second foreground.
4. The image processing apparatus according to claim 2 , wherein the circuitry is further configured to:
control the display screen to display a UI (User Interface); and
set a specific direction as the movement direction of the object in the long-exposure image based on the display of the UI.
5. The image processing apparatus according to claim 4 , wherein the circuitry is further configured to set the specific direction as the movement direction of the object in the long-exposure image based on a user operation in the movement direction of the object in the long-exposure image.
6. The image processing apparatus according to claim 4 , wherein the circuitry is further configured to set the specific direction as the movement direction of the object in the long-exposure image based on a user selection from object movement direction candidates in the plurality of consecutively-photographed images.
7. The image processing apparatus according to claim 1 , wherein the circuitry is further configured to:
position each of the plurality of consecutively-photographed images; and
generate the first synthesis mask based on the image gradient information of the positioned long-exposure image.
8. An image processing method, comprising:
in an image processing apparatus that includes a display screen and circuitry:
controlling, by the circuitry, the display screen to display a frame and a long-exposure image of a plurality of consecutively-photographed images;
determining, by the circuitry, a movement direction of an object in the long-exposure image based on the displayed frame;
extracting, by the circuitry, image gradient information of the long-exposure image, wherein the image gradient information indicates image gradient components in the movement direction of the object in the long-exposure image;
generating, by the circuitry, a synthesis mask based on the extracted image gradient information, wherein the plurality of consecutively-photographed images have different exposure times;
synthesizing, by the circuitry, the plurality of consecutively-photographed images based on the generated synthesis mask;
generating, by the circuitry, an output image based on the synthesis of the plurality of the consecutively-photographed images; and
controlling, by the circuitry, the display screen to display the output image.
9. A non-transitory computer-readable medium having stored thereon computer-executable instructions that, when executed by a processor, cause the processor to execute operations, the operations comprising:
controlling a display screen to display a frame and a long-exposure image of a plurality of consecutively-photographed images;
determining a movement direction of an object in the long-exposure image based on the displayed frame;
extracting image gradient information of the long-exposure image, wherein the image gradient information indicates image gradient components in the movement direction of the object in the long-exposure image;
generating a synthesis mask based on the extracted image gradient information, wherein the plurality of consecutively-photographed images have different exposure times;
synthesizing the plurality of consecutively-photographed images based on the generated synthesis mask;
generating an output image based on the synthesis of the plurality of the consecutively-photographed images; and
controlling the display screen to display the output image.