Image generation apparatus and image generation method

1. An image generation apparatus that generates, from a plurality of video sequences obtained by photographing a same object, a new video sequence which represents the same object, said image generation apparatus comprising:

an image receiving unit operable to receive a first video sequence including frames having a first resolution and a second video sequence including frames having a second resolution which is higher than the first resolution, each frame of the first video sequence being obtained with a first exposure time, and each frame of the second video sequence being obtained with a second exposure time which is longer than the first exposure time; and

an image integration unit operable to generate, from the first video sequence and the second video sequence, the new video sequence including frames having a resolution which is equal to or higher than the second resolution, at a frame rate which is equal to or higher than a frame rate of the first video sequence, by reducing a difference between a value of each frame of the second video sequence and a sum of values of frames of the new video sequence which are included within an exposure period of a frame of the second video sequence.

2. The image generation apparatus according to claim 1 ,

wherein each frame of the second video sequence is obtained by open exposure between the frames of the second video sequence.

3. The image generation apparatus according to claim 1 , further comprising

a constraint unit operable to specify a constraint to be satisfied by pixel values of the new video sequence to be generated, based on continuity between pixel values of spatially and temporally adjacent pixels,

wherein said image integration unit is operable to generate the new video sequence in which the specified constraint is satisfied.

4. The image generation apparatus according to claim 1 , further comprising:

a motion estimation unit operable to estimate a motion of the same object from at least one of the plurality of video sequences received by said image receiving unit; and

a constraint unit operable to specify a constraint to be satisfied by pixel values of the new video sequence to be generated, based on a result of the estimated motion,

wherein said image integration unit is operable to generate the new video sequence in which the specified constraint is satisfied.

5. The image generation apparatus according to claim 4 ,

wherein said motion estimation unit is operable to calculate a confidence level of the estimated motion, and

said image integration unit is operable to generate the new video sequence, using the specified constraint based on a result of the estimated motion for an image area with a high confidence level calculated by said motion estimation unit, and using a predetermined constraint other than the specified constraint based on the estimated motion for an image area with a low confidence level.

6. The image generation apparatus according to claim 5 ,

wherein said motion estimation unit is operable to estimate the motion of the same object on the basis of blocks obtained by dividing each of the frames which constitute the first video sequence, and to calculate, as the confidence level, a negative value of a sum of squares of differences in pixel values between the blocks, and

said image integration unit is operable to generate the new video sequence by setting, as the image area with the high confidence level, a block with a confidence level greater than a predetermined value, and setting, as the image area with the low confidence level, a block with a confidence level equal to or smaller than the predetermined value.

7. The image generation apparatus according to claim 4 ,

wherein said motion estimation unit further includes an attitude sensor signal receiving unit operable to receive a signal from an attitude sensor that detects an attitude of an imaging device which images an object, and is operable to estimate the motion of the same object using the signal received by said attitude sensor receiving unit.

8. The image generation apparatus according to claim 1 ,

wherein said image integration unit is operable to extract chrominance information from the first video sequence, to generate a new intermediate video sequence from the second video sequence and luminance information obtained from the first video sequence, and to generate the new video sequence by adding the chrominance information to the new intermediate video sequence.

9. The image generation apparatus according to claim 1 ,

wherein said image integration unit is operable to calculate an amount of temporal change between frames of at least one of the plurality of video sequences, and to generate the new video sequence by setting a temporal boundary between frames between which the calculated amount of temporal change exceeds a predetermined value.

10. The image generation apparatus according to claim 1 ,

wherein said image integration unit is further operable to calculate a value indicating a confidence level of the new video sequence, and to output the calculated value and the new video sequence.

11. An image generation method for generating a new video sequence from a plurality of video sequences, said image generation method comprising:

receiving a first video sequence including frames having a first resolution and a second video sequence including frames having a second resolution which is higher than the first resolution, each frame of the first video sequence being obtained with a first exposure time, and each frame of the second video sequence being obtained with a second exposure time which is longer than the first exposure time; and

generating, from the first video sequence and the second video sequence, the new video sequence including frames having a resolution which is equal to or higher than the second resolution, at a frame rate which is equal to or higher than a frame rate of the first video sequence, by reducing a difference between a value of each frame of the second video sequence and a sum of values of frames of the new video sequence which are included within an exposure period of a frame of the second video sequence.

12. A computer-readable medium having a computer program recorded thereon, said computer program for generating a new video sequence from a plurality of video sequences, said computer program causing a computer to execute:

receiving a first video sequence including frames having a first resolution and a second video sequence including frames having a second resolution which is higher than the first resolution, each frame of the first video sequence being obtained with a first exposure time, and each frame of the second video sequence being obtained with a second exposure time which is longer than the first exposure time; and

generating, from the first video sequence and the second video sequence, the new video sequence including frames having a resolution which is equal to or higher than the second resolution, at a frame rate which is equal to or higher than a frame rate of the first video sequence, by reducing a difference between a value of each frame of the second video sequence and a sum of values of frames of the new video sequence which are included within an exposure period of a frame of the second video sequence.