Three-dimensional image processing apparatus, three-dimensional image-pickup apparatus, three-dimensional image-pickup method, and program
View Patent ↗A sense of three-dimensionality and thickness is restored to a subject and a high-quality three-dimensional image with a low sense of a cardboard cutout effect is obtained, regardless of the cause of the cardboard cutout effect. In a three-dimensional image capturing apparatus (three-dimensional image processing apparatus) ( 1000 ), a depth generation unit ( 103 ) obtains L depth information and R depth information from a three-dimensional image, and an image correction unit ( 104 ) executes a shadow enhancement process that is stronger on the inside of an object than on the outside of the object, based on the L depth information and the R depth information. Through this, the occurrence of the cardboard cutout effect can be suppressed, and a high-quality three-dimensional image with a reduced cardboard cutout effect can be obtained.
1. A three-dimensional image processing apparatus that performs an image correction process on a left eye image and a right eye image contained in a three-dimensional image obtained through a dual-lens technique or a multiple-viewpoint technique, the apparatus comprising:
an image corrector configured to:
obtain a subject object from the left eye image and right eye image;
perform an enhancement process configured to emphasize a shadow or shade produced by protrusions and recesses in a surface of the subject object using a first strength on an inward region of the obtained subject object; and
perform an enhancement process configured to emphasize a shadow or shade produced by the protrusions and recesses in the surface of the subject object using a second strength that is weaker than the first strength on an outward region of the obtained subject object; wherein
the image corrector includes an R image corrector and an L image corrector, and
the R image corrector includes:
an R image local tone converter configured to perform the enhancement process on a right eye image signal IS_R and output the right eye image signal on which the enhancement process has been performed as a corrected right eye image signal OS_R;
an R image strength generator configured to determine a strength of an image correction process based on the right eye image distance image; and
an R image synthesizer configured to synthesize the right eye image signal IS_R and the corrected right eye image signal OS_R based on the strength determined by the R image strength generator; and
the L image corrector includes:
an L image local tone converter configured to perform the enhancement process on a left eye image signal IS_L and output the left eye image signal on which the enhancement process has been performed as a corrected left eye image signal OS_L;
an L image strength generator configured to determine a strength of an image correction process based on the left eye image distance image; and
an L image synthesizer configured to synthesize the left eye image signal IS_L and the corrected left eye image signal OS_L based on the strength determined by the L image strength generator.
2. The three-dimensional image processing apparatus according to claim 1 , wherein the second strength is zero.
3. The three-dimensional image processing apparatus according to claim 1 , wherein:
the R image local tone converter includes:
an R image surrounding brightness detector configured to:
detect a first representative brightness value of a first pixel of interest corresponding to the right eye image signal IS_R and a first periphery pixel in the periphery of the first pixel of interest; and
output an R image surrounding brightness signal US_R that takes the first detected representative brightness value as a signal value; and
an R image second dynamic tone corrector configured to obtain the corrected right eye image signal OS_R by performing a first dynamic tone correction process based on the right eye image signal IS_R and the R image surrounding brightness signal US_R, the first dynamic tone correction process including:
obtaining the corrected right eye image signal OS_R by performing a first tone conversion process based on a first tone conversion property in which when the value of the right eye image signal IS_R is fixed at a first predetermined value within a predetermined input range of the right eye image signal IS_R, and the value of the corrected right eye image signal OS_R decreases as the value of the R image surrounding brightness signal US_R increases when the value of the right eye image signal IS_R is less than or equal to the value of the R image surrounding brightness signal US_R; and
obtaining the corrected right eye image signal OS_R by taking the right eye image signal IS_R as the corrected right eye image signal OS_R when the value of the right eye image signal IS_R is less than the value of the R image surrounding brightness signal US_R; and
wherein the L image local tone converter includes:
an L image surrounding brightness detector configured to:
detect a second brightness value of a second pixel of interest corresponding to the left eye image signal IS_L and a second periphery pixel in the periphery of the second pixel of interest; and
output an L image surrounding brightness signal US_L that takes the detected second brightness value as a signal value; and
an L image second dynamic tone corrector configured to:
obtain the corrected left eye image signal OS_L by performing a second dynamic tone correction process based on the left eye image signal IS_L and the L image surrounding brightness signal US_L, the second dynamic tone correction process including:
obtaining the corrected left eye image signal OS_L by performing a second tone conversion process based on a second tone conversion property in which, when the value of the left eye image signal IS_L is fixed at a second predetermined value within a predetermined input range of the left eye image signal IS_L, and the value of the corrected left eye image signal OS_L decreases as the value of the L image surrounding brightness signal US_L increases when the value of the left eye image signal IS_L is less than or equal to the value of the L image surrounding brightness signal US_L; and
obtaining the corrected left eye image signal OS_L by taking the left eye image signal IS_L as the corrected left eye image signal OS_L when the value of the left eye image signal IS_L is greater than the value of the L image surrounding brightness signal US_L.
4. The three-dimensional image processing apparatus according to claim 1 ,
wherein the R image local tone converter includes:
an R image second surrounding brightness detector configured to:
detect a first representative brightness value of a first pixel of interest corresponding to the right eye image signal IS_R, and a first periphery pixel in the periphery of the first pixel of interest;
obtain an R image surrounding brightness signal US_R that takes the first detected representative brightness value as a signal value;
obtain a right eye offset value ΔUS_R that takes on a higher value the greater an amount of change in the right eye image signal IS_R is in a predetermined image region; and
obtain an R image corrected surrounding brightness signal US_R′ by adding the right eye offset value ΔUS_R to the R image surrounding brightness signal US_R; and
an R image dynamic tone corrector configured to obtain the corrected right eye image signal OS_R by performing a dynamic tone correction process based on the right eye image signal IS_R and the R image corrected surrounding brightness signal US_R′, the dynamic tone correction process obtaining the corrected right eye image signal OS_R by performing a tone conversion process based on a tone conversion property in which the value of the corrected right eye image signal OS_R decreases as the value of the R image corrected surrounding brightness signal US_R′ increases when the value of the right eye image signal IS_R is fixed at a predetermined value in a predetermined input range of the right eye image signal IS_R; and
wherein the L image local tone converter includes:
an L image second surrounding brightness detector configured to:
detect a second brightness value of a second pixel of interest corresponding to the left eye image signal IS_L, and a second periphery pixel in the periphery of the second pixel of interest;
obtain an L image surrounding brightness signal US_L that takes the second detected brightness value as a signal value;
obtain a left eye offset value ΔUS_L that takes on a higher value the greater an amount of change in the left eye image signal IS_L is in a predetermined image region; and
obtain an L image corrected surrounding brightness signal US_L′ by adding the left eye offset value ΔUS_L to the L image surrounding brightness signal US_L; and
an L image dynamic tone corrector configured to obtain the corrected left eye image signal OS_L by performing a dynamic tone correction process based on the left eye image signal IS_L and the L image corrected surrounding brightness signal US_L′, the dynamic tone correction process obtaining the corrected left eye image signal OS_L by performing a tone conversion process based on a tone conversion property in which the value of the corrected left eye image signal OS_L decreases as the value of the L image corrected surrounding brightness signal US_L′ increases when the value of the left eye image signal IS_L is fixed at a predetermined value in a predetermined input range of the left eye image signal IS_L.
5. The three-dimensional image processing apparatus according to claim 1 , wherein:
the R image local tone converter includes:
an R image second surrounding brightness detector configured to:
detect a first representative brightness value of a first pixel of interest corresponding to the right eye image signal IS_R, and a first periphery pixel in the periphery of the first pixel of interest;
obtain an R image surrounding brightness signal US_R that takes the first detected brightness value as a signal value;
obtain a right eye offset value ΔUS_R that takes on a higher value the greater an amount of change in the right eye image signal IS_R is in a predetermined image region; and
obtain an R image corrected surrounding brightness signal US_R′ by adding the right eye offset value ΔUS_R to the R image surrounding brightness signal US_R; and
an R image coefficient computation processor configured to:
determine a coefficient k that takes on a lower value the higher the value of a difference between the right eye image signal IS_R and the R image corrected surrounding brightness signal US_R′ according to the formula ((IS_R)−(US_R′)); and
obtain the corrected right eye image signal OS_R according to the formula OS_R=IS_R+k×((IS_R)−(US_R′)) using the determined coefficient k; and
wherein the L image local tone converter includes:
an L image second surrounding brightness detector configured to:
detect a second representative brightness value of a second pixel of interest corresponding to the left eye image signal IS_L, and a second periphery pixel in the periphery of the first pixel of interest;
obtain an L image surrounding brightness signal US_L that takes the second detected representative brightness value as a signal value;
obtains a left eye offset value ΔUS_L that takes on a higher value the greater an amount of change in the left eye image signal IS_L is in a predetermined image region; and
obtain an L image corrected surrounding brightness signal US_L′ by adding the left eye offset value ΔUS_L to the L image surrounding brightness signal US_L; and
an L image coefficient computation processor configured to:
determine a coefficient k′ that takes on a lower value the higher the value of a difference between the left eye image signal IS_L and the L image corrected surrounding brightness signal US_L′ according to the formula ((IS_L)−(US_L′)); and
obtain the corrected left eye image signal OS_L through OS_L=IS_L+k′×((IS_L)−(US_L′)) using the determined coefficient k′.
6. The three-dimensional image processing apparatus according to claim 1 , wherein:
the R image local tone converter includes:
an R image second surrounding brightness detector configured to:
detect a first representative brightness value of a first pixel of interest corresponding to the right eye image signal IS_R, and a first periphery pixel in the periphery of the pixel of interest;
obtain an R image surrounding brightness signal US_R that takes the detected first representative brightness value as a signal value;
obtain a right eye offset value ΔUS_R that takes on a higher value the greater an amount of change in the right eye image signal IS_R is in a predetermined image region;
and obtain an R image corrected surrounding brightness signal US_R′ by adding the right eye offset value ΔUS_R to the R image surrounding brightness signal US_R; and
an R image coefficient computation processor configured to:
determine a coefficient k that takes on a lower value the higher the value of a difference between the right eye image signal IS_R and the R image corrected surrounding brightness signal US_R′ according to the formula ((IS_R)−(US_R′));
obtain a signal LPF ((k+p)×(IS_R−US_R′)) by setting a coefficient p, such that 0 is less than or equal to p which is less than or equal to 1, and performing a bandwidth limiting process on a signal obtained according to the formula (k+p)×(IS_R−US_R′); and
obtain the corrected right eye image signal OS_R according to the formula OS_R=IS_R−p×(IS_R−US_R′)+LPF((k+p)×(IS_R−US_R′)) using the obtained signal LPF((k+p)×(IS_R−US_R′)), and
wherein the L image local tone converter includes:
an L image second surrounding brightness detector configured to:
detect a second representative brightness value of a second pixel of interest corresponding to the left eye image signal IS_L, and second periphery a pixel in the periphery of the second pixel of interest;
obtain an L image surrounding brightness signal US_L that takes the second detected brightness value as a signal value;
obtain a left eye offset value ΔUS_L that takes on a higher value the greater an amount of change in the left eye image signal IS_L is in a predetermined image region; and
obtain an L image corrected surrounding brightness signal US_L′ by adding the left eye offset value ΔUS_L to the L image surrounding brightness signal US_L; and
an L image coefficient computation processor configured to:
determine a coefficient k′ that takes on a lower value the higher the value of a difference between the left eye image signal IS_L and the L image corrected surrounding brightness signal US_L′ according to the formula ((IS_L)−(US_L′));
obtain a signal LPF ((k′+p)×(IS_L−US_L′)) by setting a coefficient p, such that 0 is less than or equal to p which is less than or equal to 1, and performing a bandwidth limiting process on a signal obtained according to the formula (k+p)×(IS_L−US_L′); and
obtain the corrected left eye image signal OS_L according to the formula OS_L=IS_L−p×(IS_L−US_L′)+LPF((k′+p)×(IS_L−US_L′)) using the obtained signal LPF((k′+p)×(IS_L−US_L′)).