Super-resolution device, super-resolution method, super-resolution program, and super-resolution system
View Patent ↗An albedo estimating section produces an albedo image of an object from an original image captured by an image-capturing section by using light source information estimated by a light source information estimating section and shape information of the object obtained by a shape information obtaining section. An albedo super-resolution section performs super-resolution of the albedo image according to a conversion rule obtained from an albedo DB. A super-resolution section produces a high-resolution image obtained by performing super-resolution of the original image by using the super-resolution albedo image, the light source information and the shape information.
1. A super-resolution device, comprising:
an image-capturing section for imaging an object by an imaging device;
a light source information estimating section for estimating light source information including at least one of an illuminance, a direction and a position of a light source illuminating the object;
a shape information obtaining section for obtaining, as shape information, surface normal information or three-dimensional position information of the object;
an albedo estimating section for producing an albedo image of the object from an original image captured by the image-capturing section by using the light source information and the shape information;
an albedo database storing a conversion rule for converting a low-resolution albedo image to a high-resolution albedo image;
an albedo super-resolution section for obtaining the conversion rule from the albedo database to perform super-resolution of the albedo image produced by the albedo estimating section according to the conversion rule; and
a super-resolution section for producing a high-resolution image resolution-increased from the original image by using the high-resolution albedo image obtained by the albedo super-resolution section, the light source information and the shape information.
2. The super-resolution device of claim 1 , comprising a diffuse reflection/specular reflection separating section for separating the original image into a diffuse reflection image and a specular reflection image,
wherein the albedo estimating section produces an albedo image from the diffuse reflection image separated by the diffuse reflection/specular reflection separating section, instead of the original image.
3. The super-resolution device of claim 1 , comprising a diffuse reflection/specular reflection separating section for separating the original image into a diffuse reflection image and a specular reflection image, wherein:
the image-capturing section obtains a polarization state of the object; and
the diffuse reflection/specular reflection separating section performs the separation by using the polarization state obtained by the image-capturing section.
4. The super-resolution device of claim 1 , wherein the conversion rule stored in the albedo database is obtained by a learning process using an albedo image having the same resolution as the original image and an albedo image having a higher resolution than the original image.
5. The super-resolution device of claim 1 , comprising a super-resolution determination section for estimating a reliability of a super-resolution process according to the conversion rule stored in the albedo database for an albedo image produced by the albedo estimating section,
wherein when the reliability is evaluated to be low by the super-resolution determination section, the albedo super-resolution section performs super-resolution of the albedo image without using the conversion rule stored in the albedo database.
6. An super-resolution device, comprising:
an image-capturing section for imaging an object by an imaging device;
a light source information estimating section for estimating light source information including at least one of a direction and a position of a light source illuminating the object;
a shape information obtaining section for obtaining, as shape information, surface normal information or three-dimensional position information of the object;
an albedo estimating section for producing a pseudo-albedo image of the object from an original image captured by the image-capturing section by using the light source information and the shape information;
an albedo database storing a conversion rule for converting a low-resolution pseudo-albedo image to a high-resolution pseudo-albedo image;
an albedo super-resolution section for obtaining the conversion rule from the albedo database to increase a resolution of the pseudo-albedo image produced by the albedo estimating section according to the conversion rule; and
a super-resolution section for producing a high-resolution image resolution-increased from the original image by using the high-resolution pseudo-albedo image obtained by the albedo super-resolution section, the light source information and the shape information.
7. The super-resolution device of claim 6 , comprising a diffuse reflection/specular reflection separating section for separating the original image into a diffuse reflection image and a specular reflection image,
wherein the albedo estimating section produces a pseudo-albedo image from the diffuse reflection image separated by the diffuse reflection/specular reflection separating section, instead of the original image.
8. The super-resolution device of claim 6 , comprising a diffuse reflection/specular reflection separating section for separating the original image into a diffuse reflection image and a specular reflection image, wherein:
the image-capturing section obtains a polarization state of the object; and
the diffuse reflection/specular reflection separating section performs the separation by using the polarization state obtained by the image-capturing section.
9. The super-resolution device of claim 6 , wherein the conversion rule stored in the albedo database is obtained by a learning process using a pseudo-albedo image having the same resolution as the original image and a pseudo-albedo image having a higher resolution than the original image.
10. The super-resolution device of claim 6 , comprising a super-resolution determination section for estimating a reliability of a super-resolution process according to the conversion rule stored in the albedo database for a pseudo-albedo image produced by the albedo estimating section,
wherein when the reliability is evaluated to be low by the super-resolution determination section, the albedo super-resolution section performs super-resolution of the pseudo-albedo image without using the conversion rule stored in the albedo database.
11. The super-resolution device of claim 1 or 6 , comprising a diffuse reflection/specular reflection separating section for separating the original image into a diffuse reflection image and a specular reflection image, wherein:
the super-resolution section performs super-resolution of the specular reflection image separated by the diffuse reflection/specular reflection separating section; and
the super-resolution section produces the high-resolution image by using the super-resolution specular reflection image.
12. The super-resolution device of claim 11 , wherein the super-resolution section performs super-resolution of the specular reflection image by using a process of increasing a resolution of the shape information.
13. A super-resolution method, comprising:
a first step of obtaining an original image by imaging an object;
a second step of estimating light source information including at least one of an illuminance, a direction and a position of a light source illuminating the object;
a third step of obtaining, as shape information, surface normal information or three-dimensional position information of the object;
a fourth step of producing an albedo image of the object from the original image by using the light source information and the shape information;
a fifth step of obtaining a conversion rule from an albedo database storing a conversion rule for converting a low-resolution albedo image to a high-resolution albedo image to perform super-resolution of the albedo image according to the conversion rule; and
a sixth step of producing a high-resolution image resolution-increased from the original image by using the high-resolution albedo image obtained in the fifth step, the light source information and the shape information.
14. A non-transitory computer-readable recording medium for storing therein a super-resolution program for instructing a computer to perform:
a first step of producing an albedo image of an object from an original image obtained by imaging the object by using light source information including at least one of an illuminance, a direction and a position of a light source illuminating the object, and shape information being surface normal information or three-dimensional position information of the object;
a second step of obtaining a conversion rule from an albedo database storing a conversion rule for converting a low-resolution albedo image to a high-resolution albedo image to perform super-resolution of the albedo image according to the conversion rule; and
a third step of producing a high-resolution image resolution-increased from the original image by using the high-resolution albedo image obtained in the second step, the light source information and the shape information.