Signal processing apparatus and signal processing method
Provided is a signal processing apparatus that estimates image distortion in a case where images are captured through a transmissive body allowing light to pass through. The signal processing apparatus includes a lens distortion estimation section that estimates lens distortion based on a location of a feature point in a first image and a second image of an object. The first image is captured by an imaging section through a transmissive body and a lens that allow light to pass through. The second image is free of transmissive body distortion caused by the transmissive body and free of the lens distortion caused by the lens. The apparatus further includes a transmissive body distortion estimation section that estimates the transmissive body distortion based on the location of the feature point in the first image and a third image that is obtained by removing the estimated lens distortion from the first image.
1. A signal processing apparatus, comprising:
a central processing unit (CPU) configured to:
estimate lens distortion in a first image of an object based on a location of a feature point in the first image and a location of a feature point in a second image of the object, wherein
the first image is captured by a camera through a transmissive body and a lens,
the lens distortion is caused by the lens,
the transmissive body and the lens allow light to pass through, and
the second image is free of:
transmissive body distortion caused by the transmissive body, and
the lens distortion caused by the lens;
remove the estimated lens distortion from the first image to obtain a third image; and
estimate the transmissive body distortion based on the location of the feature point in the first image and a location of a feature point in the third image.
2. The signal processing apparatus according to claim 1 , wherein
the CPU is further configured to remove the estimated transmissive body distortion and the estimated lens distortion from a fourth image, and
the fourth image is captured by the camera through the transmissive body and the lens.
3. The signal processing apparatus according to claim 2 , wherein the CPU is further configured to:
generate a distortion correction table indicating correspondence between:
a location of a pixel in the second image free of the transmissive body distortion and the lens distortion, and
a location of a pixel in the first image that includes the transmissive body distortion and the lens distortion; and
remove the estimated transmissive body distortion and the estimated lens distortion from the fourth image based on the distortion correction table.
4. The signal processing apparatus according to claim 2 , wherein the CPU is further configured to:
estimate a lens distortion function indicative of the lens distortion;
estimate a transmissive body distortion function indicative of the transmissive body distortion; and
remove the estimated transmissive body distortion and the estimated lens distortion from the fourth image based on by using the transmissive body distortion function and the lens distortion function.
5. The signal processing apparatus according to claim 1 , wherein
the CPU is further configured to:
estimate the lens distortion based on a lens distortion model; and
estimate the transmissive body distortion based on a transmissive body distortion model, and
the transmissive body distortion model is different from the lens distortion model.
6. The signal processing apparatus according to claim 1 , wherein the CPU is further configured to:
estimate a plurality of internal parameters of the camera and a plurality of external parameters of the camera; and
estimate the lens distortion based on the estimated plurality of internal parameters and the estimated plurality of external parameters.
7. The signal processing apparatus according to claim 6 , wherein the CPU is further configured to:
add the estimated lens distortion and the estimated transmissive body distortion to the second image to obtain a fifth image;
calculate a reprojection error that indicates a difference between the location of the feature point in the first image and a location of a feature point in the fifth image; and
determine, based on the reprojection error, whether a process of the estimation of the lens distortion and the estimation of the transmissive body distortion has converged.
8. The signal processing apparatus according to claim 1 , wherein the CPU is further configured to:
detect the feature point in the first image and the feature point in the third image;
calculate the location of the feature point in the second image based on a plurality of internal parameters of the camera and a plurality of external parameters of the camera;
estimate the lens distortion based on the location of the detected feature point in the first image and the calculated location of the feature point in the second image; and
estimate the transmissive body distortion based on the location of the detected feature point in the first image and the location of the detected feature point in the third image.
9. The signal processing apparatus according to claim 1 , wherein the CPU is further configured to remove the estimated lens distortion from the first image to generate the third image.
10. The signal processing apparatus according to claim 1 , wherein the object has a pattern.
11. A signal processing method, comprising:
estimating lens distortion in a first image of an object based on a location of a feature point in the first image and a location of a feature point in a second image of the object, wherein
the first image is captured by a camera through a transmissive body and a lens,
the lens distortion is caused by the lens,
the transmissive body and the lens allow light to pass through, and
the second image is free of:
transmissive body distortion caused by the transmissive body, and
the lens distortion caused by the lens;
removing the estimated lens distortion from the first image to obtain a third image; and
estimating the transmissive body distortion based on the location of the feature point in the first image and a location of a feature point in the third image.
12. A non-transitory computer-readable medium having stored thereon computer-executable instructions which, when executed by a processor, cause the processor to execute operations, the operations comprising:
estimating lens distortion in a first image of an object based on a location of a feature point in the first image and a location of a feature point in a second image of the object, wherein
the first image is captured by a camera through a transmissive body and a lens,
the lens distortion is caused by the lens,
the transmissive body and the lens allow light to pass through, and
the second image is free of:
transmissive body distortion caused by the transmissive body, and
the lens distortion caused by the lens;
removing the estimated lens distortion from the first image to obtain a third image; and
estimating the transmissive body distortion based on the location of the feature point in the first image and a location of a feature point in the third image.