Hierarchical processing using image deformation
A hierarchical pattern matching process is improved for use in three-dimensional reconstruction by calculating a disparity field for low-resolution data, and using this low-resolution disparity field to estimate the disparity field for higher-resolution images. By pre-warping a template according to the estimated disparity field, improved correlation results can be obtained.
1. A method comprising the steps of:
obtaining a first image and a second image of an object from two camera poses;
generating a hierarchy of lower-resolution representations of the first image and the second image;
obtaining a disparity field that characterizes motion between a lower-resolution representation of the first image and a corresponding one of the lower-resolution representations of the second image;
upsampling the disparity field to a higher resolution of the hierarchy to provide an estimated disparity field at the higher resolution;
warping a higher-resolution representation of the first image by distorting the higher-resolution representation according to the upsampled disparity field to align a processing mesh for the higher-resolution representation of the first image with a higher-resolution representation of the second image, thereby providing a warped image;
correlating the warped image to the higher-resolution representation of the second image to obtain a disparity error field; and
combining the upsampled disparity field and the disparity error field to obtain a higher-resolution disparity field.
2. The method of claim 1 further comprising repeating the steps of the method for a next higher-resolution level of the hierarchy.
3. The method of claim 1 wherein the higher-resolution representation of the first image is a full resolution image.
4. The method of claim 1 wherein the lower-resolution representation of the first image has a smallest size in the hierarchy.
5. The method of claim 1 further comprising applying the higher-resolution disparity field to a camera model to obtain a three-dimensional surface reconstruction of the object.
6. The method of claim 1 wherein the hierarchy includes at least one of a three-level pyramid, a four-level pyramid, and a five-level pyramid.
7. The method of claim 1 wherein the first image and the second image have a size of 96 pixels in one dimension.
8. The method of claim 1 wherein the first image and the second image have a size of 128 pixels in one dimension.
9. The method of claim 1 wherein the two camera poses correspond to two offset optical channels in a multi-aperture three-dimensional camera.
10. The method of claim 1 wherein the two camera poses correspond to two independent camera positions.
11. A computer program product comprising computer-executable code embodied on a non-transitory computer-readable medium that, when executing on one or more computing devices, performs the steps of:
obtaining a first image and a second image of an object from two camera poses;
generating a hierarchy of lower-resolution representations of the first image and the second image;
obtaining a disparity field that characterizes motion between a lower-resolution representation of the first image and a corresponding one of the lower-resolution representations of the second image;
upsampling the disparity field to a higher resolution of the hierarchy to provide an estimated disparity field at the higher resolution;
warping a higher-resolution representation of the first image by distorting the higher-resolution representation according to the upsampled disparity field to align a processing mesh for the higher-resolution representation of the first image with a higher-resolution representation of the second image, thereby providing a warped image;
correlating the warped image to the higher-resolution representation of the second image to obtain a disparity error field; and
combining the upsampled disparity field and the disparity error field to obtain a higher-resolution disparity field.
12. The computer program product of claim 11 further comprising code for repeating the steps for a next higher-resolution level of the hierarchy.
13. The computer program product of claim 11 wherein the higher-resolution representation of the first image is a full resolution image.
14. The computer program product of claim 11 wherein the lower-resolution representation of the first image has a smallest size in the hierarchy.
15. The computer program product of claim 11 further comprising code for performing the step of applying the higher-resolution disparity field to a camera model to obtain a three-dimensional surface reconstruction of the object.
16. The computer program product of claim 11 wherein the hierarchy includes at least one of a three-level pyramid, a four-level pyramid, and a five-level pyramid.
17. The computer program product of claim 11 wherein the first image and the second image have a size of 96 pixels in one dimension.
18. The computer program product of claim 11 wherein the first image and the second image have a size of 128 pixels in one dimension.
19. The computer program product of claim 11 wherein the two camera poses correspond to two offset optical channels in a multi-aperture three-dimensional camera.
20. The computer program product of claim 11 wherein the two camera poses correspond to two independent camera positions.