Techniques for rapid stereo reconstruction from images
View Patent ↗Stereo image reconstruction techniques are described. An image from a root viewpoint is translated to an image from another viewpoint. Homography fitting is used to translate the image between viewpoints. Inverse compositional image alignment is used to determine a homography matrix and determine a pixel in the translated image.
1. A computer-implemented method comprising:
performing stereo matching on an image pair including left and right images;
applying a homography matrix transform to points in a root image plane to produce correspondence points in a translated image plane to optimize photo-consistency between a pixel in one of the left and right images and a corresponding other one of the left and right images;
triangulating camera parameters with the correspondence points that were produced from the applied homography matrix transform to generate a three-dimensional scene; and
providing for display the three-dimensional scene generated from triangulating camera parameters with the correspondence points.
2. The method of claim 1 , further comprising:
generating a homography matrix transform using an inverse compositional image alignment (ICIA).
3. The method of claim 1 , further comprising:
generating a homography matrix transform;
defining photo-consistency between a root image and a warp window of a translated image as an implicit function of the homography matrix transform; and
improving photo-consistency of the warp window.
4. The method of claim 1 , wherein the generating a translated pixel from a root pixel comprises: using homograph-fitting to improve photo-consistency between the root pixel and the translated pixel.
5. The method of claim 1 , wherein the homography matrix comprises:
H ( p )= K l ·( R−tp T ) K r −1 ,
where K l and K r are the intrinsic matrices of respective left and right cameras,
R is the rotation matrix between the two cameras,
t is the translation vector between the two cameras, and
p=n/d, where n is the normal vector of the plane π and d is the distance from the left camera center to the plane π.
6. The method of claim 1 , further comprising generating a three-dimensional scene from multiple stereo image pairs.
7. A non-transitory computer-readable medium storing instructions, that when performed by a computer, cause the computer to:
perform stereo matching on an image pair including left and right images;
apply a homography matrix transform to points in a root image plane to produce correspondence points in a translated image plane to optimize photo-consistency between a pixel in one of the left and right images and a corresponding other one of the left and right images;
triangulate camera parameters with the correspondence points that were produced from the applied homography matrix transform to generate a three-dimensional scene; and
provide for display the three-dimensional scene generated from triangulating camera parameters with the correspondence points.
8. The computer-readable medium of claim 7 , wherein to apply stereo matching, the computer is to apply normalized cross correlation (NCC).
9. The computer-readable medium of claim 7 , further comprising instructions, which when executed by the computer, cause the computer to:
generate the homography matrix transform using an inverse compositional image alignment (ICIA).
10. The computer-readable medium of claim 7 , further comprising instructions, which when executed by the computer, cause the computer to:
generate a homography matrix transform;
define photo-consistency between a root image and a warp window of a translated image as an implicit function of the homography matrix transform; and
improve photo-consistency of the warp window.
11. The computer-readable medium of claim 7 , wherein to generate a translated pixel from a root pixel, the computer is to improve photo-consistency between the root pixel and the translated pixel.
12. The computer-readable medium of claim 7 , wherein the homography matrix comprises:
H ( p )= K l ·( R−tp T ) K r −1 ,
where K l and K r are the intrinsic matrices of respective left and right cameras,
R is the rotation matrix between the two cameras,
t is the translation vector between the two cameras, and
p=n/d, where n is the normal vector of the plane π and d is the distance from the left camera center to the plane π.
13. An apparatus comprising:
a processor configured to:
perform stereo matching on an image pair including left and right images;
apply a homography matrix transform to points in a root image plane to produce correspondence points in a translated image plane to optimize photo-consistency between a pixel in one of the left and right images and a corresponding other one of the left and right images,
triangulate camera parameters with the correspondence points that were produced from the applied homography matrix transform to generate a three-dimensional scene; and
provide for display the three-dimensional scene generated from triangulating camera parameters with the correspondence points.
14. The apparatus of claim 13 , wherein the processor is at least one of a multi-core processor and a graphics processing unit.
15. The apparatus of claim 13 , wherein the processor is to:
generate the homography matrix transform using an inverse compositional image alignment (ICIA).
16. The apparatus of claim 13 , wherein to apply a homography matrix, the processor is to:
generate a homography matrix transform;
define photo-consistency between a root image and a warp window of a translated image as an implicit function of the homography matrix transform; and
improve photo-consistency of the warp window.
17. The apparatus of claim 13 , wherein to generate a translated pixel from a root pixel, the computer is to improve photo-consistency between the root pixel and the translated pixel.
18. A system comprising:
a display device and
a computer system, wherein the computer system is communicatively coupled to the display device and wherein the computer system is to:
perform stereo matching on an image pair including left and right images;
apply a homography matrix transform to points in a root image plane to produce correspondence points in a translated image plane to optimize photo-consistency between a pixel in one of the left and right images and a corresponding other one of the left and right images;
triangulate camera parameters with the correspondence points that were produced from the applied homography matrix transform to generate a three-dimensional scene; and
provide for display the three-dimensional scene generated from triangulating camera parameters with the correspondence points.
19. The system of claim 18 , wherein the computer system is to:
generate the homography matrix transform using an inverse compositional image alignment (ICIA).
20. The system of claim 18 , wherein to apply a homography matrix, the processor is to:
generate a homography matrix transform;
define photo-consistency between a root image and a warp window of a translated image as an implicit function of the homography matrix transform; and
improve photo-consistency of the warp window.
21. The system of claim 18 , wherein to generate a translated pixel from a root pixel, the computer system is to improve photo-consistency between the root pixel and the translated pixel.