Techniques for rapid stereo reconstruction from images
View Patent ↗Stereo image reconstruction techniques are described. An image from a root viewpoint is translated to an image from another viewpoint. Homography fitting is used to translate the image between viewpoints. Inverse compositional image alignment is used to determine a homography matrix and determine a pixel in the translated image.
1. A computer-implemented method to reconstruct a point on a three dimensional surface comprising:
performing stereo matching on an image pair;
rectifying the image pair so that epipolar lines become one of horizontal or vertical;
applying stereo matching to the rectified image pair;
generating a translated pixel from a root pixel, wherein the generating comprises applying a homography matrix transform to transform a pixel in a root pixel plane to a pixel in a translated image plane via a plane tangent to said three dimensional surface;
triangulating correspondence points to generate a three-dimensional scene; and
providing the three-dimensional scene for display.
2. The method of claim 1 , wherein the applying stereo matching comprises applying normalized cross correlation (NCC).
3. The method of claim 1 , further comprising:
generating a homography matrix transform using an inverse compositional image alignment (ICIA).
4. The method of claim 1 , further comprising:
generating a homography matrix transform;
defining photo-consistency between a root image and a warp window of a translated image as an implicit function of the homography matrix transform; and
improving photo-consistency of the warp window.
5. The method of claim 1 , wherein the generating a translated pixel from a root pixel comprises: using homograph-fitting to improve photo-consistency between the root pixel and the translated pixel.
6. The method of claim 1 , wherein the homography matrix comprises:
H ( p )= K l ·( R−tp T ) K r −1 ,
where K l and K r are the intrinsic matrices of respective left and right cameras,
R is the rotation matrix between the two cameras,
t is the translation vector between the two cameras, and
p=n/d, where n is the normal vector of the plane π and d is the distance from the left camera center to the plane π.
7. The method of claim 1 , further comprising generating a three-dimensional scene from multiple stereo image pairs.
8. A non-transitory computer-readable medium storing instructions, that when performed by a computer to reconstruct a point on a three dimensional surface, cause the computer to:
perform stereo matching on an image pair;
rectify the image pair;
apply stereo matching to the rectified image pair;
generate a translated pixel from a root pixel, wherein to generate the computer is to apply a homography matrix transform to transform a pixel in a root pixel plane to a pixel in a translated image plane via a plane tangent to said three dimensional surface;
triangulate correspondence points to generate a three-dimensional scene; and
provide the three-dimensional scene for display.
9. The computer-readable medium of claim 8 , wherein to apply stereo matching, the computer is to apply normalized cross correlation (NCC).
10. The computer-readable medium of claim 8 , further comprising instructions, which when executed by the computer, cause the computer to:
generate the homography matrix transform using an inverse compositional image alignment (ICIA).
11. The computer-readable medium of claim 8 , further comprising instructions, which when executed by the computer, cause the computer to:
generate a homography matrix transform;
define photo-consistency between a root image and a warp window of a translated image as an implicit function of the homography matrix transform; and
improve photo-consistency of the warp window.
12. The computer-readable medium of claim 8 , wherein to generate a translated pixel from a root pixel, the computer is to improve photo-consistency between the root pixel and the translated pixel.
13. The computer-readable medium of claim 8 , wherein the homography matrix comprises:
H ( p )= K l ·( R−tp T ) K r −1 ,
where K l and K r are the intrinsic matrices of respective left and right cameras,
R is the rotation matrix between the two cameras,
t is the translation vector between the two cameras, and
p=n/d, where n is the normal vector of the plane π and d is the distance from the left camera center to the plane π.
14. An apparatus to reconstruct a point on a three dimensional surface comprising:
a processor configured to:
generate a translated pixel from a root pixel, wherein to generate, the computer is to apply a homography matrix transform to transform a pixel in a root pixel plane to a pixel in a translated image plane via a plane tangent to said three dimensional surface;
triangulate correspondence points to generate a three-dimensional scene; and
provide the three-dimensional scene for display.
15. The apparatus of claim 14 , wherein the processor is at least one of a multi-core processor and a graphics processing unit.
16. The apparatus of claim 14 , wherein the processor is to:
generate the homography matrix transform using an inverse compositional image alignment (ICIA).
17. The apparatus of claim 14 , wherein to apply a homography matrix, the processor is to:
generate a homography matrix transform;
define photo-consistency between a root image and a warp window of a translated image as an implicit function of the homography matrix transform; and
improve photo-consistency of the warp window.
18. The apparatus of claim 14 , wherein to generate a translated pixel from a root pixel, the computer is to improve photo-consistency between the root pixel and the translated pixel.
19. A system to reconstruct a point on a three dimensional surface comprising:
a display device and
a computer system, wherein the computer system is communicatively coupled to the display device and wherein the computer system is to:
generate a translated pixel from a root pixel, wherein to generate the computer is to apply a homography matrix transform to transform a pixel in a root pixel plane to a pixel in a translated image plane via a plane tangent to said three dimensional surface;
triangulate correspondence points to generate a three-dimensional scene; and
provide the three-dimensional scene for display.
20. The system of claim 19 , wherein the computer system is to:
generate the homography matrix transform using an inverse compositional image alignment (ICIA).
21. The system of claim 19 , wherein to apply a homography matrix, the processor is to:
generate a homography matrix transform;
define photo-consistency between a root image and a warp window of a translated image as an implicit function of the homography matrix transform; and
improve photo-consistency of the warp window.
22. The system of claim 19 , wherein to generate a translated pixel from a root pixel, the computer system is to improve photo-consistency between the root pixel and the translated pixel.