Learning-based matching for active stereo systems
A first and second image of a scene are captured. Each of a plurality of pixels in the first image is associated with a disparity value. An image patch associated with each of the plurality of pixels of the first image and the second image is mapped into a binary vector. Thus, values of pixels in an image are mapped to a binary space using a function that preserves characteristics of values of the pixels. The difference between the binary vector associated with each of the plurality of pixels of the first image and its corresponding binary vector in the second image designated by the disparity value associated with each of the plurality of pixels of the first image is determined. Based on the determined difference between binary vectors, correspondence between the plurality of pixels of the first image and the second image is established.
1. A method, comprising:
capturing a first image and a second image of a scene, wherein the first image and the second image form a stereo pair and each comprise a plurality of pixels, and further wherein each of the plurality of pixels in the first image is associated with a disparity value;
mapping an image patch associated with each of the plurality of pixels of the first image and the second image into a binary vector;
determining a difference between the binary vector associated with each of the plurality of pixels of the first image and its corresponding binary vector in the second image designated by the disparity value associated with each of the plurality of pixels of the first image; and
determining correspondence between the plurality of pixels of the first image and the second image based on the determined difference between binary vectors.
2. The method of claim 1 , further comprising:
initializing each of the plurality of pixels in the first image with a disparity hypothesis.
3. The method of claim 2 , further comprising:
sampling a plurality of disparity hypotheses for each of the plurality of pixels in the first image; and
replacing the disparity value associated with each of the plurality of pixels of the first image based a lowest matching score of the plurality of disparity hypotheses.
4. The method of claim 3 , wherein sampling a plurality of disparity hypotheses comprises comparing a matching cost associated with the disparity value of each pixel relative to a matching cost associated with the disparity value of one or more of its neighboring pixels.
5. The method of claim 1 , wherein mapping the image patch associated with each of the plurality of pixels into a binary vector further comprises:
providing each pixel of the plurality of pixels of the first image and the second image as input signals to a binary decision tree, wherein the binary decision tree outputs a binary vector representation of its input signal.
6. The method of claim 5 , further comprising:
sparsely sampling the image patch associated with the plurality of pixels of the first image and the second image by traversing the binary decision tree.
7. The method of claim 1 , wherein determining the difference between the binary vector associated with each of the plurality of pixels of the first image and its corresponding binary vector in the second image further comprises:
computing a hamming distance between the binary vectors.
8. The method of claim 1 , further comprising:
generating a disparity map based at least in part on the determined correspondence between the plurality of pixels of the first image and the second image.
9. An electronic device, comprising:
a pair of cameras to capture a first image and a second image of a scene, wherein the first image and the second image form a stereo pair and each comprise a plurality of pixels, and further wherein each of the plurality of pixels in the first image is associated with a disparity value; and
a processor configured to:
map an image patch associated with each of the plurality of pixels of the first image and the second image into a binary vector;
determine a difference between the binary vector associated with each of the plurality of pixels of the first image and its corresponding binary vector in the second image designated by the disparity value associated with each of the plurality of pixels of the first image; and
determine correspondence between the plurality of pixels of the first image and the second image based on the determined difference between binary vectors.
10. The electronic device of claim 9 , wherein the processor is further configured to:
initialize each of the plurality of pixels in the first image with a disparity hypothesis.
11. The electronic device of claim 10 , wherein the processor is further configured to:
sample a plurality of disparity hypotheses for each of the plurality of pixels in the first image; and
replace the disparity value associated with each of the plurality of pixels of the first image based a lowest matching score of the plurality of disparity hypotheses.
12. The electronic device of claim 11 , wherein the processor is further configured to:
compare a matching cost associated with the disparity value of each pixel relative to a matching cost associated with the disparity value of one or more of its neighboring pixels.
13. The electronic device of claim 12 , wherein the processor is further configured to:
propagate disparity values associated with matching costs lower than disparity values of neighboring pixels to one or more of its neighboring pixels.
14. The electronic device of claim 9 , wherein the processor is further configured to:
provide each pixel of the plurality of pixels of the first image and the second image as input signals to a binary decision tree, wherein the binary decision tree outputs a binary vector representation of its input signal.
15. The electronic device of claim 14 , wherein the processor is further configured to:
sparsely sample the image patch associated with the plurality of pixels of the first image and the second image by traversing the binary decision tree.
16. The electronic device of claim 9 , wherein the processor is further configured to:
computing a hamming distance between the binary vectors.
17. The electronic device of claim 9 , wherein the processor is further configured to:
generate a disparity map based at least in part on the determined correspondence between the plurality of pixels of the first image and the second image.
18. A method, comprising:
capturing, at a pair of cameras, a first image and a second image of a scene, wherein the first image and the second image form a stereo pair and each comprise a plurality of pixels, and further wherein each of the plurality of pixels in the first image is associated with a disparity value;
providing each pixel of the plurality of pixels of the first image and the second image as input signals to a binary decision tree; and
mapping an image patch associated with each of the plurality of pixels of the first image and the second image into a binary vector.
19. The method of claim 18 , wherein mapping the image patch associated with each of the plurality of pixels of the first image and the second image into a binary vector reduces a dimensionality of image patch data while retaining discriminative data for establishing pixel correspondences.
20. The method of claim 18 , wherein each node of the binary decision tree includes a set of learned parameters defining a binary split of data reaching that node.