System for combining multiple disparity maps
There is provided a method of processing image data, comprising the steps of: (a) providing a plurality of images of a scene; (b) generating a disparity map for each of at least two pairs of images from the plurality of images; (c) transforming each of the disparity maps into a common coordinate system; and (d) merging the transformed disparity maps to provide a single representation of the depth information of the scene.
1. A method of processing image data, said method comprising the steps of:
(a) providing a plurality of images of a scene;
(b) generating a disparity map for each of at least two pairs of images from said plurality of images;
(c) transforming each of said disparity maps into a common coordinate system; and
(d) merging said transformed disparity maps to provide a single representation of depth information of said scene.
2. A method as claimed in claim 1 , wherein said common coordinate system comprises a position of a viewpoint of one of said plurality of images.
3. A method as claimed in claim 2 , wherein said one of said plurality of images is a centralmost of said plurality of images.
4. A method as claimed in claim 1 , wherein said step (c) comprises applying a shear and scale transformation to each of said plurality of said disparity maps.
5. A method as claimed in claim 1 , wherein said step (c) produces a plurality of disparities at each location in said common coordinate system.
6. A method as claimed in claim 5 , wherein the largest of said disparities is selected at each said location in said common coordinate system.
7. A method as claimed in claim 1 , wherein a function is defined to record locations in said single representation for which no disparity is available.
8. A method as claimed in claim 7 , wherein said function is used to generate a disparity value for each said location in said single representation for which no disparity is available.
9. A method as claimed in claim 8 , wherein said disparity value is generated by interpolation between values of disparity at each side of a region for which no disparity is available.
10. A method as claimed in claim 1 , wherein each of said transformed disparity maps is expressed in terms of a parameterisation.
11. A method as claimed in claim 10 , wherein parameters of said parameterisation comprise a viewpoint parameter, a position, and a scale factor of said transformation.
12. A method as defined in claim 10 , wherein said parameterisation comprises parameters of said transformation between each of said disparity maps and said disparity map of said common coordinate system, which said parameters are determined using a search procedure.
13. A method as claimed in claim 12 , wherein said search procedure comprises minimising squared distances between disparity values at points of each of said disparity maps and said disparity map of said common coordinate system.
14. A method as claimed in claim 12 , wherein said search procedure comprises maximising a mutual information of each of said disparity maps and said disparity map of said common coordinate system.
15. A method as claimed in claim 14 , wherein said mutual information comprises a measure of a dependence between (i) a joint probability distribution, and (ii) a product distribution of disparities in each of said disparity maps and said disparity map of said common coordinate system.
16. A method as claimed in claim 12 , wherein said search procedure comprises a Golden Section search.
17. A method as claimed in claim 1 , where said single representation of said step (d) comprises an average of said transformed disparity maps.
18. A method as claimed in claim 17 , wherein said average is a weighted average.
19. A method as claimed in claim 18 , wherein said weighted average has weights derived from uncertainties in each of said disparity maps.
20. A method as claimed in claim 18 , wherein said weighted average has weights based upon an index of each of said disparity maps.
21. A method as claimed in claim 1 , applied to an output of a stereo-matching algorithm.
22. A method of generating a depth map of a scene, including a method of processing image data, said method comprising the steps of:
(a) providing a plurality of images of a scene;
(b) generating a disparity map for each of at least two pairs of images from said plurality of images;
(c) transforming each of said disparity maps into a common coordinate system; and
(d) merging said transformed disparity maps to provide a single representation of depth information of said scene.
23. A method of synthesising a novel view of a scene, comprising generating depth information for the novel view in accordance with a method of generating a depth map of a scene, including a method of processing image data, said method comprising the steps of:
(a) providing a plurality of images of a scene;
(b) generating a disparity map for each of at least two pairs of images from said plurality of images;
(c) transforming each of said disparity maps into a common coordinate system; and
(d) merging said transformed disparity maps to provide a single representation of depth information of said scene.
24. A method of evaluating a stereo-matching algorithm, said method comprising:
(i) providing a plurality of images of a scene;
(ii) generating a disparity map for each of at least two pairs of images from said plurality of images;
(ii) transforming each of said disparity maps into a common coordinate system, each of said transformed disparity maps being expressed in terms of a parameterisation;
(iv) merging said transformed disparity maps to provide a single representation of depth information of said scene; and
(v) optimising a consistency measure with respect to said parameterisation, a value of said consistency measure being indicative of a performance of said algorithm.
25. A method of determining parameter settings for a stereo-matching algorithm, said method comprising:
(i) providing a plurality of images of a scene;
(ii) generating a disparity map for each of at least two pairs of images from said plurality of images;
(iii) transforming each of said disparity maps into a common coordinate system, each of said transformed disparity maps being expressed in terms of parameterisation.
(iv) merging said transformed disparity maps to provide a single representation of depth information of said scene;
(v) optimising a consistency measure with respect to said parameterisation and;
(vi) varying parameters of said parameterisation and repeating said steps (i) to (v) until at least two of said disparity maps achieve a substantially fixed measure of consistency.
26. A program stored on a computer-readable medium for controlling a computer to perform a method of processing image data, said method comprising the steps of:
(a) providing a plurality of images of a scene;
(b) generating a disparity map for each of at least two pairs of images from said plurality of images;
(c) transforming each of said disparity maps into a common coordinate system; and
(d) merging said transformed disparity maps to provide a single representation of depth information of said scene.
27. An apparatus for processing image data comprising:
(a) means for providing a plurality of images of a scene;
(b) means for generating a disparity map for each of at least two pairs of images from said plurality of images;
(c) means for transforming each of said disparity maps into a common coordinate system; and
(d) means for merging said transformed disparity maps to provide a single representation of depth information of said scene.