Determination of disparity

1. A disparity determining method, comprising:

downsampling a first image and a second image, to obtain a first image with a first size and a second image with the first size;

performing a first stage of disparity computation on the first image with the first size and the second image with the first size, to obtain a first disparity map with the first size;

upsampling the first disparity map to obtain an upsampled disparity map with a second size; and

performing a second stage of disparity computation on a first image with the second size and a second image with the second size based on the upsampled disparity map with the second size to obtain a second disparity map with the second size;

wherein performing the second stage of disparity computation on the first image with the second size and the second image with the second size based on the upsampled disparity map with the second size to obtain the second disparity map with the second size comprises:

determining an image feature related to the first image and an image feature related to the second image on which the second stage of disparity computation is based, based on the first image with the second size and the second image with the second size;

determining a disparity search range corresponding to the second stage of disparity computation; and

determining the second disparity map with the second size based on the image feature related to the first image, the image feature related to the second image, and the disparity search range corresponding to the second stage of disparity computation;

wherein the image feature related to the second image carries a disparity offset determined based on the upsampled disparity map with the second size, or the disparity search range corresponding to the second stage of disparity computation is determined based on the upsampled disparity map with the second size.

2. The disparity determining method of claim 1 , further comprising:

in response to a number of times of disparity computation not reaching a threshold of times, cyclically performing an operation of performing a next stage of disparity computation on a first image with a size corresponding to the next stage of disparity computation and a second image with the size corresponding to the next stage of disparity computation based on an upsampled disparity map, with the size corresponding to the next stage of disparity computation, of a disparity map obtained by a current stage of disparity computation, until it is determined that the number of times of disparity computation reaches the threshold of times; and

taking a disparity map obtained by a last stage of disparity computation as a final disparity map between the first image and the second image, wherein a size corresponding to each stage of disparity computation is a size of an image on which this stage of disparity computation is based.

3. The disparity determining method of claim 1 , wherein a size of a disparity map obtained by a last stage of disparity computation in respective stages of disparity computation is consistent with original sizes of the first image and the second image.

4. The disparity determining method of claim 1 , wherein

when the image feature related to the second image carries the disparity offset determined based on the upsampled disparity map with the second size, determining the second disparity map with the second size based on the image feature related to the first image, the image feature related to the second image, and the disparity search range corresponding to the second stage of disparity computation, comprises:

performing disparity computation based on the image feature related to the first image, the image feature related to the second image, and the disparity search range corresponding to the second stage of disparity computation, to obtain a disparity adjustment map with the second size; and

combining the upsampled disparity map with the second size and the disparity adjustment map with the second size, to obtain the second disparity map with the second size.

5. The disparity determining method of claim 4 , wherein the disparity search range corresponding to the second stage of disparity computation is expressed as [−D 0 , D 0 ], and D 0 is a positive value not greater than M 0 (K−1),

wherein K represents an upsampling step between the first disparity map and the upsampled disparity map with the second size, and is a value not less than 1, and M 0 represents a maximum disparity error corresponding to the first stage of disparity computation, and is a value not less than 1.

6. The disparity determining method of claim 4 , wherein the image feature related to the first image or the image feature related to the second image comprises at least one of:

RGB components of each pixel and a gradient of each pixel in the image.

7. The disparity determining method of claim 4 , wherein performing disparity computation based on the image feature related to the first image, the image feature related to the second image, and the disparity search range corresponding to the second stage of disparity computation, to obtain the disparity adjustment map with the second size comprises:

constructing a cost volume based on the image feature related to the first image, the image feature related to the second image, and the disparity search range corresponding to the second stage of disparity computation; and

determining a disparity adjustment value between each pair of matching pixels in the first image with the second size and the second image with the second size based on the cost volume, to obtain the disparity adjustment map with the second size.

8. The disparity determining method of claim 7 , wherein

determining the disparity adjustment value between each pair of matching pixels in the first image with the second size and the second image with the second size based on the cost volume comprises:

performing cost aggregation on matching costs in the cost volume based on an aggregation window corresponding to the second stage of disparity computation;

determining a minimum cumulative cost of each pixel in the first image with the second size within the disparity search range corresponding to the second stage of disparity computation, based on a result of the cost aggregation; and

determining the disparity adjustment value between each pixel in the first image with the second size and each matching pixel in the second image with the second size, based on the minimum cumulative cost.

9. The disparity determining method of claim 4 , wherein

performing disparity computation based on the image feature related to the first image, the image feature related to the second image, and the disparity search range corresponding to the second stage of disparity computation, to obtain the disparity adjustment map with the second size comprises:

determining a disparity generation neural network, the disparity generation neural network being configured to determine a disparity adjustment value between each pair of matching pixels in an image pair based on an image feature of each image in the image pair and a corresponding disparity search range, to obtain a disparity adjustment map with a corresponding size; and

processing the image feature related to the first image, the image feature related to the second image, and the disparity search range corresponding to the second stage of disparity computation based on the disparity generation neural network, to obtain the disparity adjustment map with the second size.

10. The disparity determining method of claim 1 , wherein

when the disparity search range corresponding to the second stage of disparity computation is determined based on the upsampled disparity map with the second size, determining the second disparity map with the second size based on the image feature related to the first image, the image feature related to the second image, and the disparity search range corresponding to the second stage of disparity computation, comprises:

performing disparity computation based on the image feature related to the first image, the image feature related to the second image, and the disparity search range corresponding to the second stage of disparity computation, to obtain the second disparity map with the second size.

11. The disparity determining method of claim 10 , wherein determining the disparity search range corresponding to the second stage of disparity computation based on the upsampled disparity map with the second size comprises:

determining a disparity search adjustment range corresponding to the second stage of disparity computation, based on an upsampling step between the first disparity map and the upsampled disparity map with the second size, and a maximum disparity error corresponding to the first stage of disparity computation; and

adjusting the disparity search adjustment range corresponding to the second stage of disparity computation by using a disparity value in the upsampled disparity map with the second size as a reference, to obtain the disparity search range corresponding to the second stage of disparity computation and carrying a corresponding disparity offset.

12. The disparity determining method of claim 11 , wherein the disparity search adjustment range corresponding to the second stage of disparity computation is expressed as [−D 0 , D 0 ], and the disparity search range corresponding to the second stage of disparity computation and carrying the corresponding disparity offset is expressed as [−D 0 +disp1, D 0 +disp1],

wherein disp1 represents the disparity value in the upsampled disparity map with the second size, D 0 is a positive value not greater than M 0 (K−1), K represents the upsampling step and is a value not less than 1, and M 0 represents the maximum disparity error corresponding to the first stage of disparity computation and is a value not less than 1.

13. The disparity determining method of claim 10 , wherein the image feature related to the first image or the image feature related to the second image comprises at least one of:

RGB components of each pixel and a gradient of each pixel in the image.

14. The disparity determining method of claim 10 , wherein

performing disparity computation based on the image feature related to the first image, the image feature related to the second image, and the disparity search range corresponding to the second stage of disparity computation, to obtain the second disparity map with the second size comprises:

constructing a cost volume based on the image feature related to the first image, the image feature related to the second image, and the disparity search range corresponding to the second stage of disparity computation; and

determining a disparity between each pair of matching pixels in the first image with the second size and the second image with the second size based on the cost volume, to obtain the second disparity map with the second size.

15. The disparity determining method of claim 14 , wherein determining the disparity between each pair of matching pixels in the first image with the second size and the second image with the second size based on the cost volume comprises:

performing cost aggregation on matching costs in the cost volume based on an aggregation window corresponding to the second stage of disparity computation;

determining a minimum cumulative cost of each pixel in the first image with the second size within the disparity search range corresponding to the second stage of disparity computation, based on a result of the cost aggregation; and

determining the disparity between each pixel in the first image with the second size and each matching pixel in the second image with the second size, based on the minimum cumulative cost.

16. The disparity determining method of claim 10 , wherein performing disparity computation based on the image feature related to the first image, the image feature related to the second image, and the disparity search range corresponding to the second stage of disparity computation, to obtain the second disparity map with the second size comprises:

determining a disparity generation neural network, the disparity generation neural network being configured to determine a disparity between each pair of matching pixels in an image pair based on an image feature of each image in the image pair and a corresponding disparity search range, to obtain a disparity map with a corresponding size; and

processing the image feature related to the first image, the image feature related to the second image, and the disparity search range corresponding to the second stage of disparity computation based on the disparity generation neural network, to obtain the second disparity map with the second size.

17. The disparity determining method of claim 1 , wherein downsampling the first image and the second image to obtain the first image with the first size and the second image with the first size comprises:

determining a threshold of times of disparity computation;

determining a downsampling factor required for downsampling the first image and the second image, based on the threshold of times; and

downsampling the first image and the second image based on the downsampling factor, wherein the greater the threshold of times is, the greater the downsampling factor is.

18. The disparity determining method of claim 17 , wherein the threshold of times is N, the downsampling factor is (N−1)th power of K, K is an upsampling step or a downsampling step between images on which two adjacent stages of disparity computation are based, K is a value not less than 1, and N is a positive integer not less than 2.

19. An electronic device, comprising:

a processor; and

a memory that stores a program, the program comprising instructions that, when executed by the processor, cause the processor to:

downsample a first image and a second image, to obtain a first image with a first size and a second image with the first size;

perform a first stage of disparity computation on the first image with the first size and the second image with the first size, to obtain a first disparity map with the first size;

upsample the first disparity map to obtain an upsampled disparity map with a second size; and

perform a second stage of disparity computation on a first image with the second size and a second image with the second size based on the upsampled disparity map with the second size to obtain a second disparity map with the second size;

wherein the instructions, when executed by the processor, further cause the processor to:

determine an image feature related to the first image and an image feature related to the second image on which the second stage of disparity computation is based, based on the first image with the second size and the second image with the second size;

determine a disparity search range corresponding to the second stage of disparity computation; and

determine the second disparity map with the second size based on the image feature related to the first image, the image feature related to the second image, and the disparity search range corresponding to the second stage of disparity computation;

wherein the image feature related to the second image carries a disparity offset determined based on the upsampled disparity map with the second size, or the disparity search range corresponding to the second stage of disparity computation is determined based on the upsampled disparity map with the second size.

20. A non-transitory computer-readable storage medium that stores a program, the program comprising instructions that, when executed by a processor of an electronic device, cause the electronic device to:

downsample a first image and a second image, to obtain a first image with a first size and a second image with the first size;

perform a first stage of disparity computation on the first image with the first size and the second image with the first size, to obtain a first disparity map with the first size;

upsample the first disparity map to obtain an upsampled disparity map with a second size; and

perform a second stage of disparity computation on a first image with the second size and a second image with the second size based on the upsampled disparity map with the second size to obtain a second disparity map with the second size;

wherein the instructions, when executed by the processor, further cause the electronic device to:

determine an image feature related to the first image and an image feature related to the second image on which the second stage of disparity computation is based, based on the first image with the second size and the second image with the second size;

determine a disparity search range corresponding to the second stage of disparity; computation; and

determine the second disparity map with the second size based on the image feature related to the first image, the image feature related to the second image, and the disparity search range corresponding to the second stage of disparity computation;

wherein the image feature related to the second image carries a disparity offset determined based on the upsampled disparity map with the second size, or the disparity search range corresponding to the second stage of disparity computation is determined based on the upsampled disparity map with the second size.