Device and method for depth estimation using color images

1. A device for performing depth estimation on first and second images captured using one or more cameras having a color filter array, each image of the first and second images comprising multiple color channels, each color channel of the multiple color channels corresponding to a respective color channel of the color filter array, and the device being configured to perform depth estimation by estimating disparity from the color channels of the first and second images, wherein the first and second images are original images that are not processed by image signal processing (ISP).

2. A device of claim 1 , wherein each image of the first and second images comprises multiple coordinates, each coordinate of the multiple coordinates corresponding to a sample on a respective color channel.

3. A device of claim 1 , wherein the device is configured to identify overlapping portions of the first and second images and to perform the depth estimation based on that identification.

4. A device of claim 1 , wherein the first and second images are images captured from spatially offset locations.

5. A device of claim 1 , wherein the color channels comprise at least two color channels that correspond to different colors.

6. A device of claim 1 , wherein the color channels comprise at least two color channels that correspond to a same color.

7. A device of claim 6 , wherein the same color is green or yellow.

8. A device of claim 5 , the device being configured to perform the depth estimation by estimating disparity from the at least two color channels of the first and second images.

9. A device of claim 1 , the device being configured to estimate disparity from the color channels without having performed a non-linear operation on the color channels.

10. A device of claim 1 , wherein the device comprises an image signal processor and the device is configured to estimate disparity from the color channels without having processed the color channels by the image signal processor.

11. A device of claim 1 , the device being configured to estimate disparity from the color channels independently of any conversion of the color channels to an RGB color space.

12. A device of claim 1 , wherein the color channels are color channels formed by the color filter array.

13. A device of claim 1 , wherein the cameras are spaced-apart cameras comprised in the device and configured to capture images of overlapping fields of view.

14. A device of claim 1 , wherein performing the depth estimation comprises:

for each color channel of the first and second images, estimating a cost volume for differences between those color channels, and estimating a disparity based on the cost volume.

15. A method of training a machine learning algorithm to perform depth estimation on first and second images captured using one or more cameras having a color filter array, the method comprising:

configuring a first instance of the machine learning algorithm to receive multiple color channels, each color channel of the multiple color channels corresponding to a respective color channel of the color filter array and to perform depth estimation by estimating disparity from the color channels of the first and second images, wherein the first and second images are original images that are not processed by image signal processing (ISP);

comparing an output of the first instance of the machine learning algorithm with an expected output; and

forming a second instance of the machine learning algorithm based on a result of the comparison.

16. A method of claim 15 , comprising unstacking the multiple color channels.

17. A method of claim 15 , wherein the machine learning algorithm is an end-to-end trainable algorithm.

18. A method of claim 15 , comprising:

receiving color image training data;

estimating, by a programmed computer, color channels based on the training data; and

providing the estimated color channels as input to the first instance of the machine learning algorithm.

19. A method of claim 15 , wherein performing the depth estimation comprises:

for each color channel of the first and second images, estimating a cost volume for differences between those color channels, and estimating a disparity based on the cost volume.

20. A method of claim 15 , wherein performing the depth estimation comprises:

estimating a common cost volume for differences between all the color channels of the first and second images; and

estimating a disparity based on the common cost volume.