Asymmetric normalized correlation layer for deep neural network feature matching

1. A method comprising:

obtaining a first image of a scene using a first image sensor of an electronic device and a second image of the scene using a second image sensor of the electronic device;

generating a first feature map from the first image and a second feature map from the second image;

generating a third feature map based on the first feature map and the second feature map, using an asymmetric search window; and

generating a depth map by restoring spatial resolution to the third feature map.

2. The method of claim 1 , wherein generating the first feature map and the second feature map comprises:

modifying at least one of the first image and the second image to generate a calibrated image pair; and

using the calibrated image pair to generate the first feature map and the second feature map.

3. The method of claim 1 , further comprising:

identifying high-frequency components and a low-frequency component of the first feature map, wherein the high-frequency components are used to restore the spatial resolution to the third feature map.

4. The method of claim 1 , wherein the asymmetric search window includes at least two different distances for at least two different directions in the asymmetric search window.

5. The method of claim 1 , wherein the first feature map and the second feature map are generated in parallel using different feature extractors in a neural network.

6. The method of claim 1 , wherein generating the third feature map comprises:

applying a random binary mask across the first feature map and the second feature map to generate a masked first feature map and a masked second feature map; and

identifying the third feature map by calculating channel-normalized cross-correlations between the first masked feature map and shifted versions of the second masked feature map, wherein the second masked feature map is shifted multiple times based on a size of the asymmetric search window.

7. The method of claim 1 , further comprising:

obtaining a focus point within the first image; and

using the depth map to generate a Bokeh effect by blurring portions of the first image that correspond to depths that are different than a depth associated with the focus point.

8. An electronic device comprising:

a first image sensor;

a second image sensor; and

at least one processor operatively coupled to the first and second image sensors and configured to:

obtain a first image of a scene using the first image sensor and a second image of the scene using the second image sensor;

generate a first feature map from the first image and a second feature map from the second image;

generate a third feature map based on the first feature map and the second feature map, using an asymmetric search window; and

generate a depth map by restoring spatial resolution to the third feature map.

9. The electronic device of claim 8 , wherein, to generate the first feature map and the second feature map, the at least one processor is configured to:

modify at least one of the first image and the second image to generate a calibrated image pair; and

use the calibrated image pair to generate the first feature map and the second feature map.

10. The electronic device of claim 8 , wherein:

the at least one processor is further configured to identify high-frequency components and a low-frequency component of the first feature map; and

the at least one processor is configured to restore the spatial resolution to the third feature map using the high-frequency components.

11. The electronic device of claim 8 , wherein the asymmetric search window includes at least two different distances for at least two different directions in the asymmetric search window.

12. The electronic device of claim 8 , wherein the at least one processor is configured to generate the first feature map and the second feature map in parallel using different feature extractors in a neural network.

13. The electronic device of claim 8 , wherein, to generate the depth map, the at least one processor is configured to:

apply a random binary mask across the first feature map and the second feature map to generate a masked first feature map and a masked second feature map; and

identify the third feature map by calculating channel-normalized cross-correlations between the first masked feature map and shifted versions of the second masked feature map; and

wherein the at least one processor is configured to shift the second feature map multiple times based on a size of the asymmetric search window.

14. The electronic device of claim 8 , wherein the at least one processor is further configured to:

obtain a focus point within the first image; and

use the depth map to generate a Bokeh effect by blurring portions of the first image that correspond to depths that are different than a depth associated with the focus point.

15. A non-transitory machine-readable medium containing instructions that when executed cause at least one processor of an electronic device to:

obtain a first image of a scene using a first image sensor of the electronic device and a second image of the scene using a second image sensor of the electronic device;

generate a first feature map from the first image and a second feature map from the second image;

generate a third feature map based on the first feature map and the second feature map, using an asymmetric search window; and

generate a depth map by restoring spatial resolution to the third feature map.

16. The non-transitory machine-readable medium of claim 15 , further containing instructions that when executed cause the at least one processor to:

modify at least one of the first image and the second image to generate a calibrated image pair; and

use the calibrated image pair to generate the first feature map and the second feature map.

17. The non-transitory machine-readable medium of claim 15 , further containing instructions that when executed cause the at least one processor to:

identify high-frequency components and a low-frequency component of the first feature map; and

use the high-frequency components to restore the spatial resolution to the third feature map.

18. The non-transitory machine-readable medium of claim 15 , wherein the asymmetric search window includes at least two different distances for at least two different directions in the asymmetric search window.

19. The non-transitory machine-readable medium of claim 15 , wherein the instructions that when executed cause the at least one processor to generate the third feature map comprise:

instructions that when executed cause the at least one processor to:

apply a random binary mask across the first feature map and the second feature map to generate a masked first feature map and a masked second feature map; and

identify the third feature map by calculating channel-normalized cross-correlations between the first masked feature map and shifted versions of the second masked feature map, wherein the second masked feature map is shifted multiple times based on a size of the asymmetric search window.

20. The non-transitory machine-readable medium of claim 15 , further containing instructions that when executed cause the at least one processor to:

obtain a focus point within the first image; and

use the depth map to generate a Bokeh effect by blurring portions of the first image that correspond to depths that are different than a depth associated with the focus point.