Correcting depth estimations derived from image data using acoustic information

1. A method comprising:

at a device including one or more processors and non-transitory memory:

obtaining image data including one or more images of a real-world environment;

obtaining a first depth estimation characterizing a distance between the device and the real-world environment in a direction relative to the device, wherein the first depth estimation is derived from the image data;

obtaining acoustic data representing audio in the real-world environment;

obtaining a second depth estimation characterizing the distance between the device and the real-world environment in the direction relative to the device, wherein the second depth estimation is derived from the acoustic data, wherein the first depth estimation characterizes a distance between the device and a first surface in the real-world environment and the second depth estimation characterizes a distance between the device and a second surface in the real-world environment different than the first surface; and

in response to determining that a difference between the first depth estimation and the second depth estimation exceeds a threshold value, determining a confirmed depth estimation characterizing the distance between the device and the real-world environment in the direction relative to the device as the second depth estimation.

2. The method of claim 1 , further comprising:

in response to determining that the difference between the first depth estimation and the second depth estimation is less than the threshold value, determining the confirmed depth estimation as the first depth estimation.

3. The method of claim 1 , further comprising generating an acoustic wave using an audio transceiver, wherein the acoustic data includes a reflection of the acoustic wave.

4. The method of claim 1 , wherein the device corresponds to a head-mounted display.

5. The method of claim 1 , wherein the device corresponds to a robotic agent.

6. The method of claim 1 , wherein the second surface is a transparent surface or a mirrored surface.

7. The method of claim 1 , wherein the acoustic data represents audible audio in the real-world environment.

8. The method of claim 7 , wherein the audible audio is generated by an audio generating source in the real-world environment separate from the device.

9. The method of claim 7 , wherein the audible audio includes speech from a person in the real-world environment.

10. The method of claim 1 , wherein obtaining the first depth estimation includes:

determining, in each image of at least some of the one or more images in the image data, an anchor point indicating a location of the first surface in the respective image; and

computing the first depth estimation to minimize disparity between each anchor point in each image.

11. A device comprising:

non-transitory memory; and

one or more processors to:

obtain image data including one or more images of a real-world environment;

obtain a first depth estimation characterizing a distance between the device and the real-world environment in a direction relative to the device, wherein the first depth estimation is derived from the image data;

obtain acoustic data representing audio in the real-world environment;

obtain a second depth estimation characterizing the distance between the device and the real-world environment in the direction relative to the device, wherein the second depth estimation is derived from the acoustic data, wherein the first depth estimation characterizes a distance between the device and a first surface in the real-world environment and the second depth estimation characterizes a distance between the device and a second surface in the real-world environment different than the first surface; and

in response to determining that a difference between the first depth estimation and the second depth estimation exceeds a threshold value, determine a confirmed depth estimation characterizing the distance between the device and the real-world environment in the direction relative to the device as the second depth estimation.

12. The device of claim 11 , wherein the one or more processors are further to:

in response to determining that the difference between the first depth estimation and the second depth estimation is less than the threshold value, determine the confirmed depth estimation as the first depth estimation.

13. The device of claim 11 , wherein the one or more processors are further to generate an acoustic wave using an audio transceiver, wherein the acoustic data includes a reflection of the acoustic wave.

14. The device of claim 11 , wherein the second surface is a transparent surface or a mirrored surface.

15. The device of claim 11 , wherein the acoustic data represents audible audio in the real-world environment.

16. The device of claim 15 , wherein the audible audio is generated by an audio generating source in the real-world environment separate from the device.

17. The device of claim 15 , wherein the audible audio includes speech from a person in the real-world environment.

18. The device of claim 11 , wherein the one or more processors are to obtain the first depth estimation by:

determining, in each image of at least some of the one or more images in the image data, an anchor point indicating a location of the first surface in the respective image; and

computing the first depth estimation to minimize disparity between each anchor point in each image.

19. A non-transitory memory storing one or more programs which, when executed by one or more processors of a device, cause the device to:

obtain image data including one or more images of a real-world environment;

obtain a first depth estimation characterizing a distance between the device and the real-world environment in a direction relative to the device, wherein the first depth estimation is derived from the image data;

obtain acoustic data representing audio in the real-world environment;

obtain a second depth estimation characterizing the distance between the device and the real-world environment in the direction relative to the device, wherein the second depth estimation is derived from the acoustic data, wherein the first depth estimation characterizes a distance between the device and a first surface in the real-world environment and the second depth estimation characterizes a distance between the device and a second surface in the real-world environment different than the first surface; and

in response to determining that a difference between the first depth estimation and the second depth estimation exceeds a threshold value, determine a confirmed depth estimation characterizing the distance between the device and the real-world environment in the direction relative to the device as the second depth estimation.

20. The non-transitory memory of claim 19 , wherein the one or more programs, when executed by the one or more processors of the device, further cause the device to:

in response to determining that the difference between the first depth estimation and the second depth estimation is less than the threshold value, determine the confirmed depth estimation as the first depth estimation.

21. The non-transitory memory of claim 19 , wherein the acoustic data represents audible audio in the real-world environment.

22. The non-transitory memory of claim 21 , wherein the audible audio is generated by an audio generating source in the real-world environment separate from the device.

23. The non-transitory memory of claim 21 , wherein the audible audio includes speech from a person in the real-world environment.

24. The non-transitory memory of claim 19 , wherein obtaining the first depth estimation includes:

determining, in each image of at least some of the one or more images in the image data, an anchor point indicating a location of the first surface in the respective image; and

computing the first depth estimation to reduce disparity between each anchor point in each image.

25. The non-transitory memory of claim 19 , wherein the second surface is a transparent surface or a mirrored surface.