Dynamic parallax correction for visual sensor fusion

1. An augmented reality (AR) vision system, comprising:

a display configured to present a surrounding environment to eyes of a user of the AR vision system;

a depth tracker configured to measure a focal depth of a focus point in the surrounding environment, wherein the eyes of the user are focused on the focus point;

two or more image sensors,

wherein each respective one of the image sensors is configured to receive illumination from the focus point and generate a respective image of a plurality of images,

wherein at least one of the image sensors is not optically aligned with at least one other of the two or more image sensors,

wherein at least one of the images has a parallax error with at least one other of the images,

wherein each of the images has a distortion error caused by lenses; and

a controller communicatively couplable to the two or more image sensors including one or more processors configured to execute program instructions, wherein the program instructions cause the one or more processors to:

generate an interpolated look-up-table (LUT) for the measured focal depth by interpolating pixel-by-pixel between a first precalculated LUT and a second precalculated LUT using an interpolation ratio,

wherein the first precalculated LUT is configured to correct the parallax error and the distortion error at a first depth that is nearer to the user than the measured focal depth of the focus point,

wherein the second precalculated LUT is configured to correct the parallax error and the distortion error at a second depth that is further from the user than the measured focal depth of the focus point,

wherein the measured focal depth of the focus point is between the first depth and the second depth,

apply the interpolated LUT for the measured focal depth to the images to correct the parallax error and the distortion error at the measured focal depth and generate a single image of the surrounding environment, and

display the single image on the display to the user of the AR vision system.

2. The vision system of claim 1 , wherein interpolating comprises linear interpolation.

3. The vision system of claim 1 , wherein interpolating comprises piecewise-linear interpolation.

4. The vision system of claim 1 , wherein interpolating comprises bicubic interpolation.

5. The vision system of claim 1 , wherein the image sensors comprise at least one of an infrared sensor, an optical sensor, or a UV sensor.

6. The vision system of claim 1 , wherein the depth tracker is an eye tracker.

7. The vision system of claim 6 , wherein the eye tracker is configured to track the angles of vergence of the eyes of the user.

8. The vision system of claim 7 , wherein the measured focal depth of the focus point is calculated from the angles of vergence.

9. The vision system of claim 1 , wherein applying the interpolated LUT function comprises mapping or linearly transforming each pixel of the images to a corresponding pixel in the single image.

10. The vision system of claim 9 , wherein mapping or linearly transforming comprises addition or multiplication.

11. The vision system of claim 1 , wherein the precalculated LUTs are calculated using trigonometric functions and exponential functions, and the interpolated LUT function is not calculated using trigonometric functions and exponential functions.

12. The vision system of claim 1 , wherein the depth tracker is a time-of-flight sensor (ToF sensor).

13. The vision system of claim 1 , wherein the focus point is at a center of the display or an infinity focus.

14. The vision system of claim 13 , wherein the depth tracker is an IR depth tracker configured to project a beam at the center of the display.

15. The vision system of claim 1 , wherein the interpolation ratio is defined by:

(

the

⁢

⁢

measured

⁢

⁢

focal

⁢

⁢

depth

⁢

⁢

of

⁢

⁢

the

⁢

⁢

focus

⁢

⁢

point

-

the

⁢

⁢

first

⁢

⁢

depth

)

(

the

⁢

⁢

second

⁢

⁢

depth

-

the

⁢

⁢

first

⁢

⁢

depth

)

.