Systems and techniques for determining range and range-rate using time-of-flight sensors

1. An apparatus comprising:

at least one memory comprising instructions; and

at least one processor configured to execute the instructions and cause the at least one processor to:

determine a first depth map that is based on a first image frame and a second image frame, wherein the first image frame and the second image frame correspond to a first frame sequence received from a time-of-flight sensor;

calculate, based on the first depth map, a first set of three-dimensional optical flow data between the first image frame and the second image frame;

perform three-dimensional warping of at least one image frame from the first frame sequence using the first set of three-dimensional optical flow data to yield a first realigned image frame; and

determine a second depth map that is based on the first realigned image frame.

2. The apparatus of claim 1 , wherein the at least one processor is further configured to:

perform three-dimensional warping of at least one image frame from a second frame sequence using the first set of three-dimensional optical flow data to yield a second realigned image frame, wherein the second frame sequence is subsequent to the first frame sequence;

determine a third depth map that is based on the second realigned image frame;

calculate, based on the third depth map, a second set of three-dimensional optical flow data between the second image frame and a third image frame corresponding to the second frame sequence;

perform three-dimensional warping of at least one image frame from the second frame sequence using the second set of three-dimensional optical flow data to yield a third realigned image frame; and

determine a fourth depth map that is based on the third realigned image frame.

3. The apparatus of claim 1 , wherein the first image frame and the second image frame correspond to non-adjacent image frames in the first frame sequence.

4. The apparatus of claim 1 , wherein the first image frame is associated with a first set of one or more sensor parameters and the second image frame is associated with a second set of the one or more sensor parameters.

5. The apparatus of claim 4 , wherein the one or more sensor parameters include at least one of an illumination state associated with the time-of-flight sensor, a phase delay associated with the time-of-flight sensor, and an integration time associated with the time-of-flight sensor.

6. The apparatus of claim 1 , wherein the first set of three-dimensional optical flow data includes a transverse velocity and a z-axis velocity.

7. The apparatus of claim 1 , wherein the first realigned image frame corresponds to an intermediate image frame between the first image frame and the second image frame.

8. The apparatus of claim 1 , wherein the first realigned image frame corresponds to a successive image frame that is subsequent to the second image frame.

9. The apparatus of claim 1 , wherein the at least one processor is further configured to:

determine a distance between an object and an autonomous vehicle based on the second depth map, wherein the time-of-flight sensor is coupled to the autonomous vehicle.

10. A method comprising:

determining a first depth map that is based on a first image frame and a second image frame, wherein the first image frame and the second image frame correspond to a first frame sequence received from a time-of-flight sensor;

calculating, based on the first depth map, a first set of three-dimensional optical flow data between the first image frame and the second image frame;

performing three-dimensional warping of at least one image frame from the first frame sequence using the first set of three-dimensional optical flow data to yield a first realigned image frame; and

determining a second depth map that is based on the first realigned image frame.

11. The method of claim 10 , further comprising:

performing three-dimensional warping of at least one image frame from a second frame sequence using the first set of three-dimensional optical flow data to yield a second realigned image frame, wherein the second frame sequence is subsequent to the first frame sequence;

determining a third depth map that is based on the second realigned image frame;

calculating, based on the third depth map, a second set of three-dimensional optical flow data between the second image frame and a third image frame corresponding to the second frame sequence;

performing three-dimensional warping of at least one image frame from the second frame sequence using the second set of three-dimensional optical flow data to yield a third realigned image frame; and

determining a fourth depth map that is based on the third realigned image frame.

12. The method of claim 10 , wherein the first image frame and the second image frame correspond to non-adjacent image frames in the first frame sequence.

13. The method of claim 10 , wherein the first image frame is associated with a first set of one or more sensor parameters and the second image frame is associated with a second set of the one or more sensor parameters.

14. The method of claim 13 , wherein the one or more sensor parameters include at least one of an illumination state associated with the time-of-flight sensor, a phase delay associated with the time-of-flight sensor, and an integration time associated with the time-of-flight sensor.

15. The method of claim 10 , wherein the first set of three-dimensional optical flow data includes a transverse velocity and a z-axis velocity.

16. The method of claim 10 , wherein the first realigned image frame corresponds to an intermediate image frame between the first image frame and the second image frame.

17. The method of claim 10 , wherein the first realigned image frame corresponds to a successive image frame that is subsequent to the second image frame.

18. The method of claim 10 , further comprising:

determining a distance between an object and an autonomous vehicle based on the second depth map, wherein the time-of-flight sensor is coupled to the autonomous vehicle.

19. A non-transitory computer-readable media comprising instructions stored thereon which, when executed are configured to cause a computer or processor to:

determine a first depth map that is based on a first image frame and a second image frame, wherein the first image frame and the second image frame correspond to a first frame sequence received from a time-of-flight sensor;

calculate, based on the first depth map, a first set of three-dimensional optical flow data between the first image frame and the second image frame;

perform three-dimensional warping of at least one image frame from the first frame sequence using the first set of three-dimensional optical flow data to yield a first realigned image frame; and

determine a second depth map that is based on the first realigned image frame.

20. The non-transitory computer-readable media of claim 19 , comprising further instructions configured to cause the computer or the processor to:

receive a third image frame from the time-of-flight sensor, wherein the third image frame corresponds to a second frame sequence received from the time-of-flight sensor;

perform three-dimensional warping of at least one image frame from the second frame sequence using the first set of three-dimensional optical flow data to yield a second realigned image frame;

determine a third depth map that is based on the second realigned image frame;

calculate, based on the third depth map, a second set of three-dimensional optical flow data between the second image frame and the third image frame;

perform three-dimensional warping of at least one image frame from the second frame sequence using the second set of three-dimensional optical flow data to yield a third realigned image frame; and

determine a fourth depth map that is based on the third realigned image frame.