Power switching time-of-flight sensor for estimating three-dimensional positions of a plurality of keypoints along a target object
Techniques are disclosed for operating a time-of-flight (TOF) sensor. The TOF may be operated in a low power mode by repeatedly performing a low power mode sequence, which may include performing a depth frame by emitting light pulses, detecting reflected light pulses, and computing a depth map based on the detected reflected light pulses. Performing the low power mode sequence may also include performing an amplitude frame at least one time by emitting a light pulse, detecting a reflected light pulse, and computing an amplitude map based on the detected reflected light pulse. In response to determining that an activation condition is satisfied, the TOF may be switched to operate in a high accuracy mode by repeatedly performing a high accuracy mode sequence, which may include performing the depth frame multiple times.
1 . A time-of-flight sensor comprising:
an illumination unit configured to transmit first light pulses to illuminate a target object in accordance with a light synchronization signal;
a sensor unit configured to receive second light pulses in a field of view in accordance with a shutter synchronization signal, wherein the time-of-flight sensor is configured to perform a depth measurement based on the second light pulses received by the sensor unit; and
a timing generator configured to generate the light synchronization signal to control the illumination unit and the shutter synchronization signal to control the sensor unit to cause the time-of-flight sensor to switch between operating in a low power mode and operating in a high accuracy mode based on an activation condition;
wherein operating in the low power mode comprises estimating three-dimensional (3D) positions of a plurality of keypoints along the target object based on a previously captured depth map and a currently captured amplitude map of the target object;
wherein operating in the high accuracy mode comprises computing the 3D positions of the plurality of keypoints based on a currently captured depth map of the target object.
2 . The time-of-flight sensor of claim 1 , wherein the illumination unit comprises a laser light source.
3 . The time-of-flight sensor of claim 1 , wherein the sensor unit comprises a light-sensitive pixel array.
4 . The time-of-flight sensor of claim 3 , further comprising:
an optical feedback device configured to direct a portion of the first light pulses transmitted by the illumination unit to a feedback region of the light-sensitive pixel array.
5 . The time-of-flight sensor of claim 1 , wherein operating in the low power mode includes repeatedly performing a low power mode sequence and operating in the high accuracy mode includes repeatedly performing a high accuracy mode sequence.
6 . The time-of-flight sensor of claim 5 , wherein performing the low power mode sequence includes:
performing a depth frame by the illumination unit emitting multiple ones of the first light pulses, the sensor unit detecting multiple ones of the second light pulses, and computing the currently captured depth map based on the multiple ones of the second light pulses; and
performing an amplitude frame at least one time, wherein performing the amplitude frame includes the illumination unit emitting one of the first light pulses, the sensor unit detecting one of the second light pulses, and computing the currently captured amplitude map based on the one of the second light pulses.
7 . The time-of-flight sensor of claim 6 , wherein performing the depth frame further includes computing three-dimensional (3D) positions of a plurality of keypoints along a target object based on the currently captured depth map.
8 . The time-of-flight sensor of claim 7 , wherein performing the amplitude frame further includes computing two-dimensional (2D) positions of the plurality of keypoints along the target object based on the currently captured amplitude map.
9 . The time-of-flight sensor of claim 8 , wherein performing the amplitude frame further includes estimating the 3D positions of the plurality of keypoints based on the 2D positions of the plurality of keypoints.
10 . The time-of-flight sensor of claim 9 , wherein performing the amplitude frame further includes estimating the 3D positions of the plurality of keypoints further based on the previously captured depth map.
11 . The time-of-flight sensor of claim 6 , wherein performing the high accuracy mode sequence includes performing the depth frame multiple times.
12 . The time-of-flight sensor of claim 1 , wherein the time-of-flight sensor switches from operating in the low power mode to operating in the high accuracy mode in response to determining that the activation condition is satisfied.
13 . The time-of-flight sensor of claim 7 , wherein the target object is a user's hand.
14 . The time-of-flight sensor of claim 13 , wherein the activation condition is determined to be satisfied when the user's hand is interacting in a Z dimension.
15 . The time-of-flight sensor of claim 1 , wherein the time-of-flight sensor is an element of a wearable system.
16 . The time-of-flight sensor of claim 15 , wherein the wearable system is an augmented reality (AR) wearable system.
17 . A wearable device comprising:
left and right eyepieces;
left and right projectors optically linked to the left and right eyepieces; and
a time-of-flight sensor comprising:
an illumination unit configured to transmit first light pulses to illuminate a target object in accordance with a light synchronization signal;
a sensor unit configured to receive second light pulses in a field of view in accordance with a shutter synchronization signal, wherein the time-of-flight sensor is configured to perform a depth measurement based on the second light pulses received by the sensor unit; and
a timing generator configured to generate the light synchronization signal to control the illumination unit and the shutter synchronization signal to control the sensor unit to cause the time-of-flight sensor to switch between operating in a low power mode and operating in a high accuracy mode based on an activation condition;
wherein operating in the low power mode comprises estimating three-dimensional (3D) positions of a plurality of keypoints along the target object based on a previously captured depth map and a currently captured amplitude map of the target object;
wherein operating in the high accuracy mode comprises computing the 3D positions of the plurality of keypoints based on a currently captured depth map of the target object.
18 . The wearable device of claim 17 , wherein the illumination unit comprises a laser light source.
19 . The wearable device of claim 17 , wherein the sensor unit comprises a light-sensitive pixel array.
20 . The wearable device of claim 17 , wherein the wearable device is an augmented reality (AR) wearable device.