Latency reduction in an eye tracker of an autostereoscopic display device
The invention relates to a method for tracking a facial characteristic of a viewer of an autostereoscopic display device, the method comprising obtaining first sequential data on the position of the facial characteristic relative to the autostereoscopic display device; and comprising second sequential data on the speed and/or acceleration of the facial characteristic relative to the autostereoscopic display device. The first sequential data are obtained at a first latency and the second sequential data are obtained at a second latency that is lower than the first latency. As a final step, sensor fusion is performed to combine the first sequential data and the second sequential data to generate final sequential data on the position of the facial characteristic relative to the autostereoscopic display device. In this way, the final sequential data are available at a final latency that is lower than the first latency. A viewer of an autostereoscopic display device can so be presented with an improved stereoscopic image and/or have an improved virtual reality experience, when he moves and rotates his head relative to the autostereoscopic display device.
1 . A Method for tracking a facial characteristic of a viewer of an autostereoscopic display device, the method comprising:
obtaining first sequential data on a position of the facial characteristic relative to the autostereoscopic display device, wherein the first sequential data are obtained at a first latency;
obtaining second sequential data on a speed and/or acceleration of the facial characteristic relative to the autostereoscopic display device using a wearable sensor worn by the viewer, wherein the wearable sensor is configured to measure a speed and/or acceleration of the wearable sensor, and wherein the speed and/or acceleration of the facial characteristic is based on a relative positioning of the wearable sensor and the facial characteristic of the viewer, wherein the second sequential data are obtained at a second latency, the second latency being lower than the first latency; and
performing sensor fusion to combine the first sequential data and the second sequential data to generate final sequential data on the position of the facial characteristic relative to the autostereoscopic display device,
wherein the final sequential data are available at a final latency that is lower than the first latency.
2 . The Method according to claim 1 , wherein the first sequential data are obtained by using one or more cameras configured to record a facial characteristic of the viewer in an observation field of the one or more cameras.
3 . The Method according to claim 1 , wherein the second sequential data are obtained by using a wearable device that is worn on the viewer's head, the wearable device including the wearable sensor, wherein the wearable device is configured to produce data on the speed and/or acceleration of the wearable device.
4 . The Method according to claim 1 , wherein
the second sequential data are obtained by taking into account the speed and/or acceleration of the autostereoscopic display device; and
the autostereoscopic display device is configured to produce third sequential data on the speed and/or acceleration of the autostereoscopic display device.
5 . The Method according to claim 3 , wherein the wearable device comprises at least one of earbuds or headphones or glasses or headbands or tiaras or rubber bands or headgear.
6 . The Method according to claim 1 , further comprising:
obtaining third sequential data on the speed and/or acceleration of the facial characteristic relative to the autostereoscopic display device, wherein the third sequential data are obtained by using a remote device that is not worn on the viewer's head; and
performing sensor fusion to combine the first sequential data, the second sequential data, and the third sequential data to generate final sequential data on the position of the facial characteristic relative to the autostereoscopic display device.
7 . The Method according to claim 6 , wherein the remote device relies on the use of the Doppler effect, triangulation or time of flight.
8 . The Method according to claim 1 , wherein the facial characteristic comprises at least one of an eye or nose or ear or head or mouth of the viewer.
9 . The Method according to claim 1 , wherein the autostereoscopic display device comprises at least one of a television or monitor or cinema display system or mobile phone or tablet or laptop or game console.
10 . The Method according to claim 1 , wherein performing the sensor fusion comprises using a Kalman filter.
11 . A Tracking system for tracking a facial characteristic of a viewer of an autostereoscopic display device, the tracking system comprising:
a tracking camera system configured to obtain first sequential data on a position of the facial characteristic relative to the autostereoscopic display device, wherein the first sequential data are obtained at a first latency;
a secondary tracking system configured to obtain second sequential data on a speed and/or acceleration of the facial characteristic relative to the autostereoscopic display device using a wearable sensor worn by the viewer, wherein the wearable sensor is configured to measure a speed and/or acceleration of the wearable sensor, and wherein the speed and/or acceleration of the facial characteristic is based on a relative positioning of the wearable sensor and the facial characteristic of the viewer, wherein the second sequential data are obtained at a second latency, the second latency being lower than the first latency; and
a sensor fusion module that is configured to combine the first sequential data and the second sequential data to generate final sequential data on the position of the facial characteristic relative to the autostereoscopic display device.
12 . The Tracking system according to claim 11 , wherein the secondary tracking system comprises a wearable device that includes the wearable sensor and comprises one or more accelerometers.
13 . The Tracking system according to claim 11 , wherein the secondary tracking system comprises a remote device configured to measure speed and/or acceleration of the facial characteristic relative to the remote device by irradiating the facial characteristic with electromagnetic radiation or sound, and using one or more of the Doppler effect, time of flight or triangulation.
14 . An Assembly comprising:
an autostereoscopic display device; and
the tracking system according to claim 11 .
15 . The Assembly according to claim 14 , wherein the secondary tracking system comprises a remote device configured to measure speed and/or acceleration of the facial characteristic relative to the remote device by irradiating the facial characteristic with electromagnetic radiation or sound, wherein the remote device is integrated in the autostereoscopic display device.
16 . The Assembly according to claim 14 , wherein the autostereoscopic display device is configured to produce third sequential data on the speed and/or acceleration of the autostereoscopic display device.
17 . The Tracking system according to claim 12 , wherein the wearable device comprises a gyroscope and/or a magnetometer.
18 . A Tracking system for tracking a facial characteristic of a viewer of an autostereoscopic display device, the Tracking system comprising:
a tracking camera system configured to capture images of the facial characteristic of the viewer to obtain first sequential data on a position of the facial characteristic relative to the autostereoscopic display device, wherein the first sequential data are obtained at a first latency;
a remote device that is not worn on the viewer, the remote device configured to obtain second sequential data on a speed and/or acceleration of the facial characteristic relative to the remote device by providing an electromagnetic signal or sound signal toward the facial characteristic and measuring a return signal using one or more of the Doppler effect, time of flight, or triangulation, wherein the second sequential data are obtained at a second latency, the second latency being lower than the first latency; and
a sensor fusion module configured to combine the first sequential data and the second sequential data to generate final sequential data on the position of the facial characteristic relative to the autostereoscopic display device.
19 . The Tracking system according to claim 18 , comprising:
a wearable sensor worn by the viewer, wherein:
the wearable sensor is configured to obtain third sequential data, and
the sensor fusion module is configured to combine the first sequential data, the second sequential data, and the third sequential data to generate final sequential data on the position of the facial characteristic relative to the autostereoscopic display device.
20 . The Tracking system according to claim 18 , wherein the remote device and the autostereoscopic display device are substantially stationary with respect to Earth.