IP Library Granted Patent US 11,474,597
Granted Patent B2
US 11,474,597 · App. 17/087,053 · Granted Oct 18, 2022

Light field displays incorporating eye trackers and methods for generating views for a light field display using eye tracking information

Inventors: Kari Pulli (Fremont, CA); Gordon Wetzstein (Fremont, CA); Ryan Spicer (Fremont, CA); Andrew Jones (Fremont, CA); Tomi Maila (Fremont, CA); Zisimos Economou (Fremont, CA)
Assignee: GOOGLE LLC
G06F3/013G02B30/30H04N13/398
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,474,597
App. No.
17/087,053
Granted
Oct 18, 2022
Kind
B2
Abstract

A multiview autostereoscopic display includes a display area including an array of angular pixels, an eye tracker, and a processing system. Each angular pixel emits color that varies across a field of view of that angular pixel. The array of angular pixels displays different views in different viewing zones across the field of view of the display. The eye tracker detects the presence of the eyes of at least one viewer within specific viewing zones and produces eye tracking information including locations of the detected eyes within the specific viewing zones. The processing system renders a specific view for each detected eye based upon the location of the detected eye within the viewing zone with detected eyes, and generates control information for the array of angular pixels to cause the specific view for each detected eye to be displayed in the viewing zone in which that eye was detected.

Claims (67)

1. A multiview autostereoscopic display for providing a plurality of views to a viewer, comprising:

a display area comprising an array of angular pixels, where:

each angular pixel is configured for emitting light that varies across a field of view of that angular pixel; and

the array of angular pixels is configured for displaying different views in different viewing zones across a field of view of the display;

at least one eye tracker configured for (i) detecting the presence of eyes of the viewer within specific viewing zones, (ii) producing eye tracking information related to locations of the eyes so detected within the specific viewing zones, and (iii) detecting locations of the detected eyes within the specific viewing zones by performing a hierarchical eye tracking process comprising determining a location of the viewer, determining a location of a head of the viewer based on the location of the viewer so determined, and determining locations of the viewer's eyes based upon the location of the head of the viewer so determined; and

a processing system, where the processing system is configured for:

rendering a specific view for each one of the eyes so detected based upon the location of that eye within the viewing zone in which that eye was detected; and

generating control information for the array of angular pixels to cause the specific view for that eye to be displayed in the viewing zone in which that eye was detected.

2. The multiview autostereoscopic display of claim 1 , wherein the processing system is configured for rendering an updated view for each one of the eyes so detected based upon an updated location for that eye within the viewing zone in which that eye was previously detected.

3. The multiview autostereoscopic display of claim 2 , wherein the processing system is configured for rendering the updated view based upon a relative motion between a previous location for the detected eye and the updated location for the detected eye.

4. The multiview autostereoscopic display of claim 1 , wherein the processing system is configured for rendering at least one of reduced resolution views, reduced density views, and two-dimensional views for display in specific viewing zones based upon eye tracking information.

5. The multiview autostereoscopic display of claim 4 ,

wherein the eye tracking information includes confidence information indicating a level of uncertainty associated with a tracked eye location, and

wherein the processing system is further configured for adjusting the rendering according to the confidence information.

6. The multiview autostereoscopic display of claim 1 , wherein the processing system is configured for generating control information to control activation of emitters within the array of angular pixels based upon eye tracking information to prevent display of views in specific viewing zones.

7. The multiview autostereoscopic display of claim 1 , wherein the processing system is configured for rendering specific views for each one of the eyes so detected based upon the location of that eye within the viewing zone in which that eye was detected and in manner constrained by a determined viewer specific inter-pupil distance.

8. The multiview autostereoscopic display of claim 1 , wherein:

the at least one eye tracker comprises at least one gaze tracker configured for determining a gaze direction for each detected eye of the viewer; and

the processing system is configured for rendering a specific view for each detected eye based upon the location of the detected eye within the viewing zone in which the eye was detected and the gaze direction of the detected eye.

9. A multiview autostereoscopic display

for providing views viewable by a first viewer and a second viewer, comprising:

a display area comprising an array of angular pixels, where:

each angular pixel is configured for emitting light that varies across a field of view of that angular pixel; and

the array of angular pixels is configured for displaying different views in different viewing zones across a field of view of the display;

at least one eye tracker configured for (i) detecting the presence of eyes of the first viewer and eyes of the second viewer within specific viewing zones, (ii) producing eye tracking information related to locations of the eyes so detected within the specific viewing zones; and (iii) simultaneously tracking eyes of the first and second viewers; and

a processing system configured for:

rendering a specific view for each one of the eyes so detected based upon the location of that eye within the viewing zone in which that eye was detected;

generating control information for the array of angular pixels to cause the specific view for that eye to be displayed in the viewing zone in which that eye was detected;

rendering specific views based upon a first scene for each detected eye of the first viewer,

rendering additional views based upon a second scene for each detected eye of the second viewer, the second scene being at least partially different from the first scene, and

generating control information for the array of angular pixels such that, simultaneously, the specific views for each detected eye of the first viewer are displayed in viewing zones in which the eyes of the first viewer were detected, and the additional views for each detected eye of the second viewer are displayed in viewing zones in which the second viewer's eyes were detected.

10. The multiview autostereoscopic display of claim 9 , wherein the control information so generated by the processing system causes no view to be displayed outside the viewing zones in which at least one of the first and second viewers' eyes were detected.

11. A multiview autostereoscopic display

for providing views viewable by a first viewer and a second viewer, comprising:

a display area comprising an array of angular pixels, where:

each angular pixel is configured for emitting light that varies across a field of view of that angular pixel; and

the array of angular pixels is configured for displaying different views in different viewing zones across a field of view of the display;

an eye tracker that (i) includes a first eye tracker for tracking eyes of the first viewer and a second eye tracker for tracking eyes of the second viewer, and (ii) is configured for detecting the presence of eyes of the viewer within specific viewing zones, and producing eye tracking information related to locations of the eyes so detected within the specific viewing zones; and

a processing system configured for:

rendering a specific view for each one of the eyes so detected based upon the location of that eye within the viewing zone in which that eye was detected;

generating control information for the array of angular pixels to cause the specific view for that eye to be displayed in the viewing zone in which that eye was detected;

rendering specific views based upon a first scene for each detected eye of the first viewer,

rendering additional views based upon a second scene for each detected eye of the second viewer, the second scene being at least partially different from the first scene, and

generating control information for the array of angular pixels such that, simultaneously, the specific views for each detected eye of the first viewer are displayed in viewing zones in which the eyes of the first viewer were detected, and the additional views for each detected eye of the second viewer are displayed in the viewing zones in which the second viewer's eyes were detected.

12. The multiview autostereoscopic display of claim 11 , wherein the control information so generated by the processing system causes no view to be displayed outside the viewing zones in which at least one of the first and second viewers' eyes were detected.

13. A multiview autostereoscopic display for providing a plurality of views to a viewer, comprising:

a display area including an array of angular pixels;

an eye tracker; and

a processing system,

wherein each angular pixel is configured for emitting light that varies across a field of view of that angular pixel,

wherein the array of angular pixels is configured for displaying different views in different viewing zones across a field of view of the display,

wherein the eye tracker is configured for detecting the presence of eyes of the viewer within specific viewing zones, and producing eye tracking information related to locations of the eyes so detected within the specific viewing zones,

wherein the processing system is configured for rendering a specific view for each one of the eyes so detected within the viewing zone in which that eye was detected,

wherein the processing system is further configured for generating control information for the array of angular pixels to cause the specific view for each one of the eyes so detected to be displayed in the viewing zone in which that eye was detected, and

wherein the control information includes activation information for the array of angular pixels to control activation of emitters within the array of angular pixels based upon the eye tracking information to turn off a portion of the emitters contributing to views outside of the viewing zone in which that eye was detected.

14. The multiview autostereoscopic display of claim 13 , wherein the processing system is further configured for rendering at least one of reduced resolution views, reduced density views, and two-dimensional views for display in specific viewing zones based upon the eye tracking information.

15. The multiview autostereoscopic display of claim 13 , wherein the processing system is further configured for optimizing the control information for mitigating at least one of aliasing, ghosting, optical crosstalk between viewing zones, and pixel-level optical crosstalk.

16. A multiview autostereoscopic display for providing a plurality of views to a viewer, comprising:

a display area comprising an array of angular pixels, where:

each angular pixel is configured for emitting light that varies across a field of view of that angular pixel; and

the array of angular pixels is configured for displaying different views in different viewing zones across a field of view of the display;

at least one eye tracker configured for detecting the presence of eyes of the viewer within specific viewing zones and producing eye tracking information related to locations of the eyes so detected within the specific viewing zones; and

a processing system, where the processing system is configured for:

rendering a specific view for each one of the eyes so detected based upon the location of that eye within the viewing zone in which that eye was detected;

generating control information for the array of angular pixels to cause the specific view for that eye to be displayed in the viewing zone in which that eye was detected;

generating control information for a plurality of logical emitters;

the display area further including backplane circuitry configured for (i) receiving the control information for the plurality of logical emitters, and (ii) interpolating the control information for the plurality of logical emitters to control information for the array of angular pixels in the display area.

Assignments (5)
CORRECTIVE ASSIGNMENT TO CORRECT THE EXECUTION DATE FROM 3/3/2022 TO 5/4/2022 PREVIOUSLY RECORDED ON REEL 061448 FRAME 0903. ASSIGNOR(S) HEREBY CONFIRMS THE ASSIGNMENT OF ASSIGNORS INTEREST. Recorded Mar 23, 2023
From: RAXIUM INC.
To: GOOGLE LLC
Reel/Frame 063149/0640 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Aug 25, 2022
From: RAXIUM INC.
To: GOOGLE LLC
Reel/Frame 061448/0903 →
RELEASE OF SECURITY INTEREST Recorded Feb 3, 2022
From: TRIPLEPOINT CAPITAL LLC
To: RAXIUM, INC.
Reel/Frame 058880/0504 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jul 8, 2021
From: PULLI, KARI; WETZSTEIN, GORDON; SPICER, RYAN; JONES, ANDREW; MAILA, TOMI; ECONOMOU, ZISIMOS
To: RAXIUM, INC.
Reel/Frame 056796/0817 →
SECURITY INTEREST Recorded Feb 4, 2021
From: RAXIUM, INC.
To: TRIPLEPOINT CAPITAL LLC
Reel/Frame 055156/0199 →
Continuity (2)
Provisional Application 62929666 · Nov 1, 2019
Related Publication 20210132693A1 · May 6, 2021