IP Library › Granted Patent US 9,674,510
Granted Patent B2
US 9,674,510 · App. 13/683,122 · Granted Jun 6, 2017

Pulsed projection system for 3D video

Inventors: Erez L. Aiden (Cambridge, MA); Alistair K. Chan (Bainbridge Island, WA); Philip A. Eckhoff (Bellevue, WA); William Gates (Medina, WA); Peter L. Hagelstein (Carlisle, MA); Roderick A. Hyde (Redmond, WA); Jordin T. Kare (Seattle, WA); Robert Langer (Newton, MA); Eric C. Leuthardt (St Louis, MO); Nathan P. Myhrvold (Medina, WA); Michael Schnall-Levin (Cambridge, MA); Lowell L. Wood, Jr. (Bellevue, WA)
Assignee: Elwha LLC
H04N13/0418H04N13/0402H04N13/0404H04N13/047H04N13/0468
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 9,674,510
App. No.
13/683,122
Granted
Jun 6, 2017
Kind
B2
Abstract

A 3D-display system alternates between directing light representing a left-eye view of a 3D image to a viewer's left eye and directing light representing a right-eye view of a 3D image to a viewer's right eye. To direct the light towards the viewer's eyes, the system receives eye-location data relative to a display device from an eye-tracking system and uses light deflectors, such as acousto-optic, electro-optic, and passive optical deflectors, to aim the light in a particular direction.

Claims (94)

1. A method for displaying autostereoscopic 3D images, the method comprising:

providing a display device;

receiving eye-location data indicative of a first and second set of eye-locations relative to the display device, wherein the first set of eye-locations comprises a first left-eye location and a first right-eye location associated with a first viewer and the second set of eye-locations comprises a second left-eye location and a second right-eye location associated with a second viewer, the received eye-location data further indicative of movement of at least one of the first or second sets of eye-locations relative to the display device;

sequentially outputting, from the display device, light rays representing a left-eye view of the 3D images and light rays representing a right-eye view of the 3D images for the first and second viewers;

based at least in part on the received eye-location data, using an active solid-state optical deflector (i) to deflect the output light rays representing the left-eye view towards the first left-eye location and the second left-eye location and (ii) to deflect the output light rays representing the right-eye view towards the first right-eye location and the second right-eye location;

estimating a refresh time corresponding to when the display device will refresh the displayed 3D images;

estimating future eye-locations based at least in part on the indicated movement characteristics of at least one of the first or second set of eye-locations, wherein the estimated future eye-locations correspond with an expected eye-location of at least one of the first or second eye-locations at the estimated refresh time; and

deflecting the output light rays towards the estimated future eye-locations at the estimated refresh time;

wherein the light rays representing the first viewer right-eye and left-eye views are deflected towards the first set of eye-locations such that each light ray is directed to a focused spot smaller than an eye of the first viewer to prevent the first viewer right-eye and left-eye view from being viewed by an eye of the second viewer;

wherein the light rays representing the second viewer right-eye and left-eye views are deflected towards the second set of eye-locations such that each light ray is directed to a second focused spot centered on an eye of the second viewer; and

wherein the second focused spot is larger than an eye of the second viewer by an amount configured to facilitate a target amount of movement of the second viewer.

2. The method of claim 1 , wherein the display device comprises a lenticular display-device.

3. The method of claim 1 , further comprising:

receiving additional eye-location data indicative of one or more updated first or second sets of eye-locations; and

deflecting the output light rays based at least in part on the additional eye-location data.

4. The method of claim 3 , further comprising:

detecting motion in an area in front of the display device; and

in response to the detected motion, instructing the eye-tracking system to generate the additional eye-location data.

5. The method of claim 3 , wherein the eye-location data is received from an eye-tracking system, the method further comprising:

occasionally instructing the eye-tracking system to generate the additional eye-location data; and

using the estimated refresh time as a basis for determining when to instruct the eye-tracking system to generate the additional eye-location data.

6. The method of claim 3 , wherein deflecting the output light rays based at least in part on the additional eye-location data comprises:

determining whether a difference between the one or more updated first or second sets of eye-locations and a previous first or second set of eye-locations is larger than a non-zero threshold difference in the first or second eye-locations;

in response to determining that the difference between the one or more updated first or second sets of eye-locations and the previous first or second set of eye-locations is larger than the non-zero threshold difference in the first or second eye-location, deflecting the output light rays based at least in part on the additional eye-location data.

7. The method of claim 1 , wherein the display device comprises a mobile display-device.

8. The method of claim 1 , wherein the active solid-state deflectors comprise acousto-optical deflectors, and wherein the acousto-optical deflectors use surface acoustic waves to deflect the light rays.

9. The method of claim 1 , wherein using the active solid-state deflectors to deflect the output light rays comprises:

using a first type of optical deflector to coarsely adjust a direction of the light rays; and

using a second type of optical deflector to finely adjust the direction of the light rays.

10. The method of claim 9 , wherein the first type of optical deflector is an electro-optical deflector, and wherein the second type of optical deflector is an acousto-optical deflector.

11. The method of claim 1 , wherein using the active solid-state deflectors to deflect the output light rays comprises:

using a first type of optical deflector to deflect the light rays in a direction associated with the first set of eye-locations; and

using a second type of optical deflector to deflect the light rays to each of the first left eye-location and the first right eye-location.

12. The method of claim 11 , wherein the first type of optical deflector is an electro-optical deflector, and wherein the second type of optical deflector is an acousto-optical deflector.

13. The method of claim 1 , wherein sequentially outputting the light rays representing the left-eye view of the 3D images and the light rays representing the right-eye view of the 3D images comprises alternating between (i) outputting the light rays representing the left-eye view and (ii) outputting the light rays representing the right-eye view.

14. An autostereoscopic 3D-display device, comprising:

a display screen configured to sequentially display right-eye and left-eye views of 3D images to a first and second viewer;

an eye-tracking system configured to determine a first and second viewer set of eye-locations relative to the display screen, wherein the first and second viewer set of eye-locations comprises a first and second viewer left-eye location and a first and second viewer right-eye location; and

an active solid-state optical-deflection system capable of directing light rays representing the displayed first and second viewer right-eye and first and second viewer left-eye views, wherein the active solid-state deflection system is configured to direct the light rays towards the determined first and second viewer set of eye-locations;

wherein the active solid-state optical deflection system is configured to deflect the light rays representing the first viewer right-eye and left-eye views towards the first set of eye-locations such that each light ray is directed to a focused spot smaller than an eye of the first viewer to prevent the first viewer right-eye and left-eye view from being viewed by an eye of the second viewer;

wherein the light rays representing the second viewer right-eye and left-eye views are deflected towards the second viewer set of eye-locations such that each light ray is directed to a second focused spot centered on an eye of the second viewer;

wherein the second focused spot is larger than an eye of the second viewer by an amount configured to facilitate a target amount of movement of the second viewer; and

wherein the eye-tracking system is further configured to determine movement of at least one of the first and second viewer set of eye-locations, estimate a refresh time when the autostereoscopic 3D-displayed images will refresh the displayed 3D images, and estimate future first and second viewer eye-locations relative to the display device based at least in part on the determined movement pattern of the at least one of the first and second viewer set of eye-locations corresponding with the first and second viewer set of eye-locations at the estimated refresh time, and wherein the active solid-state optical deflection system is further configured to direct the light rays based at least in part on the estimated future first and second viewer eye-locations.

15. The autostereoscopic 3D-display device of claim 14 , wherein the active solid-state deflection system comprises an acousto-optical deflector, and wherein the acousto-optical deflectors use surface acoustic waves to deflect the light rays.

16. The autostereoscopic 3D-display device of claim 14 , wherein the display screen comprises a lenticular display screen.

17. The autostereoscopic 3D-display device of claim 14 , further comprising processing circuitry, wherein the eye-tracking system is further configured to determine one or more updated first and second viewer sets of eye-locations, and wherein the eye-tracking system is configured to use the estimated refresh time as a basis for timing when to determine the updated first and second viewer sets of eye-locations.

18. The autostereoscopic 3D-display device of claim 14 , wherein the autostereoscopic 3D-display device uses an acousto-optical deflection system to deflect the light rays towards a direction associated with the first and second viewer sets of eye-locations and wherein the autostereoscopic 3D-display device uses electro-optical deflectors to deflect the light rays towards each of the first and second viewer left eye-locations and the first and second viewer right eye-locations.

19. The autostereoscopic 3D-display device of claim 14 , wherein the autostereoscopic 3D-display device uses an electro-optical deflection system to deflect the light rays towards a direction associated with the first viewer set of eye locations and the second viewer set of eye-locations and wherein the autostereoscopic 3D-display device uses acousto-optical deflectors to deflect the light rays towards each of the first and second viewer left eye-locations and the first and second viewer right eye-locations.

20. The autostereoscopic 3D-display device of claim 14 , wherein the display screen sequentially displays the first and second viewer left-eye and the first and second viewer right-eye views of the 3D images by alternating between displaying the first and second viewer left-eye view of the 3D images and displaying the first and second viewer right-eye view of the 3D images.

21. A display control system for controlling the display of images by an autostereoscopic 3D-display device to a first and a second viewer, the display control system comprising:

a processor;

a set of communication interfaces comprising (a) a display-screen interface, (b) an eye-tracking system interface, and (c) a deflector-system interface;

a non-transitory computer-readable medium; and

program instructions stored on the non-transitory computer-readable medium and executable by the processor to cause the processor to:

receive, via the eye-tracking system interface, eye-location data indicative of a first and second viewer sets of eye-locations relative to the autostereoscopic 3D-display device, wherein the first and second viewer sets of eye-locations comprises a first and second viewer left-eye location and a first and second viewer right-eye location;

transmit screen control signaling via the display-screen interface directing the autostereoscopic 3D-display device to sequentially output light rays representing a first and second viewer left-eye view of 3D images and light rays representing a first and second viewer right-eye view of the 3D images; and

transmit deflector control signaling via the deflector-system interface directing an active solid-state deflector system to deflect the output light rays representing the first and second viewer left-eye views and the first and second viewer right-eye views of the 3D images towards the first and second viewer set of eye-locations in accordance with the received eye-location data;

wherein the deflector control signaling causes the active solid-state deflector system to deflect the light rays representing the first viewer right-eye and left-eye views towards the first set of eye-locations such that each light ray is directed to a focused spot smaller than an eye of the first viewer to prevent the first viewer right-eye and left-eye view from being viewed by an eye of the second viewer;

wherein the light rays representing the second viewer right-eye and left-eye views are deflected towards the second viewer set of eye-locations such that each light ray is directed to a second focused spot centered on an eye of the second viewer;

wherein the second focused spot is larger than an eye of the second viewer by an amount configured to facilitate a target amount of movement of the second viewer;

wherein the received eye-location data further indicates movement of the first and second viewer sets of eye-locations;

wherein the program instructions are further executable to cause the processor to:

i) estimate a refresh time corresponding to when the 3D-display device will refresh the displayed 3D images; and

(ii) estimate future first and second viewer eye-locations based at least in part on the indicated movement of the first and second viewer sets of eye-locations, wherein the estimated future first and second viewer eye-locations correspond with first and second viewer eye-locations at the estimated refresh time;

wherein the deflector control signaling further directs the deflector system to deflect the output light rays based on the estimated first and second viewer future eye-locations.

22. The display control system of claim 21 , wherein the autostereoscopic 3D-display device comprises a lenticular display screen.

23. The display control system of claim 21 ,

receive additional eye-location data via the eye-tracking system interface, wherein the additional eye-location data is indicative of one or more updated first and second viewer sets of eye-locations; and

transmit additional deflector control signaling, via the deflector-system interface, directing the deflector system to deflect the output light rays based at least in part on the additional eye-location data.

24. The display control system of claim 23 , wherein the program instructions are further executable to cause the processor to:

occasionally transmit eye-tracker control signaling, via the eye-tracking system interface, directing the eye-tracking system to generate the additional eye-location data;

and

use the estimated refresh time as a basis for determining when to transmit the eye-tracker control signaling.

25. The display control system of claim 23 , wherein the program instructions are further executable to cause the processor to:

detect motion in an area in front of the autostereoscopic 3D-display device; and

in response to the detected motion transmit eye-tracker control signaling, via the eye-tracking system interface, directing the eye-tracking system to generate the additional eye-location data.

26. The display control system of claim 21 , wherein the display control system is integrated within the autostereoscopic 3D-display device.

27. The display control system of claim 21 , wherein the deflector control signaling directs acousto-optical deflectors to deflect the light rays, and wherein the acousto-optical deflectors use surface acoustic waves to deflect the light rays.

28. The display control system of claim 21 , wherein the deflector system comprises acousto-optical deflectors and electro-optical deflectors, wherein the deflector control signaling directs the electro-optical deflectors to deflect the light rays coarsely, and wherein the deflector control signaling directs the acousto-optical deflectors to deflect the light rays finely.

29. The display control system of claim 21 , wherein the deflector system comprises acousto-optical deflectors and electro-optical deflectors, wherein the deflector control signaling directs the electro-optical deflectors to deflect the light rays towards a direction associated with the first and second viewer sets of eye-locations and wherein the deflector control signaling directs the acousto-optical deflectors to deflect the light rays towards each of the first and second viewer left eye-locations and the first and second viewer right eye-locations.

30. The display control system of claim 21 , wherein sequentially outputting the light rays representing the first and second viewer left-eye views and the first and second viewer right-eye views of the 3D images comprises alternating between (i) outputting the light rays representing the first and second viewer left-eye views and (ii) outputting the light rays representing the first and second viewer right-eye views.

31. A non-transitory computer-readable medium having stored thereon program instructions executable by a processor to cause an autostereoscopic 3D-display device to:

receive eye-location data indicative of a first and second viewer set of eye-locations relative to the autostereoscopic 3D-display device, wherein the first and second viewer sets of eye-locations comprises a first and second viewer left-eye location and a first and second viewer right-eye location, the received eye-location data further indicative of movement of at least one of the first or second sets of eye-locations relative to the display device;

sequentially output, from the 3D-display device, light rays representing a first and second viewer left-eye view of 3D images and light rays representing a first and second viewer right-eye view of the 3D images;

using an active sold-state optical deflector (i) to deflect the output light rays representing the first and second viewer left-eye views towards the first and second viewer left-eye locations and (ii) to deflect the output light rays representing the first and second viewer right-eye views towards the right-eye locations;

estimate a refresh time corresponding to when the display device will refresh the displayed 3D images;

estimate future eye-locations based at least in part on the indicated movement characteristics of the at least one of the first or second set of eye-locations, wherein the estimated future eye-locations correspond with an expected eye-location of at least one of the first or second eye-locations at the estimated refresh time; and

deflect the output light rays towards the estimated future eye-locations at the estimated refresh time;

wherein the active sold-state optical deflector is configured to deflect the light rays representing the first viewer right-eye and left-eye views towards the first set of eye-locations such that each light ray is directed to a focused spot smaller than an eye of the first viewer to prevent the first viewer right-eye and left-eye view from being viewed by an eye of the second viewer;

wherein the light rays representing the second viewer right-eye and left-eye views are deflected towards the second viewer set of eye-locations such that each light ray is directed to a second focused spot centered on an eye of the second viewer; and

wherein the second focused spot is larger than an eye of the second viewer by an amount configured to facilitate a target amount of movement of the second viewer.

32. The non-transitory computer-readable medium of claim 31 , wherein the display screen comprises a lenticular display-screen.

33. The non-transitory computer-readable medium of claim 31 , wherein the autostereoscopic 3D-display device comprises a mobile display-device.

34. The non-transitory computer-readable medium of claim 31 , wherein deflecting the output light rays comprises using surface acoustic waves to deflect the light rays.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Aug 5, 2015
From: AIDEN, EREZ L.; CHAN, ALISTAIR K.; ECKHOFF, PHILIP A.; GATES, WILLIAM; HAGELSTEIN, PETER L.; HYDE, RODERICK A.; KARE, JORDIN T.; LANGER, ROBERT; LEUTHARDT, ERIC C.; MYHRVOLD, NATHAN P.; SCHNALL-LEVIN, MICHAEL; WOOD, LOWELL L., JR.
To: ELWHA LLC
Reel/Frame 036257/0749 →
Continuity (1)
Related Publication 20140139652A1 · May 22, 2014