IP Library Granted Patent US 10,437,325
Granted Patent B2
US 10,437,325 · App. 15/242,341 · Granted Oct 8, 2019

Retinal see through display power level determination method and apparatus

Inventors: Christophe Le Gros (Lausanne, CH); Gatien Cosendey (Vevey, CH)
Assignee: North Inc.
G06F3/013G02B27/0093G02B27/0172G06K9/0061G06T11/60G02B2027/014
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,437,325
App. No.
15/242,341
Granted
Oct 8, 2019
Kind
B2
Abstract

Apparatuses, methods and storage medium associated with retinal see through display are disclosed herein. In embodiments, an apparatus may comprise a retinal see through display wearable by a user; a target luminance calculator to determine a required luminance level; and a display power controller to determine a power level for the retinal see through display, based at least in part on a pupil size of the user and the determined required luminance level. Other embodiments may be described and/or claimed.

Claims (47)

1. An apparatus for displaying images, comprising:

a body wearable by a user;

a retinal see through display disposed in the body, including a tinted lens having a visible light transmission rate:

a target light calculator disposed in the body to determine a required luminance level for output of the retinal see through display, based at least in part on sensed luminance of local light in a local area behind a displayed virtual image, and the visible light transmission rate of the tinted lens of the retinal see through display; and

a display power controller communicatively coupled with the target light calculator and the retinal see through display, and disposed in the body, to determine a power level for the retinal see through display to display images, based at least in part on a pupil size of the user and the determined required luminance level, wherein the pupil size is either measured or calculated based on sensed luminance of ambient light in an immediate surrounding of the user, the immediate surrounding being larger and includes the local area behind the displayed virtual image.

2. The apparatus of claim 1 , further comprising

one or more local light sensors coupled to the target light calculator, and disposed in the body, to sense the luminance of local light in the local area behind the displayed virtual image;

one or more ambient light sensors disposed in the body to sense luminance of the ambient light in the immediate surrounding of the user, and output sensed ambient light luminance data; and

a pupil size calculator coupled to the one or more ambient light sensors and to the display power controller, and disposed in the body, to estimate the pupil size of the user;

wherein the pupil size calculator is to estimate the pupil size based at least in part on the sensed ambient light luminance data.

3. The apparatus of claim 2 , wherein the pupil size calculator is to estimate the pupil size based at least in part on the sensed ambient light luminance data, using a pupil size model that models pupil sizes for various ambient light conditions.

4. The apparatus of claim 1 , further comprising a pupil size detector coupled to the display power controller, and disposed in the body to detect and output the pupil size of the user.

5. The apparatus of claim 1 , further comprising

one or more local light sensors coupled to the target light calculator, and disposed in the body, to sense the luminance of the local light and output sensed local light luminance data associated with the local area behind the displayed virtual image and

a selected one of one or more ambient light sensors or pupil size detector, wherein the one or more ambient light sensors are disposed in the body to sense the luminance of the ambient light and output sensed ambient light luminance data to a pupil size calculator that estimates the pupil size of the user, based at least in part on the sensed ambient light luminance data.

6. The apparatus of claim 5 , wherein the target light calculator is to determine the required luminance level, based at least in part on the sensed local light luminance data, using a required light model that models required light for various local light conditions for the local area behind the displayed virtual image.

7. The apparatus of claim 1 ; wherein the display power controller is to determine the power level of the retinal see through display, such that a display luminance level approximates the determined required luminance level, for the pupil size of the user.

8. The apparatus of claim 7 , further comprising a processor disposed in the body, wherein the target light calculator is a software target light calculator operated by the processor, and the display power controller is a software display power controller operated by the processor.

9. The apparatus of claim 8 , wherein the target light calculator and the display power controller are part of an optical display driver.

10. The apparatus of claim 7 , wherein the display power controller includes or has access to a lookup table of pre-calculated powers to provide various display luminance levels for various pupil sizes.

11. A method for displaying images, comprising:

determining, by a wearable device, a required luminance level for output of a retinal see through display of the wearable device, the retinal see through display having a tinted lens with a visible light transmission rate, and the determining of the required luminance level is based at least in part on sensed luminance of local light in a local area behind a displayed virtual image, and the visible light transmission rate of the tinted lens of the retinal see through display; and

determining, by the wearable device, a power level of a retinal see through display of the wearable device to display images, based at least in part on a pupil size of a user and the determined required luminance level, wherein the pupil size is either measured or calculated based on sensed luminance of ambient light in an immediate surrounding the user is in the immediate surrounding being larger and includes the local area behind the displayed virtual image.

12. The method of claim 11 , further comprising

sensing the local light in the local area behind the displayed virtual image, and outputting sensed local light luminance data, with one or more local light sensors disposed in a body of the wearable device;

sensing the ambient light of the immediate surrounding of the user, and outputting sensed ambient light luminance data, with one or more ambient light sensors disposed in a body of the wearable device; and

estimating the pupil size of the user, based at least in part on the sensed ambient light luminance data.

13. The method of claim 12 , wherein estimating comprises estimating the pupil size based at least in part on the sensed ambient light luminance data, using a pupil size model that models pupil sizes for various ambient light conditions.

14. The method of claim 11 , further comprising detecting and outputting the pupil size of the user using a pupil size detector disposed in a body of the wearable device.

15. The method of claim 11 , further comprising sensing the luminance of the local lights and outputting sensed local light luminance data associated with the local area behind the displayed virtual image, using one or more local light sensors disposed in a body of the wearable device; and

performing a selected one of:

sensing the luminance of the ambient light and outputting sensed ambient light luminance data to a pupil size calculator, using one or more ambient light sensors, wherein the pupil size calculator estimates the pupil size of the user, based at least in part on the sensed ambient light luminance data, or

detecting the pupil size of the user with a pupil size detector.

16. The method of claim 15 , wherein determining the required luminance level comprises determining the required luminance level, based at least in part on the sensed local light luminance data, using a required light model that models required light for various local light conditions for the local area behind the displayed virtual image.

17. The method of claim 11 ; wherein determining a power level comprises determining a power level of the retinal see through display, such that a display luminance level approximates the determined required luminance level, for the pupil size of the user.

18. The method of claim 17 , wherein determining a power level comprises accessing a lookup table of pre-calculated powers to provide various display luminance levels for various pupil sizes, to retrieve the power level.

19. One or more non-transitory computer-readable media comprising instructions that cause a wearable device, in response to execution of the instructions by the wearable device, to:

determine a required luminance level for output of a retinal see through display of the wearable device, the retinal see through display having a tinted lens with a visible light transmission rate, and the determining of the required luminance level is based at least in part on sensed luminance of local light in a local area behind a displayed virtual image, and the visible light transmission rate of the tinted lens of the retinal see through display; and

determine a power level of a retinal see through display of the wearable device to display images, based at least in part on a pupil size of a user and the determined required luminance level, wherein the pupil size is either measured or calculated based on sensed luminance of ambient light in an immediate surrounding the user is in the immediate surrounding being larger and includes the local area behind the displayed virtual image.

20. The one or more non-transitory computer-readable media of claim 19 , wherein the wearable device comprises one or more local light sensors and one or more ambient light sensors disposed in a body of the wearable device, wherein the one or more local light sensors are to sense the luminance of local light in the local area behind the displayed virtual image, and the one or more ambient light sensors are to sense the luminance of the ambient light and output sensed ambient light luminance data; and wherein the wearable device is further caused to estimate the pupil size of the user, based at least in part on the sensed ambient light luminance data.

21. The one or more non-transitory computer-readable media of claim 20 , wherein the wearable device is further caused to estimate the pupil size based at least in part on the sensed ambient light data, using a pupil size model that models pupil sizes for various ambient light conditions.

22. The one or more non-transitory computer-readable media of claim 19 , wherein the wearable device comprises a pupil size detector disposed in a body of the wearable device to detect and output the pupil size of the user.

23. The one or more non-transitory computer-readable media of claim 19 , wherein the wearable device comprises

one or more local light sensors disposed in the body to sense the luminance of the local light and output sensed local light luminance data associated with the local area behind the displayed virtual image; and

a selected one of one or more ambient light sensors or pupil size detector, wherein the one or more ambient light sensors are disposed in the body to sense the luminance of the ambient light and output sensed ambient light luminance data to a pupil size calculator that estimates the pupil size of the user, based at least in part on the sensed ambient light luminance data.

24. The one or more non-transitory computer-readable media of claim 23 , wherein the wearable device is further caused to determine the required luminance level, based at least in part on the sensed local light luminance data, using a required light model that models required light for various local light conditions for the local area behind the displayed virtual image.

25. The one or more non-transitory computer-readable media of claim 19 ; wherein the wearable device is further caused to determine the power level of the retinal see through display, such that a display luminance level approximates the determined required luminance level, for the pupil size of the user.

Assignments (3)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Oct 19, 2020
From: NORTH INC.
To: GOOGLE LLC
Reel/Frame 054113/0814 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jan 10, 2019
From: INTEL CORPORATION
To: NORTH INC.
Reel/Frame 048044/0034 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Aug 19, 2016
From: LE GROS, CHRISTOPHE; COSENDEY, GATIEN
To: INTEL CORPORATION
Reel/Frame 039764/0001 →
Continuity (1)
Related Publication 20180052513A1 · Feb 22, 2018
Cited By (1)
US 12,710,654