IP Library Granted Patent US 11,288,503
Granted Patent B2
US 11,288,503 · App. 16/673,348 · Granted Mar 29, 2022

Systems and methods for image adjustment based on pupil size

Inventors: Barry David Silverstein (Kirkland, WA); Bo Zhang (Kenmore, WA); Yijing Fu (Redmond, WA)
Assignee: FACEBOOK TECHNOLOGIES, LLC
G06K9/00604G06F3/013G06T7/20G06T7/90
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,288,503
App. No.
16/673,348
Granted
Mar 29, 2022
Kind
B2
Abstract

Systems and methods for adjusting an image based on pupil size are disclosed. Particularly, a system for adjusting an image being presented on a display includes a processor. The processor is configured to identify a size of a pupil of a user viewing an image presented on the display, determine an intensity map based at least on the size of the pupil, and adjust the intensity values of at least a portion of the image using the intensity map. The intensity map indicates an amount of adjustment to intensity values of at least a portion of the image being displayed.

Claims (54)

1. A non-transitory computer-readable storage medium coupled to one or more processors, the computer-readable storage medium having instructions encoded thereon that, when executed by the one or more processors, cause the one or more processors to:

identify a size of a pupil of a user viewing an image presented on a display;

determine an intensity map based at least on the size of the pupil, wherein the intensity map indicates an amount of adjustment to intensity values of at least a portion of the image, the intensity map comprising an adjustment for certain portions or pixels on the image; and

adjust the intensity values of at least the portion of the image using the intensity map.

2. The non-transitory computer-readable storage medium of claim 1 , further storing program instructions for causing the one or more processors to receive the size of the pupil from an eye tracking system.

3. The non-transitory computer-readable storage medium of claim 1 , further storing program instructions for causing the one or more processors to identify the size of the pupil by estimating the size of the pupil based at least on ambient light and an intensity of the image presented on the display.

4. The non-transitory computer-readable storage medium of claim 1 , further storing program instructions for causing the one or more processors to determining the intensity map by:

selecting the intensity map from a plurality of intensity maps, the intensity map corresponding to the size of the pupil, and wherein each of the plurality of intensity maps corresponds to a different size of the pupil.

5. The non-transitory computer-readable storage medium of claim 1 , wherein the intensity map indicates a predetermined amount of adjustment to red, green, and blue intensity values of the image to correspond to red, green, and blue intensity values of a reference image.

6. The non-transitory computer-readable storage medium of claim 1 , wherein the display further comprises a waveguide and a projection device, the projection device configured to project a light beam toward the waveguide, and the waveguide configured to receive the light beam and present the image on the display.

7. The non-transitory computer-readable storage medium of claim 6 , further storing program instructions for causing the one or more processors to adjust the image by transmitting a signal to the projection device of the display, wherein the signal is configured to cause the projection device to adjust at least a portion of a light beam in order to adjust red, green, and blue intensity values corresponding to the portion of the image presented on the display.

8. A non-transitory computer-readable storage medium coupled to one or more processors, the computer-readable storage medium having instructions encoded thereon that, when executed by the one or more processors, cause the one or more processors to:

identify a size of pupil of a user viewing an image presented on a display;

determine an intensity map based at least on the size of the pupil, wherein the intensity map indicates an amount of adjustment to intensity values of at least a portion of the image;

receive a gaze angle of the user from an eye tracking system and determine the intensity map based at least on the size of the pupil and the gaze angle of the user; and

adjust the intensity values of at least a portion of the image using the intensity map.

9. A non-transitory computer-readable storage medium coupled to one or more processors, the computer-readable storage medium having instructions encoded thereon that, when executed by the one or more processors, cause the one or more processors to:

identify a size of pupil of a user viewing an image presented on a display;

select a first intensity map from a plurality of intensity maps, the first intensity map corresponding to a smaller size of the pupil;

interpolate an intensity map based at least on the size of the pupil and the first intensity map; and

adjust intensity values of at least a portion of the image using the intensity map.

10. The non-transitory computer-readable storage medium of claim 9 , further storing program instructions for causing the one or more processors to determine the intensity map by:

selecting a second intensity map from the plurality of intensity maps, the second intensity map corresponding to a larger size of the pupil; and

interpolating the intensity map based on the size of the pupil, the first intensity map, and the second intensity map.

11. The non-transitory computer-readable storage medium of claim 10 , wherein the interpolating comprises either a linear interpolation algorithm or a weighted interpolation algorithm.

12. A method of adjusting an image on a display of a wearable device comprising:

sensing, by a processor of a wearable device, a size of a pupil of a user viewing an image presented on a display;

selecting, by the processor, an intensity map based at least on the size of the pupil, wherein the intensity map is preconfigured to store adjustment values for at least a portion of the image; and

adjusting at least the portion of the image using the intensity map.

13. The method of claim 12 , wherein determining the intensity map further comprises:

selecting, by the processor from a memory, the intensity map from a plurality of intensity maps, wherein each of the plurality of intensity maps corresponds to a respective size of a pupil.

14. The method of claim 12 , wherein the intensity map is a pre-configured map that indicates an amount of red, green, and blue intensity values for adjusting the portion of the image to correspond to red, green, and blue intensity values of a reference image.

15. The method of claim 12 , wherein presenting the image on the display comprises:

projecting, via projection device, a light beam toward a waveguide display; and

presenting, on the waveguide display, the image corresponding the light beam.

16. The method of claim 15 , wherein adjusting the image further comprises, transmitting a signal to the projection device of the display, wherein the signal is configured to cause the projection device to adjust at least a portion of a light beam in order to adjust red, green, and blue intensity values at the portion of the image presented on the display.

17. A method of adjusting an image on a display of a wearable device comprising:

determining, by a processor of a wearable device, a size of a pupil of a user viewing an image presented on a display;

selecting, by the processor from a memory, a first intensity map from a plurality of intensity maps, the first intensity map corresponding to a smaller size of the pupil;

interpolating an intensity map based at least on the size of the pupil and the first intensity map; and

adjusting at least a portion of the image using the intensity map.

18. The method of claim 17 , wherein determining the intensity map further comprises:

selecting a second intensity map from the plurality of intensity maps, the second intensity map corresponding to a larger size of the pupil; and

interpolating the intensity map based on the size of the pupil, the first intensity map, and the second intensity map.

19. A method of adjusting an image on a display of a wearable device comprising:

determining, by a processor of a wearable device, a size of a pupil of a user viewing an image presented on a display;

determining, by the processor, an intensity map based at least on the size of the pupil and a gaze angle of the user; and

adjusting at least a portion of the image using the intensity map.

20. A head wearable display comprising:

a housing configured to be worn on a head of a user;

a display of the housing configured to present an image to the user; and

one or more processors coupled to a non-transitory computer-readable storage medium having instructions encoded thereon that, when executed by one or more processors, cause the one or more processors to:

determine an intensity map based at least on information regarding a size of a pupil of the user, wherein the intensity map indicates a predetermined amount of adjustment to red, green, and blue intensity values of at least a portion of the image; and

adjust the red, green and blue intensity values of at least the portion of the image using the intensity map.

Assignments (2)
CHANGE OF NAME Recorded Jul 22, 2022
From: FACEBOOK TECHNOLOGIES, LLC
To: META PLATFORMS TECHNOLOGIES, LLC
Reel/Frame 060816/0634 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jan 6, 2020
From: SILVERSTEIN, BARRY DAVID; ZHANG, BO; FU, YIJING
To: FACEBOOK TECHNOLOGIES, LLC
Reel/Frame 051426/0676 →