IP Library Granted Patent US 11,892,636
Granted Patent B2
US 11,892,636 · App. 17/376,514 · Granted Feb 6, 2024

Systems and methods for operating a display system based on user perceptibility

Inventors: Lionel Ernest Edwin (Hollywood, FL); Ivan Li Chuen Yeoh (Wesley Chapel, FL); Samuel A. Miller (Hollywood, FL); Edwin Joseph Selker (Palo Alto, CA); Adam Charles Carlson (Miami, FL); Bjorn Nicolaas Servatius Vlaskamp (Plantation, FL); Paul M. Greco (Parkland, FL)
Assignee: MAGIC LEAP, INC.
G02B27/0172G02B27/0179G06T19/006G06V40/19G09G5/003G02B2027/011G02B2027/014G02B2027/0138G02B2027/0178G02B2027/0187G09G2354/00
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,892,636
App. No.
17/376,514
Granted
Feb 6, 2024
Kind
B2
Abstract

Systems and methods are disclosed for operating a head-mounted display system based on user perceptibility. The display system may be an augmented reality display system configured to provide virtual content on a plurality of depth planes by presenting the content with different amounts of wavefront divergence. Some embodiments include obtaining an image captured by an imaging device of the display system. Whether a threshold measure or more of motion blur is determined to be exhibited in one or more regions of the image. Based on a determination that the threshold measure or more of motion blur is exhibited in one or more regions of the image, one or more operating parameters of the wearable display are adjusted. Example operating parameter adjustments comprise adjusting the depth plane on which content is presented (e.g., by switching from a first depth plane to a second depth plane), adjusting a rendering quality, and adjusting power characteristics of the system.

Claims (40)

1. A method implemented by a display system comprising one or more processors, the display system presenting augmented reality virtual content to a user, and the method comprising:

obtaining an image of an eye of the user;

determining, from one or more portions of the image, one or more measures associated with motion blur represented in the image;

determining, based on the one or more measures, that the image shows performance of a saccade, which saccade may be a voluntary movement or an involuntary movement; and

performing, by the display system, one or more actions in response to the determining that the image shows performance of the saccade, the actions being associated with a reduction in visual perceptibility.

2. The method of claim 1 , wherein determining one or more measures associated with motion blur comprises:

extracting frequency information from the image; and

determining a magnitude of motion blur based on the frequency information, the magnitude being utilized as the measure of motion blur.

3. The method of claim 2 , wherein determining one or more measures associated with motion blur comprises:

convolving one or more image kernels with at least a portion of the image; and

determining the measure based on the convolution.

4. The method of claim 1 , wherein determining one or more measures comprises determining a plurality of measures, each measure being associated with a respective motion blur determination scheme; and

wherein determining that the image shows performance of a saccade is based on the plurality of measures.

5. The method of claim 1 , wherein the one or more measures are averaged, and wherein determining that the image shows performance of a saccade comprises determining that the average exceeds a predetermined threshold value.

6. The method of claim 1 , wherein determining one or more measures comprises determining a plurality of measures, and determining that the image shows performance of a saccade comprises determining that a threshold number of the measures exceeds respective predetermined threshold values.

7. The method of claim 6 , wherein the threshold number is a majority of the measures or a user-selectable threshold number.

8. The method of claim 1 , further comprising:

determining a magnitude and direction associated with the motion blur.

9. The method of claim 8 , further comprising:

accessing information identifying a three-dimensional fixation point associated with the user, the fixation point representing a gaze of the user; and

estimating an updated three-dimensional fixation point based on the determined magnitude and direction, the updated three-dimensional fixation point representing a gaze of the user subsequent to the saccade.

10. The method of claim 1 , wherein the display system further comprises:

a wearable display configured to present virtual content to a user; and

an imaging device, wherein obtaining an image of the eye of the user is performed by the imaging device.

11. The method of claim 1 , wherein determining, based on the one or more measures, that the image shows performance of a saccade comprises determining whether one or more measures associated with motion blur is exhibited in one or more regions of the image.

12. The method of claim 11 , wherein determining whether one or more measures of motion blur is exhibited in one or more regions of the image comprises:

identifying a target region of the image corresponding to a target portion of the user's eye; and

determining whether the measure or more of motion blur is exhibited in the identified region of the image.

13. The method of claim 12 , wherein the display system further comprises an infrared light source, wherein the target portion of the user's eye comprises an illuminated portion of the user's eye, wherein the illuminated portion of the user's eye is illuminated by the infrared light source.

14. The method of claim 12 , wherein the target portion of the user's eye comprises one or more eye features selected from the group consisting of a pupil, at least a portion of an iris, and a portion of a limbic boundary.

15. The method of claim 11 , further comprising:

adjusting, based at least in part on a determination that the image shows performance of a saccade, one or more power settings of the display system, or one or more virtual content rendering settings.

16. The method of claim 11 , further comprising:

changing, in response to a determination that the image shows performance of a saccade, a depth plane on which virtual content is presented.

17. The method of claim 1 , where, the display system further comprises a stack of waveguides, wherein one or more of the waveguides are configured to output light with different amounts of wavefront divergence than others of the waveguides.

18. The method of claim 1 , wherein performing one or more actions comprises:

determining an accommodation vergence mismatch; and

in response to determining that the accommodation vergence mismatch exceeds a threshold, change a wavefront divergence of the outputted light forming the virtual content.

19. The method of claim 1 , wherein performing one or more actions comprises switching depth planes of at least a portion of the virtual content presented by the display system.

20. The method of claim 1 , wherein performing one or more actions comprises reducing rendering quality of at least a portion of the virtual content presented by the display system.

Assignments (5)
SECURITY INTEREST Recorded Oct 28, 2025
From: MAGIC LEAP, INC.; MENTOR ACQUISITION ONE, LLC; MOLECULAR IMPRINTS, INC.
To: CITIBANK, N.A., AS COLLATERAL AGENT
Reel/Frame 073388/0027 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Sep 30, 2022
From: EDWIN, LIONEL ERNEST; YEOH, IVAN LI CHUEN; MILLER, SAMUEL A.; SELKER, EDWIN JOSEPH; CARLSON, ADAM C.; VLASKAMP, BJORN NICOLAAS SERVATIUS; GRECO, PAUL M.
To: MAGIC LEAP, INC.
Reel/Frame 061276/0239 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Sep 20, 2022
From: EDWIN, LIONEL ERNEST; YEOH, IVAN LI CHUEN; MILLER, SAMUEL A.; SELKER, EDWIN JOSEPH; CARLSON, ADAM C.; VLASKAMP, BJORN NICOLAAS SERVATIUS; GRECO, PAUL M.
To: MAGIC LEAP, INC.
Reel/Frame 061157/0725 →
SECURITY INTEREST Recorded May 24, 2022
From: MOLECULAR IMPRINTS, INC.; MENTOR ACQUISITION ONE, LLC; MAGIC LEAP, INC.
To: CITIBANK, N.A., AS COLLATERAL AGENT
Reel/Frame 060338/0665 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jul 15, 2021
From: EDWIN, LIONEL ERNEST; YEOH, IVAN LI CHUEN; MILLER, SAMUEL A.; SELKER, EDWIN JOSEPH; CARLSON, ADAM C.; VLASKAMP, BJORN NICOLAAS SERVATIUS; GRECO, PAUL M.
To: MAGIC LEAP, INC.
Reel/Frame 056872/0500 →
Continuity (4)
Continuation 16389529 · Apr 19, 2019
Provisional Application 62660180 · Apr 19, 2018
Provisional Application 62702153 · Jul 23, 2018
Related Publication 20210341742A1 · Nov 4, 2021