IP Library Granted Patent US 12,504,633
Granted Patent B2
US 12,504,633 · App. 18/901,851 · Granted Dec 23, 2025

Display system and method for providing variable accommodation cues using multiple intra-pupil parallax views formed by light emitter arrays

Inventor: Michael Anthony Klug (Austin, TX)
Assignee: Magic Leap, Inc.
G02B27/0172G02B6/0076G02B25/001G02B27/102G02B27/149G02B27/30G02B30/24G09G3/003H04N13/344H04N13/398G02B2027/0134G02B2027/0178
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,504,633
App. No.
18/901,851
Granted
Dec 23, 2025
Kind
B2
Abstract

A display system is configured to direct a plurality of parallactically-disparate intra-pupil images into a viewer's eye. The parallactically-disparate intra-pupil images provide different parallax views of a virtual object, and impinge on the pupil from different angles. The wavefronts of light forming the images approximate a continuous divergent wavefront and provide selectable accommodation cues for the user, depending on the amount of parallax disparity between the intra-pupil images. The images may be formed by an emissive micro-display. Each pixel formed by the micro-display may be formed by one of a group of light emitters, which are at different locations such that the emitted light takes different paths to the eye to provide different amounts of parallax disparity.

Claims (21)

1 . A method for displaying virtual image content using a head-mounted display system, the method performed by one or more processors of the display system, the method comprising:

determining light emitters of each of an array of groups of light emitters to activate based upon a level of parallax disparity associated with images formed by the light emitters, the array of groups of light emitters included in a micro-display of the display system;

activating a first light emitter of the groups of light emitters to form a first parallactically-disparate intra-pupil image;

activating a second light emitter of the groups of light emitters to form a second parallactically-disparate intra-pupil image; and

displaying a virtual object to appear to be on a depth plane by injecting image light including the first and second parallactically-disparate intra-pupil images of the object into a same eye of a viewer,

wherein the first and second parallactically-disparate intra-pupil images provide different views of the virtual object for a same eye of a user of the display system, and

wherein activating the first light emitter of the groups of light emitters overlaps in time with activating the second light emitter of the groups of light emitters, to inject the first and second parallactically-disparate intra-pupil images into the same eye of the user within a flicker fusion threshold that is 1/60 of a second.

2 . The method of claim 1 , wherein the first and second parallactically-disparate intra-pupil images are injected into the same eye of the user simultaneously.

3 . The method of claim 1 , wherein the display system further comprises an array of light collimators overlying the light emitters, wherein each light collimator is associated with one of the groups of light emitters and extends across all light emitters of the associated group of light emitters.

4 . The method of claim 3 , wherein the light collimators are lenslets.

5 . The method of claim 3 , wherein the light collimators are lenticular lenslets configured to provide different beams of light, from light emitters of an associated group of light emitters, to different locations along a first axis.

6 . The method of claim 1 , wherein the display system further comprises an array of selectively-activated shutters for selectively transmitting the image light to the eye from different locations.

7 . The method of claim 6 , wherein the array of selectively-activated shutters is disposed within an eye-box volume of the display system.

8 . The method of claim 6 , wherein the display system further comprises a pupil relay combiner eyepiece configured to relay the image light to the eye of the user, wherein the array of selectively-activated shutters are configured to regulate propagation of the image light to the pupil relay combiner eyepiece.

9 . The method of claim 8 , wherein the pupil relay combiner eyepiece comprises a plurality of waveguides, each waveguide comprising in-coupling optical elements and out-coupling optical elements.

10 . The method of claim 6 , wherein the shutters comprise chemical species having reversibly changeable states, the states providing different amounts of light transmission.

11 . The method of claim 6 , wherein the shutters comprise shutters comprise selectively-movable physical structures.

12 . The method of claim 1 , further comprising:

determining a gaze of the eye using an eye tracking sensor of the display system; and

selecting image content for the intra-pupil images based upon the determined gaze of the eye.

13 . The method of claim 1 , wherein the first and second parallactically-disparate intra-pupil images are injected into the eye at different angles.

Assignments (2)
SECURITY INTEREST Recorded Oct 29, 2025
From: MAGIC LEAP, INC.; MENTOR ACQUISITION ONE, LLC; MOLECULAR IMPRINTS, INC.
To: CITIBANK, N.A., AS COLLATERAL AGENT
Reel/Frame 073430/0225 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Dec 16, 2024
From: KLUG, MICHAEL ANTHONY
To: MAGIC LEAP, INC.
Reel/Frame 069597/0770 →
Continuity (6)
Continuation 18482893 · Oct 8, 2023
Continuation 17706407 · Mar 28, 2022
Continuation 16803563 · Feb 27, 2020
Provisional Application 62815225 · Mar 7, 2019
Provisional Application 62812142 · Feb 28, 2019
Related Publication 20250020935A1 · Jan 16, 2025
References Cited (143)
US 5914810A · Watts · 1999 [cited by applicant]
US 6445406B1 · Taniguchi et al. · 2002 [cited by applicant]
US 6466185B2 · Sullivan et al. · 2002 [cited by applicant]
US 6590573B1 · Geshwind · 2003 [cited by applicant]
US 6850221B1 · Tickle · 2005 [cited by applicant]
US 8573784B2 · Yeh et al. · 2013 [cited by applicant]
US 8950867B2 · Macnamara · 2015 [cited by applicant]
US 9081426B2 · Armstrong · 2015 [cited by applicant]
US 9215293B2 · Miller · 2015 [cited by applicant]
US 9310559B2 · Macnamara · 2016 [cited by applicant]
US 9348143B2 · Gao et al. · 2016 [cited by applicant]
US D758367S · Natsume · 2016 [cited by applicant]
US 9417452B2 · Schowengerdt et al. · 2016 [cited by applicant]
US 9470906B2 · Kaji et al. · 2016 [cited by applicant]
US 9547174B2 · Gao et al. · 2017 [cited by applicant]
US 9671566B2 · Abovitz et al. · 2017 [cited by applicant]
US 9740006B2 · Gao · 2017 [cited by applicant]
US 9791700B2 · Schowengerdt · 2017 [cited by applicant]
US 9841563B2 · Lapstun · 2017 [cited by applicant]
US 9851563B2 · Gao et al. · 2017 [cited by applicant]
US 9857591B2 · Welch et al. · 2018 [cited by applicant]
US 9874749B2 · Bradski et al. · 2018 [cited by applicant]
US 10106153B1 · Xiao et al. · 2018 [cited by applicant]
US 10831038B2 · Inaba et al. · 2020 [cited by applicant]
US 11073693B2 · Akutsu et al. · 2021 [cited by applicant]
US 11231544B2 · Lin et al. · 2022 [cited by applicant]
US 11231584B2 · Klug et al. · 2022 [cited by applicant]
US 11287657B2 · Klug · 2022 [cited by applicant]
US 11815688B2 · Klug · 2023 [cited by applicant]
US 11874486B2 · Bartlett · 2024 [cited by examiner]
US 12158586B2 · Klug · 2024 [cited by applicant]
US 20020113752A1 · Sullivan et al. · 2002 [cited by applicant]
US 20030197933A1 · Sudo et al. · 2003 [cited by applicant]
US 20060028436A1 · Armstrong · 2006 [cited by applicant]
US 20060279567A1 · Schwerdtner et al. · 2006 [cited by applicant]
US 20070081123A1 · Lewis · 2007 [cited by applicant]
US 20080084544A1 · Hall et al. · 2008 [cited by applicant]
US 20080198431A1 · Schwerdtner · 2008 [cited by applicant]
US 20090005961A1 · Grabowski et al. · 2009 [cited by applicant]
US 20090237914A1 · Lai et al. · 2009 [cited by applicant]
US 20100232016A1 · Landa et al. · 2010 [cited by applicant]
US 20110310232A1 · Wilson et al. · 2011 [cited by applicant]
US 20120127062A1 · Bar-Zeev et al. · 2012 [cited by applicant]
US 20120162549A1 · Gao et al. · 2012 [cited by applicant]
US 20120306860A1 · Hatta et al. · 2012 [cited by applicant]
US 20130082922A1 · Miller · 2013 [cited by applicant]
US 20130117377A1 · Miller · 2013 [cited by applicant]
US 20130125027A1 · Abovitz · 2013 [cited by applicant]
US 20130126618A1 · Gao · 2013 [cited by applicant]
US 20130208234A1 · Lewis · 2013 [cited by applicant]
US 20130242262A1 · Lewis · 2013 [cited by applicant]
US 20130286053A1 · Fleck et al. · 2013 [cited by applicant]
US 20140063077A1 · Wetzstein et al. · 2014 [cited by applicant]
US 20140071539A1 · Gao · 2014 [cited by applicant]
US 20140177023A1 · Gao et al. · 2014 [cited by applicant]
US 20140218468A1 · Gao et al. · 2014 [cited by applicant]
US 20140267420A1 · Schowengerdt et al. · 2014 [cited by applicant]
US 20140306866A1 · Miller et al. · 2014 [cited by applicant]
US 20150016777A1 · Abovitz et al. · 2015 [cited by applicant]
US 20150103306A1 · Kaji et al. · 2015 [cited by applicant]
US 20150178939A1 · Bradski et al. · 2015 [cited by applicant]
US 20150205126A1 · Schowengerdt · 2015 [cited by applicant]
US 20150205259A1 · Kim et al. · 2015 [cited by applicant]
US 20150222883A1 · Welch · 2015 [cited by applicant]
US 20150222884A1 · Cheng · 2015 [cited by applicant]
US 20150234190A1 · Schowengerdt · 2015 [cited by applicant]
US 20150235418A1 · Schowengerdt · 2015 [cited by applicant]
US 20150235446A1 · Schowengerdt · 2015 [cited by applicant]
US 20150235448A1 · Schowengerdt · 2015 [cited by applicant]
US 20150260994A1 · Akutsu et al. · 2015 [cited by applicant]
US 20150268415A1 · Schowengerdt et al. · 2015 [cited by applicant]
US 20150302652A1 · Miller et al. · 2015 [cited by applicant]
US 20150309263A2 · Abovitz et al. · 2015 [cited by applicant]
US 20150326570A1 · Publicover et al. · 2015 [cited by applicant]
US 20150346490A1 · Tekolste et al. · 2015 [cited by applicant]
US 20150346495A1 · Welch et al. · 2015 [cited by applicant]
US 20150370075A1 · Ato et al. · 2015 [cited by applicant]
US 20160011419A1 · Gao · 2016 [cited by applicant]
US 20160026253A1 · Bradski et al. · 2016 [cited by applicant]
US 20160116739A1 · TeKolste et al. · 2016 [cited by applicant]
US 20160139402A1 · Lapstun · 2016 [cited by applicant]
US 20160238845A1 · Alexander et al. · 2016 [cited by applicant]
US 20160270656A1 · Samec et al. · 2016 [cited by applicant]
US 20160377865A1 · Alexander et al. · 2016 [cited by applicant]
US 20170078652A1 · Hua et al. · 2017 [cited by applicant]
US 20170091917A1 · Bronstein et al. · 2017 [cited by applicant]
US 20170102545A1 · Hua et al. · 2017 [cited by applicant]
US 20170118458A1 · Gronholm et al. · 2017 [cited by applicant]
US 20180025640A1 · Micks et al. · 2018 [cited by applicant]
US 20180031838A1 · Browne · 2018 [cited by applicant]
US 20180052331A1 · Inaba · 2018 [cited by applicant]
US 20180113311A1 · Klug · 2018 [cited by examiner]
US 20180348960A1 · Shinohara et al. · 2018 [cited by applicant]
US 20190056587A1 · Zhang et al. · 2019 [cited by applicant]
US 20190075252A1 · Zhao et al. · 2019 [cited by applicant]
US 20190253667A1 · Valli · 2019 [cited by applicant]
US 20190293939A1 · Sluka · 2019 [cited by applicant]
US 20190302583A1 · Taniguchi et al. · 2019 [cited by applicant]
US 20190317266A1 · Shinohara et al. · 2019 [cited by applicant]
US 20190324183A1 · Shinohara et al. · 2019 [cited by applicant]
US 20200033803A1 · Christmas · 2020 [cited by applicant]
US 20200279483A1 · Eather et al. · 2020 [cited by applicant]
US 20200301147A1 · Klug · 2020 [cited by examiner]
US 20210096565A1 · Xie et al. · 2021 [cited by applicant]
US 20220146834A1 · Klug et al. · 2022 [cited by applicant]
US 20220221724A1 · Klug · 2022 [cited by applicant]
US 20240036331A1 · Klug · 2024 [cited by applicant]
US 20240061300A1 · Van Heugten · 2024 [cited by examiner]
CN 104570366A · 2015 [cited by applicant]
CN 104749779A · 2017 [cited by applicant]
CN 109073905A · 2018 [cited by applicant]
CN 106485233A · 2020 [cited by applicant]
JP H0991468A · 1997 [cited by applicant]
JP H11196351A · 1999 [cited by applicant]
JP 2001042257A · 2001 [cited by applicant]
JP 2001197522A · 2001 [cited by applicant]
JP 2002131694A · 2002 [cited by applicant]
JP 2002228978A · 2002 [cited by applicant]
JP 2002228987A · 2002 [cited by applicant]
JP 2002318365A · 2002 [cited by applicant]
JP 2001078234A · 2004 [cited by applicant]
JP 2007518113A · 2007 [cited by applicant]
JP 2008219788A · 2008 [cited by applicant]
JP 2011244349A · 2011 [cited by applicant]
JP 2013024910A · 2013 [cited by applicant]
JP 2020118963A · 2020 [cited by applicant]
WO 2016105282A1 · 2016 [cited by applicant]
WO 2020176783A1 · 2020 [cited by applicant]
ARToolKit: https://web.archive.org/web/20051013062315/http://www.hitl.washington.edu:80/artoolkit/documentation/hardware.htm, archived Oct. 13, 2005. [cited by applicant]
Azuma, “A Survey of Augmented Reality,” Teleoperators and Virtual Environments 6, 4 (Aug. 1997), pp. 355-385. https://web.archive.org/web/20010604100006/http://www.cs.unc.edu/azuma/ARpresence.pdf. [cited by applicant]
Azuma, “Predictive Tracking for Augmented Realty,” TR95-007, Department of Computer Science, UNC-Chapel Hill, NC, Feb. 1995. [cited by applicant]
Bimber, et al., “Spatial Augmented Reality—Merging Real and Virtual Worlds,” 2005 https://web.media.mit.edu/raskar/book/BimberRaskarAugmentedRealityBook.pdf. [cited by applicant]
EP24192846.4 Extended European Search Report dated Nov. 22, 2024. [cited by applicant]
Huang F.-C et al., “The Light Field Stereoscope—Immersive Computer Graphics via Factored Near-Eye Light Field displays with Focus Cues”, SIGGRAPH Emerging Technologies (Jul. 2015) in 12 pages. [cited by applicant]
International Preliminary Report on Patentability for PCT Application No. PCT/US20/20183, dated Aug. 25, 2021. [cited by applicant]
International Search Report and Written Opinimn for PCT Application No. PCT/US20/20183, dated Jun. 25, 2020. [cited by applicant]
Jacob, “Eye Tracking in Advanced Interface Design,” Human-Computer Interaction Lab Naval Research Laboratory, Washington, D.C. / paper/ in Virtual Environments and Advanced Interface Design, ed. by W. Barfield and T.A. … [cited by applicant]
Kanebako, et al., “Time-multiplexing display module for high-density directional display,” SPIE—International Society for Optical Engineering, Proceedings, vol. 6803, Feb. 14, 2008. [cited by applicant]
Kim et al., “Full parallax multifocus three-dimensional display using a slanted light source array,” Optical Engineering, vol. 50, No. 11, Nov. 1, 2011. [cited by applicant]
Kim, et al., “Development of three types of multi-focus 3D display,” Proc. of SPIE vol. 8043, May 19, 2011. [cited by applicant]
Maimone, et al., “High Efficiency Near-Eye Light Field Display,” ACM Trans. Graph, article, Nov. 1, 2013. [cited by applicant]
Tanriverdi and Jacob, “Interacting With Eye Movements in Virtual Environments,” Department of Electrical Engineering and Computer Science, Tufts University, Medford, MA-paper/Proc. AMC CHI 2000 Human Factors in Computin… [cited by applicant]
Wetzstein G. et al., “Tensor Displays: Compressive Light Field Synthesis using Multilayer Displays with Directional Backlighting”, Association for Computing Machinery (Jul. 2012); 11 pages. [cited by applicant]