IP Library Granted Patent US 12,693,536
Granted Patent B2
US 12,693,536 · App. 18/490,169 · Granted Jul 28, 2026

System and method for presenting image content on multiple depth planes by providing multiple intra-pupil parallax views

Inventors: Michael Anthony Klug (Austin, TX); Robert Konrad (Palo Alto, CA); Gordon Wetzstein (Palo Alto, CA); Brian T. Schowengerdt (Seattle, WA); Michal Beau Dennison Vaughn (Round Rock, TX)
Assignee: Magic Leap, Inc.
G02B27/0172G02B27/0179G02B30/24G02B30/34H04N13/128H04N13/339H04N13/341H04N13/344H04N13/383H04N13/398G02B2027/0134G02B2027/0187H04N13/324H04N13/346H04N2213/008
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,693,536
App. No.
18/490,169
Filed
Oct 19, 2023
Granted
Jul 28, 2026
Kind
B2
Art Unit
2481
USPC
348/53
Abstract

An augmented reality display system is configured to direct a plurality of parallactically-disparate intra-pupil images into a viewer's eye. The parallactically-disparate intra-pupil images provide different parallax views of a virtual object, and impinge on the pupil from different angles. In the aggregate, the wavefronts of light forming the images approximate a continuous divergent wavefront and provide selectable accommodation cues for the user, depending on the amount of parallax disparity between the intra-pupil images. The amount of parallax disparity is selected using a light source that outputs light for different images from different locations, with spatial differences in the locations of the light output providing differences in the paths that the light takes to the eye, which in turn provide different amounts of parallax disparity. Advantageously, the wavefront divergence, and the accommodation cue provided to the eye of the user, may be varied by appropriate selection of parallax disparity, which may be set by selecting the amount of spatial separation between the locations of light output.

Claims (23)

1 . A head-mountable display system comprising:

a light source configured to emit light from different output locations;

a spatial light modulator configured to receive the light emitted by the light source and to modulate the received light to form virtual content, wherein the light that is emitted by the light source from at least two of the different output locations is received and modulated simultaneously at different respective areas of the spatial light modulator to form images of the virtual content, and wherein the light that is emitted from the at least two different output locations of the light source and that is received at the different respective areas of the spatial light modulator has a same polarization state; and

projection optics configured to direct the modulated light from the spatial light modulator toward an eye of a wearer of the head-mountable display system to inject the images of the virtual content into the eye,

wherein the display system is configured to display the virtual content on a depth plane by simultaneously injecting, into the eye, a set of images of the virtual content that are output simultaneously from the different respective areas of the spatial light modulator that have a lateral separation.

2 . The display system of claim 1 , wherein the light source includes at least two light emitters that emit light of different colors.

3 . The display system of claim 1 , wherein the light source comprises a plurality of selectively-activated light-emitting regions that respectively include the different output locations, wherein each of the light-emitting regions comprises at least one light emitter, and wherein the light from the different light-emitting regions of the light source is received and modulated by the different areas of the spatial light modulator.

4 . The display system of claim 3 , wherein at least two of the light-emitting regions overlap on the light source.

5 . The display system of claim 3 , wherein the light-emitting regions are distinctly arranged on the light source without overlapping.

6 . The display system of claim 3 , wherein the at least one light emitter includes at least one light emitting diode.

7 . The display system of claim 3 , wherein at least one of the light-emitting regions is substantially rectangular in shape or substantially circular in shape.

8 . The display system of claim 3 , wherein at least two of the light-emitting regions are configured to emit light of different luminance.

9 . The display system of claim 3 , wherein one or more of the light emitters is configured to emit light of selectively variable intensity.

10 . The display system of claim 3 , wherein the light from different light-emitting regions of the light source is output at different angles from the spatial light modulator.

11 . The display system of claim 3 , further comprising a controller that is programmed to vary a size of at least one of the light-emitting regions.

12 . The display system of claim 1 , further comprising a lens structure between the light source and the spatial light modulator.

13 . The display system of claim 12 , wherein the lens structure is configured to convert spatial differences in the light emitted from the light source to angular differences in propagation of the light to the spatial light modulator.

14 . The display system of claim 1 , further comprising a polarizing beam splitter arranged to direct at least a portion of the light from the light source toward the spatial light modulator.

15 . The display system of claim 1 , wherein the spatial light modulator includes one or more of a liquid crystal on silicon panel, a transmissive liquid crystal panel, a digital light processing device, or a MEMs device.

16 . The display system of claim 1 , wherein the projection optics is further configured to collimate the light that is output at different angles from the spatial light modulator.

17 . The display system of claim 1 , further comprising a lenslet array between the spatial light modulator and the projection optics, wherein the lenslet array is configured to direct the light that is output from the different areas of the spatial light modulator at different angles to the projection optics.

18 . The display system of claim 1 , further comprising a prism between the spatial light modulator and the projection optics, wherein the prism is configured to direct the light that is output from the different areas of the spatial light modulator at different angles to the projection optics.

19 . The display system of claim 1 , wherein at least two of the different areas of the spatial light modulator are interleaved such that the at least two different areas share at least one pixel.

Assignments (2)
SECURITY INTEREST Recorded Oct 20, 2025
From: MAGIC LEAP, INC.; MENTOR ACQUISITION ONE, LLC; MOLECULAR IMPRINTS, INC.
To: CITIBANK, N.A., AS COLLATERAL AGENT
Reel/Frame 073031/0206 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jan 21, 2024
From: KLUG, MICHAEL ANTHONY; VAUGHN, MICHAL BEAU DENNISON; KONRAD, ROBERT; WETZSTEIN, GORDON; SCHOWENGERDT, BRIAN T.
To: MAGIC LEAP, INC.
Reel/Frame 066190/0461 →
Continuity (5)
Continuation 18168494 · Feb 13, 2023
Continuation 17581773 · Jan 21, 2022
Continuation 15789895 · Oct 20, 2017
Provisional Application 62411490 · Oct 21, 2016
Related Publication 20240045215A1 · Feb 8, 2024
References Cited (142)
US 5596339A · Furness, III et al. · 1997 [cited by applicant]
US 5943171A · Budd et al. · 1999 [cited by applicant]
US 6445406B1 · Taniguchi et al. · 2002 [cited by applicant]
US 6466185B2 · Sullivan et al. · 2002 [cited by applicant]
US 6525699B1 · Suyama et al. · 2003 [cited by applicant]
US 6590573B1 · Geshwind · 2003 [cited by applicant]
US 6850221B1 · Tickle · 2005 [cited by applicant]
US 8573784B2 · Yeh et al. · 2013 [cited by applicant]
US 8873149B2 · Bohn · 2014 [cited by examiner]
US 8950867B2 · Macnamara · 2015 [cited by applicant]
US 9081426B2 · Armstrong · 2015 [cited by applicant]
US 9215293B2 · Miller · 2015 [cited by applicant]
US 9310559B2 · Macnamara · 2016 [cited by applicant]
US 9348143B2 · Gao et al. · 2016 [cited by applicant]
US D758367S · Natsume · 2016 [cited by applicant]
US 9417452B2 · Schowengerdt et al. · 2016 [cited by applicant]
US 9470906B2 · Kaji et al. · 2016 [cited by applicant]
US 9547174B2 · Gao et al. · 2017 [cited by applicant]
US 9671566B2 · Abovitz et al. · 2017 [cited by applicant]
US 9740006B2 · Gao · 2017 [cited by applicant]
US 9791700B2 · Schowengerdt · 2017 [cited by applicant]
US 9841563B2 · Lapstun · 2017 [cited by applicant]
US 9851563B2 · Gao et al. · 2017 [cited by applicant]
US 9857591B2 · Welch et al. · 2018 [cited by applicant]
US 9874749B2 · Bradski et al. · 2018 [cited by applicant]
US 10031338B2 · Alexander et al. · 2018 [cited by applicant]
US 10106153B1 · Xiao et al. · 2018 [cited by applicant]
US 10444507B2 · Urey · 2019 [cited by examiner]
US 10455221B2 · Gronholm et al. · 2019 [cited by applicant]
US 10469833B2 · Hua et al. · 2019 [cited by applicant]
US 10831038B2 · Inaba et al. · 2020 [cited by applicant]
US 11073693B2 · Akutsu et al. · 2021 [cited by applicant]
US 11231544B2 · Lin et al. · 2022 [cited by applicant]
US 11231584B2 · Klug et al. · 2022 [cited by applicant]
US 11287657B2 · Klug · 2022 [cited by applicant]
US 20020051118A1 · Takagi et al. · 2002 [cited by applicant]
US 20020113752A1 · Sullivan et al. · 2002 [cited by applicant]
US 20030197933A1 · Sudo et al. · 2003 [cited by applicant]
US 20060028400A1 · Lapstun et al. · 2006 [cited by applicant]
US 20060028436A1 · Armstrong · 2006 [cited by applicant]
US 20070081123A1 · Lewis · 2007 [cited by applicant]
US 20080084544A1 · Hall et al. · 2008 [cited by applicant]
US 20090005961A1 · Grabowski et al. · 2009 [cited by applicant]
US 20110216383A1 · Haussler · 2011 [cited by examiner]
US 20120019908A1 · Choi et al. · 2012 [cited by applicant]
US 20120127062A1 · Bar-Zeev et al. · 2012 [cited by applicant]
US 20120162549A1 · Gao et al. · 2012 [cited by applicant]
US 20120306860A1 · Hatta et al. · 2012 [cited by applicant]
US 20130082922A1 · Miller · 2013 [cited by applicant]
US 20130117377A1 · Miller · 2013 [cited by applicant]
US 20130125027A1 · Abovitz · 2013 [cited by applicant]
US 20130126618A1 · Gao · 2013 [cited by applicant]
US 20130208234A1 · Lewis · 2013 [cited by applicant]
US 20130242262A1 · Lewis · 2013 [cited by applicant]
US 20130285885A1 · Nowatzyk et al. · 2013 [cited by applicant]
US 20130286053A1 · Fleck et al. · 2013 [cited by applicant]
US 20140063077A1 · Wetzstein et al. · 2014 [cited by applicant]
US 20140071539A1 · Gao · 2014 [cited by applicant]
US 20140177023A1 · Gao et al. · 2014 [cited by applicant]
US 20140218468A1 · Gao et al. · 2014 [cited by applicant]
US 20140267420A1 · Schowengerdt et al. · 2014 [cited by applicant]
US 20140306866A1 · Miller et al. · 2014 [cited by applicant]
US 20140368641A1 · Strandemar et al. · 2014 [cited by applicant]
US 20150016777A1 · Abovitz et al. · 2015 [cited by applicant]
US 20150103306A1 · Kaji et al. · 2015 [cited by applicant]
US 20150178939A1 · Bradski et al. · 2015 [cited by applicant]
US 20150205126A1 · Schowengerdt · 2015 [cited by applicant]
US 20150205259A1 · Kim et al. · 2015 [cited by applicant]
US 20150222883A1 · Welch · 2015 [cited by applicant]
US 20150222884A1 · Cheng · 2015 [cited by applicant]
US 20150234190A1 · Schowengerdt · 2015 [cited by applicant]
US 20150235418A1 · Schowengerdt · 2015 [cited by applicant]
US 20150235446A1 · Schowengerdt · 2015 [cited by applicant]
US 20150235448A1 · Schowengerdt · 2015 [cited by applicant]
US 20150260994A1 · Akutsu et al. · 2015 [cited by applicant]
US 20150268415A1 · Schowengerdt et al. · 2015 [cited by applicant]
US 20150302652A1 · Miller et al. · 2015 [cited by applicant]
US 20150309263A2 · Abovitz et al. · 2015 [cited by applicant]
US 20150326570A1 · Publicover et al. · 2015 [cited by applicant]
US 20150346490A1 · TeKolste et al. · 2015 [cited by applicant]
US 20150346495A1 · Welch · 2015 [cited by examiner]
US 20150370075A1 · Ato et al. · 2015 [cited by applicant]
US 20160011419A1 · Gao · 2016 [cited by applicant]
US 20160026253A1 · Bradski et al. · 2016 [cited by applicant]
US 20160116739A1 · TeKolste et al. · 2016 [cited by applicant]
US 20160139402A1 · Lapstun · 2016 [cited by applicant]
US 20160238845A1 · Alexander et al. · 2016 [cited by applicant]
US 20160270656A1 · Samec et al. · 2016 [cited by applicant]
US 20160320620A1 · Maimone · 2016 [cited by examiner]
US 20170068091A1 · Greenberg · 2017 [cited by examiner]
US 20170078652A1 · Hua et al. · 2017 [cited by applicant]
US 20170091917A1 · Bronstein et al. · 2017 [cited by applicant]
US 20170102545A1 · Hua et al. · 2017 [cited by applicant]
US 20180017801A1 · Chang et al. · 2018 [cited by applicant]
US 20180025640A1 · Micks et al. · 2018 [cited by applicant]
US 20180031838A1 · Browne · 2018 [cited by applicant]
US 20180038944A1 · Hellmig · 2018 [cited by examiner]
US 20180052331A1 · Inaba · 2018 [cited by applicant]
US 20180113311A1 · Klug et al. · 2018 [cited by applicant]
US 20190075252A1 · Zhao et al. · 2019 [cited by applicant]
US 20190253667A1 · Valli · 2019 [cited by applicant]
US 20200033803A1 · Christmas · 2020 [cited by applicant]
US 20200279483A1 · Eather et al. · 2020 [cited by applicant]
US 20200301147A1 · Klug · 2020 [cited by applicant]
US 20210096565A1 · Xie et al. · 2021 [cited by applicant]
US 20220146834A1 · Klug et al. · 2022 [cited by applicant]
US 20220221724A1 · Klug · 2022 [cited by applicant]
CN 104570366A · 2015 [cited by applicant]
CN 104749779A · 2017 [cited by applicant]
CN 106485233A · 2020 [cited by applicant]
JP H0991468A · 1997 [cited by applicant]
JP H11194299A · 1999 [cited by applicant]
JP 2002131694A · 2002 [cited by applicant]
JP 2002156599A · 2002 [cited by applicant]
JP 2002228978A · 2002 [cited by applicant]
JP 2001078234A · 2004 [cited by applicant]
JP 3722381B2 · 2005 [cited by examiner]
JP 2006287592A · 2006 [cited by applicant]
JP 2007033655A · 2007 [cited by applicant]
JP 2007163734A · 2007 [cited by applicant]
JP 2009020251A · 2009 [cited by applicant]
JP 2011244349A · 2011 [cited by applicant]
JP 2013161035A · 2013 [cited by applicant]
WO 2013049248A2 · 2013 [cited by applicant]
WO 2018075968A1 · 2018 [cited by applicant]
WO WO2018213727A1 · 2018 [cited by examiner]
WO 2020176783A1 · 2020 [cited by applicant]
ARToolKit: https://web.archive.org/web/20051013062315/http://www.hitl.washington.edu:80/artoolkit/documentation/hardware.htm, archived Oct. 13, 2005. [cited by applicant]
Azuma, “A Survey of Augmented Reality,” Teleoperators and Virtual Environments 6, 4 (Aug. 1997), pp. 355-385. https://web.archive.org/web/20010604100006/http://www.cs.unc.edu [cited by applicant]
Azuma, “Predictive Tracking for Augmented Realty,” TR95-007, Department of Computer Science, UNC-Chapel Hill, NC, Feb. 1995. [cited by applicant]
Bimber, et al., “Spatial Augmented Reality—Merging Real and Virtual Worlds,” 2005 https://web.media.mit.edu/raskar/book/BimberRaskarAugmentedRealityBook.pdf. [cited by applicant]
Huang F.-C. et al., “The Light Field Stereoscope—Immersive Computer Graphics via Factored Near-Eye Light Field displays with Focus Cues”, SIGGRAPH Emerging Technologies (Jul. 2015) in 12 pages. [cited by applicant]
International Preliminary Report on Patentability for PCT Application No. PCT/US2017/57730, dated Apr. 23, 2019. [cited by applicant]
International Search Report and Written Opinion for PCT Application No. PCT/US2017/057730, dated Dec. 27, 2017. [cited by applicant]
Jacob, “Eye Tracking in Advanced Interface Design,” Human-Computer Interaction Lab Naval Research Laboratory, Washington, D.C. / paper/ in Virtual Environments and Advanced Interface Design, ed. by W. Barfield and T.A. … [cited by applicant]
Kanebako, et al., “Time-multiplexing display module for high-density directional display,” SPIE—International Society for Optical Engineering, Proceedings, vol. 6803, Feb. 14, 2008. [cited by applicant]
Kim et al., “Full parallax multifocus three-dimensional display using a slanted light source array,” Optical Engineering, vol. 50, No. 11, Nov. 1, 2011. [cited by applicant]
Kim, et al., “Development of three types of multi-focus 3D display,” Proc. of SPIE vol. 8043, May 19, 2011. [cited by applicant]
Maimone, et al., “High Efficiency Near-Eye Light Field Display,” ACM Trans. Graph, article, Nov. 1, 2013. [cited by applicant]
Tanriverdi and Jacob, “Interacting With Eye Movements in Virtual Environments,” Department of Electrical Engineering and Computer Science, Tufts University, Medford, MA—paper/Proc. AMC CHI 2000 Human Factors in Computin… [cited by applicant]
Wetzstein G. et al., “Tensor Displays: Compressive Light Field Synthesis using Multilayer Displays with Directional Backlighting”, Association for Computing Machinery (Jul. 2012); 11 pages. [cited by applicant]
JP2024-112280 Office Action mailed May 16, 2025. [cited by applicant]