IP Library Granted Patent US 12,277,658
Granted Patent B2
US 12,277,658 · App. 18/431,758 · Granted Apr 15, 2025

Systems and methods for mixed reality

Inventors: Brian T. Schowengerdt (Seattle, WA); Mathew D. Watson (Bellevue, WA); David Tinch (Laramie, WY); Ivan Li Chuen Yeoh (Tampa, FL); John Graham Macnamara (Plantation, FL); Lionel Ernest Edwin (Plantation, FL); Michael Anthony Klug (Austin, TX); William Hudson Welch (Fort Lauderdale, FL)
Assignee: Magic Leap, Inc.
G06T19/006G02B5/18G02B26/08G02B27/0172G02B27/0944G02B27/0955G02B27/0972G02B27/10G02B27/106G02B27/30G02B27/42G02B2006/1215G02B2027/0123G02B2027/0178
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,277,658
App. No.
18/431,758
Granted
Apr 15, 2025
Kind
B2
Abstract

A virtual image generation system comprises a planar optical waveguide having opposing first and second faces, an in-coupling (IC) element configured for optically coupling a collimated light beam from an image projection assembly into the planar optical waveguide as an in-coupled light beam, a first orthogonal pupil expansion (OPE) element associated with the first face of the planar optical waveguide for splitting the in-coupled light beam into a first set of orthogonal light beamlets, a second orthogonal pupil expansion (OPE) element associated with the second face of the planar optical waveguide for splitting the in-coupled light beam into a second set of orthogonal light beamlets, and an exit pupil expansion (EPE) element associated with the planar optical waveguide for splitting the first and second sets of orthogonal light beamlets into an array of out-coupled light beamlets that exit the planar optical waveguide.

Claims (33)

1. A virtual image generation system comprising:

a planar optical waveguide comprising a plurality of substrates including a primary substrate having a first thickness and at least two secondary substrates having second thicknesses, and at least two semi-reflective interfaces respectively disposed between the substrates;

a pre-pupil expansion (PPE) element configured for using one or more beam-multiplication techniques to pre-expand an entrance pupil of a collimated light beam from an image projection assembly;

an in-coupling (IC) element configured for optically coupling the collimated light beam from the PPE, for propagation as an in-coupled light beam within the planar optical waveguide, wherein the at least two semi-reflective interfaces are each configured for splitting the in-coupled light beam into a plurality of primary light beamlets that propagate within the primary substrate; and

one or more diffractive optical elements (DOEs) associated with the planar optical waveguide for further splitting the plurality of primary light beamlets into an array of out-coupled light beamlets that exit a face of the planar optical waveguide,

wherein the substrates are optically transparent materials and each of the at least two semi-reflective interfaces comprises a semi-reflective coating.

2. The virtual image generation system of claim 1 , wherein the at least two secondary substrates comprise two secondary substrates disposed adjacent one another, each of the second thicknesses is less than the first thickness and further comprising a collimating element disposed between the image projection assembly and the PPE.

3. The virtual image generation system of claim 1 , wherein the at least two secondary substrates comprise two secondary substrates disposed adjacent one another and the first thickness is at least twice each of the second thicknesses.

4. The virtual image generation system of claim 1 , wherein the at least two secondary substrates comprise two secondary substrates disposed adjacent one another and the second thicknesses are substantially equal to each other.

5. The virtual image generation system of claim 1 , wherein the at least two secondary substrates comprise two secondary substrates disposed adjacent one another and the first thickness is a non-multiple of at least one of the second thicknesses.

6. The virtual image generation system of claim 1 , wherein two or more of the at least two secondary substrates have second thicknesses that are not substantially equal to each other.

7. The virtual image generation system of claim 1 , wherein the first thickness and the second thicknesses are selected, such that spacings between centers of at least two adjacent ones of the out-coupled light beamlets are equal to or less than a width of the collimated light beam.

8. The virtual image generation system of claim 1 , wherein the first thickness and the second thicknesses are selected, such that no gap resides between edges of greater than half of adjacent ones of the out-coupled light beamlets.

9. The virtual image generation system of claim 1 , wherein

the at least two semi-reflective coatings are respectively disposed between the substrates via one of physical vapor deposition (PVD), ion-assisted deposition (IAD), and ion beam sputtering (IBS).

10. The virtual image generation system of claim 9 , wherein each of the at least two semi-reflective coatings is composed of one or more of a metal (Au, Al, Ag, Ni—Cr, Cr and so on), dielectric (Oxides, Fluorides and Sulfides), and semiconductors (Si, Ge).

11. The virtual image generation system of claim 1 , wherein adjacent ones of the plurality of substrates are composed of materials having different indices of refraction.

12. The virtual image generation system of claim 1 , wherein the collimated light beam defines an entrance pupil, and the out-coupled light beamlet array defines an exit pupil larger than the entrance pupil.

13. The virtual image generation system of claim 1 , wherein the out-coupled light beamlet array is a two-dimensional out-coupled light beamlet array.

14. The virtual image generation system of claim 1 , further comprising:

a display subsystem having an image projection assembly configured for generating the collimated light beam,

wherein the image projection assembly comprises a scanning device configured for scanning the collimated light beam.

15. A virtual image generation system comprising:

a planar optical waveguide comprising a plurality of substrates including a primary substrate having a first thickness and at least two secondary substrates having second thicknesses, and at least two semi-reflective interfaces respectively disposed between the substrates;

an in-coupling (IC) element configured for optically coupling a collimated light beam from an image projection assembly for propagation as an in-coupled light beam within the planar optical waveguide, wherein the at least two semi-reflective interfaces are each configured for splitting the in-coupled light beam into a plurality of primary light beamlets that propagate within the primary substrate; and

one or more diffractive optical elements (DOEs) associated with the planar optical waveguide for further splitting the plurality of primary light beamlets into an array of out-coupled light beamlets that exit a face of the planar optical waveguide,

wherein the at least two semi-reflective interfaces are configured for splitting the in-coupled light beam into at least two in-coupled light beamlets, wherein the one or more DOEs comprises an orthogonal pupil expansion (OPE) element configured for respectively splitting the at least two in-coupled light beamlets into at least two sets of orthogonal light beamlets, the at least two semi-reflective interfaces are further configured for splitting the at least two sets of orthogonal light beamlets into at least four sets of orthogonal light beamlets, wherein the one or more DOEs comprises an exit pupil expansion (EPE) element configured for splitting the at least four sets of orthogonal light beamlets into the set of out-coupled light beamlets,

and wherein the substrates are optically transparent materials and each of the at least two semi-reflective interfaces comprises a semi-reflective coating.

16. The virtual image generation system of claim 15 , wherein the OPE element and EPE element are disposed on a face of the optical planar waveguide.

17. The virtual image generation system of claim 15 , wherein the at least two in-coupled light beamlets propagate within the planar optical waveguide via total internal reflection (TIR) along a first optical path that intersects the OPE element, such that portions of the at least two in-coupled light beamlets are diffracted as the at least two sets of orthogonal light beamlets that propagate within the planar optical waveguide via TIR along second parallel optical paths.

18. The virtual image generation system of claim 17 , wherein the second parallel optical paths are orthogonal to the first optical path.

19. The virtual image generation system of claim 17 , wherein the at least two sets of orthogonal light beamlets intersect the EPE element, such that portions of the at least two sets of orthogonal light beamlets are diffracted as the out-coupled set of light beamlets out of the face of the planar optical waveguide.

20. The virtual image generation system of claim 15 , wherein the EPE element is configured for imparting a convex wavefront profile on the out-coupled light beamlet array exiting the planar optical waveguide, the convex wavefront profile having a center of radius at a focal point to produce an image at a given focal plane.

Assignments (4)
SECURITY INTEREST Recorded Oct 31, 2025
From: MAGIC LEAP, INC.; MENTOR ACQUISITION ONE, LLC; MOLECULAR IMPRINTS, INC.
To: CITIBANK, N.A., AS COLLATERAL AGENT
Reel/Frame 073422/0549 →
SECURITY INTEREST Recorded Oct 28, 2025
From: MAGIC LEAP, INC.; MENTOR ACQUISITION ONE, LLC; MOLECULAR IMPRINTS, INC.
To: CITIBANK, N.A., AS COLLATERAL AGENT
Reel/Frame 073388/0027 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Aug 5, 2025
From: SCHOWENGERDT, BRIAN T.; WATSON, MATHEW D.
To: MAGIC LEAP, INC.
Reel/Frame 071941/0540 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Aug 5, 2025
From: MACNAMARA, JOHN GRAHAM; EDWIN, LIONEL ERNEST; YEOH, IVAN LI-CHUEN; SCHOWENGERDT, BRIAN T.; WATSON, MATHEW D.; WELCH, WILLIAM HUDSON; KLUG, MICHAEL ANTHONY; TINCH, DAVID ALAN
To: MAGIC LEAP, INC.
Reel/Frame 071941/0674 →
Continuity (7)
Division 18296141 · Apr 5, 2023
Continuation 17389684 · Jul 30, 2021
Continuation 16903279 · Jun 16, 2020
Continuation 15980947 · May 16, 2018
Provisional Application 62509499 · May 22, 2017
Provisional Application 62506841 · May 16, 2017
Related Publication 20240177429A1 · May 30, 2024
References Cited (107)
US 6897992B2 · Kikuchi · 2005 [cited by applicant]
US 9417452B2 · Schowengerdt et al. · 2016 [cited by applicant]
US 9612403B2 · Abovitz et al. · 2017 [cited by applicant]
US 9671566B2 · Abovitz et al. · 2017 [cited by applicant]
US 9791700B2 · Schowengerdt · 2017 [cited by applicant]
US 9857591B2 · Welch et al. · 2018 [cited by applicant]
US 10755481B2 · Schowengerdt et al. · 2020 [cited by applicant]
US 11107288B2 · Schowengerdt et al. · 2021 [cited by applicant]
US 11651566B2 · Schowengerdt et al. · 2023 [cited by applicant]
US 11935206B2 · Schowengerdt · 2024 [cited by examiner]
US 20040257664A1 · Hashimoto et al. · 2004 [cited by applicant]
US 20090190222A1 · Simmonds et al. · 2009 [cited by applicant]
US 20110026128A1 · Baker et al. · 2011 [cited by applicant]
US 20120062998A1 · Schultz et al. · 2012 [cited by applicant]
US 20120176682A1 · Dejong · 2012 [cited by applicant]
US 20130329301A1 · Travis · 2013 [cited by applicant]
US 20140111865A1 · Kobayashi · 2014 [cited by applicant]
US 20140192418A1 · Suzuki · 2014 [cited by applicant]
US 20140334777A1 · Dubroca et al. · 2014 [cited by applicant]
US 20150153569A1 · Yonekubo · 2015 [cited by examiner]
US 20150235471A1 · Schowengerdt · 2015 [cited by applicant]
US 20150241844A1 · Horikawa · 2015 [cited by applicant]
US 20150309264A1 · Abovitz et al. · 2015 [cited by applicant]
US 20150346490A1 · Tekolste et al. · 2015 [cited by applicant]
US 20160011419A1 · Gao et al. · 2016 [cited by applicant]
US 20160077338A1 · Robbins et al. · 2016 [cited by applicant]
US 20160124228A1 · Bahmandar et al. · 2016 [cited by applicant]
US 20160178910A1 · Giudicelli et al. · 2016 [cited by applicant]
US 20170010466A1 · Klug et al. · 2017 [cited by applicant]
US 20170038579A1 · Yeoh et al. · 2017 [cited by applicant]
US 20170097506A1 · Schowengerdt et al. · 2017 [cited by applicant]
US 20170131545A1 · Wall et al. · 2017 [cited by applicant]
US 20170293141A1 · Schowengerdt et al. · 2017 [cited by applicant]
US 20170299860A1 · Wall et al. · 2017 [cited by applicant]
US 20180149873A1 · Edwin et al. · 2018 [cited by applicant]
AU 2023201026B2 · 2023 [cited by applicant]
CN 106537219 · 2017 [cited by applicant]
JP 2006276198 · 2006 [cited by applicant]
JP 2007219106 · 2007 [cited by applicant]
JP 2013061480 · 2013 [cited by applicant]
JP 2014052408 · 2014 [cited by applicant]
JP 2014085425 · 2014 [cited by applicant]
JP 2014132328 · 2014 [cited by applicant]
JP 2015106105 · 2015 [cited by applicant]
JP 2015191032 · 2015 [cited by applicant]
JP 2017090561 · 2017 [cited by applicant]
KR 1020120014597 · 2012 [cited by applicant]
KR 1020150112852 · 2015 [cited by applicant]
KR 102205000 · 2021 [cited by applicant]
WO WO2016105285 · 2016 [cited by applicant]
WO WO2017044873 · 2017 [cited by applicant]
PCT Invitation to Pay Additional Fees and partial search report mailed Jul. 11, 2018 for PCT application No. PCT/US2018/032883, Applicant Magic Leap, Inc., 2 pages. [cited by applicant]
PCT International Search Report and Written Opinion for PCT Application No. PCT/US2018/032883, Applicant Magic Leap, Inc., dated Sep. 25, 2018 (41 pages). [cited by applicant]
Extended European Search Report for EP Patent Appln. No. 18802379.1 dated May 27, 2020. [cited by applicant]
Non-Final Office Action for U.S. Appl. No. 15/980,947 dated Aug. 21, 2019. [cited by applicant]
Amendment Response to NFOA for U.S. Appl. No. 15/980,947 dated Feb. 20, 2020. [cited by applicant]
Notice of Allowance for U.S. Appl. No. 15/980,947 dated Mar. 16, 2020. [cited by applicant]
Notice of Allowance for U.S. Appl. No. 15/980,947 dated Jul. 2, 2020. [cited by applicant]
Notice of Allowance for U.S. Appl. No. 16/903,279 dated Apr. 27, 2021. [cited by applicant]
Non-Final Office Action for U.S. Appl. No. 16/903,279 dated Dec. 1, 2020. [cited by applicant]
Amendment Response to NFOA for U.S. Appl. No. 16/903,279 dated Mar. 1, 2021. [cited by applicant]
Foreign Response for CN Patent Appln. No. 201880034011.8 dated Apr. 8, 2021. [cited by applicant]
Foreign OA for KR Patent Appln. No. 10-2019-7036766 dated Jul. 9, 2021. [cited by applicant]
Foreign OA for CN Patent Appln. No. 201880034011.8 dated May 26, 2021. [cited by applicant]
Foreign Response for CN Patent Appln. No. 201880034011.8 dated Aug. 9, 2021. [cited by applicant]
Foreign OA for JP Patent Appln. No. 2019-563122 dated Aug. 24, 2021. [cited by applicant]
Foreign NOA for CN Patent Appln. No. 201880034011.8 dated Aug. 31, 2021. [cited by applicant]
Foreign Response for KR Patent Appln. No. 10-2019-7036766 dated Sep. 8, 2021. [cited by applicant]
Foreign Response for JP Patent Appln. No. 2019-563122 dated Nov. 16, 2021. [cited by applicant]
Foreign FOA for JP Patent Appln. No. 2019-563122 dated Jan. 6, 2022. [cited by applicant]
Foreign Exam Report for IN Patent Appln. No. 201947049345 dated Apr. 11, 2022 In English. [cited by applicant]
Foreign NOA for JP Patent Appln. No. 2019-563122 dated Jun. 10, 2022. [cited by applicant]
Foreign OA for CN Patent Appln. No. 202111339558.9 dated Jun. 10, 2022 with English translation. [cited by applicant]
Non-Final Office Action for U.S. Appl. No. 17/389,684 dated Sep. 8, 2022. [cited by applicant]
Foreign Response for AU Patent Appln. No. 2018270948 dated Oct. 20, 2022. [cited by applicant]
Foreign Response for CN Patent Appln. No. 202111339558.9 dated Oct. 24, 2022. [cited by applicant]
Foreign OA for CN Patent Appln. No. 202111339558.9 dated Nov. 3, 2022 (with English translation). [cited by applicant]
Foreign Exam Report for EP Patent Appln. No. 18803279.1 dated Oct. 27, 2022. [cited by applicant]
Amendment Response to NFOA for U.S. Appl. No. 17/389,684 dated Dec. 7, 2022. [cited by applicant]
Notice of Allowance for U.S. Appl. No. 17/389,684 dated Jan. 9, 2023. [cited by applicant]
Foreign Response for CN Patent Appln. No. 202111339558.9 dated Jan. 3, 2023. [cited by applicant]
Foreign OA for KR Patent Appln. No. 10-2022-7005055 dated Jan. 11, 2023. [cited by applicant]
Foreign Response for EP Patent Appln. No. 18803279.1 dated Feb. 22, 2023. [cited by applicant]
Foreign Response for KR Patent Appln. No. 10-2022-7005055 dated Mar. 8, 2023. [cited by applicant]
Foreign Exam Report for AU Patent Appln. No. 2023201026 dated Apr. 3, 2023. [cited by applicant]
Foreign Decision of Rejection for CN Patent Appln. No. 202111339558.9 dated Mar. 1, 2023 (with English translation). [cited by applicant]
Foreign Response for CN Patent Appln. No. 202111339558.9 dated May 26, 2023. [cited by applicant]
Foreign Response for CN Patent Appln. No. 202111313475.2 dated May 22, 2023. [cited by applicant]
Foreign OA for JP Patent Appln. No. 2022-72295 dated May 29, 2023. [cited by applicant]
Foreign NOA for KR Patent Appln. No. 10-2022-7005055 dated Jul. 3, 2023. [cited by applicant]
Foreign Response for JP Patent Appln. No. 2022-72295 dated Aug. 25, 2023. [cited by applicant]
Foreign Response for AU Patent Appln. No. 2023201026 dated Oct. 30, 2023. [cited by applicant]
Notice of Allowance for U.S. Appl. No. 18/296,141 dated Nov. 3, 2023. [cited by applicant]
Foreign OA for JP Patent Appln. No. 2022-72295 dated Oct. 16, 2023 (with English translation). [cited by applicant]
Foreign NOA for AU Patent Appln. No. 2023201026 dated Nov. 8, 2023. [cited by applicant]
Foreign Response for JP Patent Appln. No. 2022-72295 dated Jan. 19, 2024. [cited by applicant]
Foreign OA for KR Patent Appln. No. 10-2023-7033436 dated Jan. 16, 2024. [cited by applicant]
Foreign NOA for IL Patent Appln. No. 2686755 dated Aug. 6, 2023 in English. [cited by applicant]
Foreign Hearing Notice for IN Patent Appln. No. 201947049345 dated Mar. 5, 2024 in English. [cited by applicant]
Foreign Response for KR Patent Appln. No. 10-2023-7033436 dated Mar. 14, 2024. [cited by applicant]
Foreign Refusal Order for IN Patent Appln. No. 201947049345 dated Mar. 22, 2024 in English. [cited by applicant]
Foreign NOA for JP Patent Appln. No. 2022-72295 dated Apr. 2, 2024. [cited by applicant]
Foreign NOA for IL Patent Appln. No. 308323 dated Mach 12, 2024 in English. [cited by applicant]
Foreign NOA for KR Patent Appln. No. 10-2023-7033436 dated Apr. 11, 2024. [cited by applicant]
Foreign NOA for KR Patent Appln. No. 10-2024-7021428 dated Oct. 14, 2024. [cited by applicant]
Foreign NOA for EP Patent Appln. No. 18803279.1 dated Jan. 15, 2025. [cited by applicant]
Foreign OA for JP Patent Appln. No. 2024-74384 dated Feb. 12, 2025 (with English translation). [cited by applicant]