IP Library Granted Patent US 12,242,055
Granted Patent B2
US 12,242,055 · App. 17/984,120 · Granted Mar 4, 2025

Separated pupil optical systems for virtual and augmented reality and methods for displaying images using same

Inventors: Michael Anthony Klug (Austin, TX); Scott C. Cahall (Fairport, NY); Hyunsun Chung (Weston, FL)
Assignee: Magic Leap, Inc.
G02B27/0101G02B27/0172G02B27/0988G02B27/4205G02B30/24G03H1/2249H04N9/3102H04N23/56G02B2027/0105G02B2027/0125G02B2027/0127G02B2027/0134G02B27/1006G02B27/141G03H2001/2226G03H2001/2255G03H2001/2615
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,242,055
App. No.
17/984,120
Granted
Mar 4, 2025
Kind
B2
Abstract

A method of operating an AR system to display an image viewable by a user's eyes includes tracking, by an eye-tracking subsystem, a position of the user's eyes and determining, based on the position, a focus depth of the user's eyes. The method also includes selecting, from a plurality of light-guiding optical elements, a subset of light-guiding optical elements configured to focus light at a depth plane corresponding to the focus depth of the user's eyes, producing a plurality of light beams using a subset of sub-light sources of a plurality of sub-light sources, the subset of sub-light sources being configured to illuminate the subset of light-guiding optical elements, and imaging the plurality of light beams through an imaging system and onto the subset of light-guiding optical elements such that the image is generated at the depth plane corresponding to the focus depth of the user's eyes.

Claims (28)

1. A method of operating an AR system to display an image viewable by a user's eyes, the method comprising:

tracking, by an eye-tracking subsystem, positions of the user's eyes;

determining, based on the positions, a focus depth of the user's eyes;

providing a plurality of sub-light sources;

activating a subset of sub-light sources of the plurality of sub-light sources based on the focus depth of the user's eyes, wherein activating a sub-light source causes the sub-light source to produce a light beam originating from the sub-light source, thereby producing a plurality of light beams;

encoding the plurality of light beams with image data, thereby producing a plurality of encoded light beams;

providing a plurality of light-guiding optical elements, wherein each of a subset of light-guiding optical elements of the plurality of light-guiding optical elements corresponds to one sub-light source of the subset of sub-light sources;

focusing each of the plurality of encoded light beams onto a corresponding one of the subset of light-guiding optical elements of the plurality of light-guiding optical elements; and

projecting, from the subset of light-guiding optical elements that correspond to the subset of sub-light sources, the plurality of encoded light beams into the user's eyes so that they appear to originate from a depth plane corresponding to the focus depth of the user's eyes.

2. The method of claim 1 , wherein others of the plurality of light-guiding optical elements are not illuminated by the subset of sub-light sources.

3. The method of claim 1 , wherein each sub-light source of the plurality of sub-light sources is configured to produce multiple light beams and wherein each light beam of the multiple light beams is spatially separated.

4. The method of claim 1 , wherein encoding the plurality of light beams with the image data comprises using a spatial light modulator to provide the plurality of encoded light beams.

5. The method of claim 4 , wherein each light-guiding optical element of the subset of light-guiding optical elements comprises a respective in-coupling grating that admits the plurality of encoded light beams into the light-guiding optical element.

6. The method of claim 5 , wherein each respective in-coupling grating is arranged around an optical axis that extends from the spatial light modulator to the plurality of light-guiding optical elements.

7. The method of claim 5 , wherein admitting each of the plurality of encoded light beams at the respective in-coupling grating of the corresponding one of the subset of light-guiding optical elements of the plurality of light-guiding optical elements comprises encountering the respective in-coupling grating only once.

8. The method of claim 4 , wherein the plurality of sub-light sources are arranged around an optical axis that extends from the spatial light modulator to the plurality of sub-light sources.

9. The method of claim 4 , further comprising modifying a shape of a pupil formed by each of the plurality of encoded light beams using a mask.

10. The method of claim 9 , wherein the mask is adjacent to the plurality of light-guiding optical elements.

11. The method of claim 4 , further comprising modifying a size of a pupil formed by each of the plurality of encoded light beams using an optical element, wherein the optical element is adjacent to the plurality of light-guiding optical elements.

12. The method of claim 1 , wherein focusing each of the plurality of encoded light beams onto the corresponding one of the subset of light-guiding optical elements comprises using an injection optical system.

13. The method of claim 12 , wherein the injection optical system is characterized by an eccentric cross-section along an optical path of the injection optical system.

14. The method of claim 1 , wherein each beam of the plurality of light beams differs from other beams of the plurality of light beams in at least one light property.

15. The method of claim 14 , wherein the at least one light property comprises color.

16. The method of claim 14 , wherein the at least one light property comprises polarization.

17. The method of claim 1 , wherein each of the plurality of sub-light sources are spatially separated from each other.

18. The method of claim 1 , further comprising increasing a numerical aperture of the plurality of sub-light sources using a pupil expander.

19. The method of claim 18 , wherein the pupil expander comprises a film having a prism pattern disposed thereon.

20. The method of claim 1 , wherein activating the subset of sub-light sources comprises deactivating remaining sub-light sources of the plurality of sub-light sources.

Assignments (3)
SECURITY INTEREST Recorded Oct 31, 2025
From: MAGIC LEAP, INC.; MENTOR ACQUISITION ONE, LLC; MOLECULAR IMPRINTS, INC.
To: CITIBANK, N.A., AS COLLATERAL AGENT
Reel/Frame 073422/0549 →
SECURITY INTEREST Recorded Feb 7, 2023
From: MAGIC LEAP, INC.; MENTOR ACQUISITION ONE, LLC; MOLECULAR IMPRINTS, INC.
To: CITIBANK, N.A., AS COLLATERAL AGENT
Reel/Frame 062681/0065 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Nov 10, 2022
From: KLUG, MICHAEL ANTHONY; CAHALL, SCOTT C.; CHUNG, HYUNSUN
To: MAGIC LEAP, INC.
Reel/Frame 061719/0670 →
Continuity (4)
Continuation 16882011 · May 22, 2020
Continuation 15146296 · May 4, 2016
Provisional Application 62156809 · May 4, 2015
Related Publication 20230062857A1 · Mar 2, 2023
References Cited (89)
US 5742262A · Tabata et al. · 1998 [cited by applicant]
US 6215590B1 · Okano · 2001 [cited by examiner]
US 6646716B1 · Ramanujan et al. · 2003 [cited by applicant]
US 6648475B1 · Roddy et al. · 2003 [cited by applicant]
US 6700552B2 · Kollin et al. · 2004 [cited by applicant]
US 7180837B2 · Yi et al. · 2007 [cited by applicant]
US 7205960B2 · David · 2007 [cited by applicant]
US 7573640B2 · Nivon et al. · 2009 [cited by applicant]
US 7719767B2 · Lin et al. · 2010 [cited by applicant]
US 8254031B2 · Levola · 2012 [cited by applicant]
US 8649099B2 · Schultz et al. · 2014 [cited by applicant]
US 8913324B2 · Schrader · 2014 [cited by applicant]
US 8949099B1 · Lai et al. · 2015 [cited by applicant]
US 8965152B2 · Simmonds · 2015 [cited by applicant]
US 11402629B2 · Klug et al. · 2022 [cited by applicant]
US 11526007B2 · Klug et al. · 2022 [cited by applicant]
US 20040109135A1 · Watanabe et al. · 2004 [cited by applicant]
US 20040174348A1 · David · 2004 [cited by applicant]
US 20060250580A1 · Silverstein et al. · 2006 [cited by applicant]
US 20090009998A1 · Malstrom · 2009 [cited by examiner]
US 20090190095A1 · Ellinger et al. · 2009 [cited by applicant]
US 20100079865A1 · Saarikko et al. · 2010 [cited by applicant]
US 20110019258A1 · Levola · 2011 [cited by examiner]
US 20110157667A1 · Lacoste et al. · 2011 [cited by applicant]
US 20110205514A1 · Kita · 2011 [cited by applicant]
US 20110242661A1 · Simmonds · 2011 [cited by examiner]
US 20110261174A1 · Silverstein et al. · 2011 [cited by applicant]
US 20120154557A1 · Perez · 2012 [cited by examiner]
US 20130088780A1 · Jarvenpaa · 2013 [cited by examiner]
US 20130322810A1 · Robbins · 2013 [cited by examiner]
US 20140035959A1 · Lapstun · 2014 [cited by applicant]
US 20140043688A1 · Schrader · 2014 [cited by applicant]
US 20140140653A1 · Brown et al. · 2014 [cited by applicant]
US 20140192418A1 · Suzuki · 2014 [cited by applicant]
US 20160327789A1 · Klug et al. · 2016 [cited by applicant]
US 20160377866A1 · Alexander et al. · 2016 [cited by applicant]
US 20180267309A1 · Klug et al. · 2018 [cited by applicant]
US 20200355918A1 · Klug et al. · 2020 [cited by applicant]
JP 0384516A · 1991 [cited by applicant]
JP 08166559A · 1996 [cited by applicant]
JP 10262165A · 1998 [cited by applicant]
JP 2007033669A · 2007 [cited by applicant]
JP 2009186794A · 2009 [cited by applicant]
JP 2014224846A · 2014 [cited by applicant]
WO 2013188464A1 · 2013 [cited by applicant]
WO 2014155288A2 · 2014 [cited by applicant]
U.S. Appl. No. 15/146,296, Final Office Action mailed on May 28, 2021, 11 pages. [cited by applicant]
U.S. Appl. No. 15/146,296, Final Office Action mailed on Feb. 26, 2020, 12 pages. [cited by applicant]
U.S. Appl. No. 15/146,296, Final Office Action mailed on Jan. 24, 2018, 12 pages. [cited by applicant]
U.S. Appl. No. 15/146,296, Final Office Action mailed on Dec. 14, 2018, 15 pages. [cited by applicant]
U.S. Appl. No. 15/146,296, Non-Final Office Action mailed on Feb. 19, 2021, 10 pages. [cited by applicant]
U.S. Appl. No. 15/146,296, Non-Final Office Action mailed on Jan. 24, 2022, 13 pages. [cited by applicant]
U.S. Appl. No. 15/146,296, Non-Final Office Action mailed on Jun. 22, 2018, 13 pages. [cited by applicant]
U.S. Appl. No. 15/146,296, Non-Final Office Action mailed on Oct. 8, 2019, 14 pages. [cited by applicant]
U.S. Appl. No. 15/146,296, Non-Final Office Action mailed on Sep. 15, 2017, 9 pages. [cited by applicant]
U.S. Appl. No. 15/146,296, Notice of Allowance mailed on May 3, 2022, 9 pages. [cited by applicant]
U.S. Appl. No. 15/988,255, Non-Final Office Action mailed on Oct. 22, 2019, 9 pages. [cited by applicant]
U.S. Appl. No. 15/988,255, Non-Final Office Action mailed on Jan. 22, 2020, 9 pages. [cited by applicant]
U.S. Appl. No. 16/882,011, Final Office Action mailed on May 6, 2022, 7 pages. [cited by applicant]
U.S. Appl. No. 16/882,011, Non-Final Office Action mailed on Jan. 31, 2022, 12 pages. [cited by applicant]
U.S. Appl. No. 16/882,011, Notice of Allowance mailed on Aug. 12, 2022, 8 pages. [cited by applicant]
Australian Application No. 2016258618, First Examination Report mailed on Jul. 16, 2020, 4 pages. [cited by applicant]
Australian Application No. 2016258618, Notice of Acceptance mailed on Apr. 23, 2021, 3 pages. [cited by applicant]
Canadian Application No. 2981107, Office Action mailed on May 31, 2022, 3 pages. [cited by applicant]
Chinese Application No. 201680025736.1, Office Action mailed on Jul. 3, 2020, 6 pages (5 pages of Original Document and 1 pages of English Translation). [cited by applicant]
Chinese Application No. 202110217120.7, Office Action mailed on Jul. 27, 2022, 12 pages (7 pages of Original Document and 5 pages of English Translation). [cited by applicant]
European Application No. 16789991.3, Extended European Search Report mailed on Apr. 4, 2018, 10 pages. [cited by applicant]
European Application No. 16789991.3, Office Action mailed on May 15, 2020, 5 pages. [cited by applicant]
European Application No. 16789991.3, Office Action mailed on Jul. 19, 2019, 6 pages. [cited by applicant]
Israel Application No. 254886, Office Action mailed on Mar. 4, 2021, 6 pages (3 pages of Original Document and 3 pages of English Translation). [cited by applicant]
Indian Application No. 201747040489, First Examination Report mailed on Dec. 16, 2020, 6 pages. [cited by applicant]
Indian Application No. 202148026514, First Examination Report mailed on Mar. 29, 2022, 6 pages. [cited by applicant]
Japanese Application No. 2017-557194, Office Action mailed on Feb. 19, 2020, 9 pages (4 pages of Original Document and 5 pages of English Translation). [cited by applicant]
Japanese Application No. 2020-219772, Office Action mailed on Nov. 26, 2021, 5 pages (2 pages of Original Document and 3 pages of English Translation). [cited by applicant]
Korean Application No. 10-2017-7034746, Office Action mailed on Jan. 18, 2023, 12 pages. [cited by applicant]
New Zealand Application No. 735943, First Examination Report mailed on Jul. 22, 2020, 5 pages. [cited by applicant]
New Zealand Application No. 735943, Second Examination Report mailed on Feb. 18, 2021, 2 pages. [cited by applicant]
New Zealand Application No. 735943, Third Examination Report mailed on Mar. 3, 2021, 1 page. [cited by applicant]
New Zealand Application No. 775650, First Examination Report mailed on Jun. 14, 2022, 2 pages. [cited by applicant]
International Application No. PCT/US2016/030717, International Preliminary Report on Patentability mailed on Nov. 16, 2017, 12 pages. [cited by applicant]
International Application No. PCT/US2016/030717, International Search Report and Written Opinion mailed on Aug. 5, 2016, 12 pages. [cited by applicant]
Application No. CN202110217120.7 , “Office Action”, Mar. 31, 2023, 7 pages. [no translation available]. [cited by applicant]
Application No. KR10-2017-7034746 , “Notice of Decision to Grant” and English translation, Jun. 16, 2023, 4 pages. [cited by applicant]
Application No. NZ775650 , “Second Examination Report”, Apr. 5, 2023, 1 page. [cited by applicant]
Canadian Patent Application No. 2981107, “Office Action”, Feb. 7, 2023, 4 pages. [cited by applicant]
Canadian Patent Application No. 2981107, “Office Action”, Sep. 29, 2023, 4 pages. [cited by applicant]
Chinese Patent Application No. 202110217120.7, “Office Action”, Sep. 9, 2023, 5 pages. [no translation available]. [cited by applicant]
Korean Patent Application No. 10-2023-7031506, “Office Action” and English translation, Oct. 17, 2023, 6 pages. [cited by applicant]
KR10-2023-7031506, “Notice of Decision to Grant” and English translation, Jun. 18, 2024, 4 pages. [cited by applicant]