IP Library Granted Patent US 12,287,189
Granted Patent B2
US 12,287,189 · App. 18/400,476 · Granted Apr 29, 2025

Light field display metrology

Inventors: Ivan Li Chuen Yeoh (Wesley Chapel, FL); Lionel Ernest Edwin (Hollywood, FL); Samuel A. Miller (Hollywood, FL)
Assignee: Magic Leap, Inc.
G01B11/14G01B11/22G02B27/01G02B27/017G06F3/013G06T15/20G06T19/006G09G3/003G09G3/006G09G5/02H04N13/144H04N13/327H04N13/344H04N13/383H04N13/398G02B2027/011G02B2027/0178G06F2203/011G06T3/20G06T3/40G06T3/60G06T2207/10012G06T2207/10052G09G3/2092G09G3/3406G09G2320/0233G09G2320/0242G09G2320/0276G09G2320/028G09G2320/029G09G2320/0626G09G2320/0666G09G2320/0673G09G2320/0693G09G2340/0464G09G2360/16H04N13/395
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,287,189
App. No.
18/400,476
Granted
Apr 29, 2025
Kind
B2
Abstract

Examples of a light field metrology system for use with a display are disclosed. The light field metrology may capture images of a projected light field, and determine focus depths (or lateral focus positions) for various regions of the light field using the captured images. The determined focus depths (or lateral positions) may then be compared with intended focus depths (or lateral positions), to quantify the imperfections of the display. Based on the measured imperfections, an appropriate error correction may be performed on the light field to correct for the measured imperfections. The display can be an optical display element in a head mounted display, for example, an optical display element capable of generating multiple depth planes or a light field display.

Claims (17)

1. An optical metrology system for image correction of a display, the system comprising:

at least one camera configured to capture one or more images of virtual content being projected by the display, wherein the display is configured to present the virtual content at a plurality of depth planes; and

a hardware processor programmed with executable instructions to:

access the one or more images captured by the at least one camera;

determine, based at least partly on the one or more images, at least one value indicating a luminance of each of a plurality of points on the display;

calculate, based at least partly on said determined values, at least one correction for the display, wherein the at least one correction includes a chromatic balancing correction; and

store the at least one correction in memory that is associated with the display,

wherein the display comprises a plurality of display layers, and

wherein to calculate the chromatic balancing correction, the hardware processor is programmed to:

identify a color cluster associated with at least one of the plurality of display layers, the color cluster comprising at least one additional display layer;

for different points of the at least one display layer, compare the value associated with luminance corresponding to the point on the at least one display layer with a value associated with luminance corresponding to the point on the at least one additional display layer; and

calculate an amount that lowers different values associated with luminance of the at least one display layer to a lower value associated with its corresponding point of the at least one additional layer.

2. The optical metrology system of claim 1 , wherein at least one of the display layers comprises a depth layer configured to present at least a portion of the virtual content at one of the plurality of depth planes.

3. The optical metrology system of claim 1 , wherein at least one of the display layers comprises a color layer configured to present at least a portion of the virtual content formed of light that is within a particular range of wavelengths.

4. The optical metrology system of claim 1 , wherein each of the display layers comprises one or more waveguides.

5. The optical metrology system of claim 1 , wherein the at least one camera comprises a light field camera or a digital camera having a small depth of focus.

6. The optical metrology system of claim 1 , wherein the virtual content comprises at least one of a checkerboard pattern, a geometric pattern, or a stochastic pattern.

Assignments (3)
SECURITY INTEREST Recorded Oct 31, 2025
From: MAGIC LEAP, INC.; MENTOR ACQUISITION ONE, LLC; MOLECULAR IMPRINTS, INC.
To: CITIBANK, N.A., AS COLLATERAL AGENT
Reel/Frame 073439/0168 →
SECURITY INTEREST Recorded Oct 28, 2025
From: MAGIC LEAP, INC.; MENTOR ACQUISITION ONE, LLC; MOLECULAR IMPRINTS, INC.
To: CITIBANK, N.A., AS COLLATERAL AGENT
Reel/Frame 073388/0027 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Dec 29, 2023
From: YEO, IVAN L.; EDWIN, LIONEL E.; MILLER, SAMUEL A.
To: MAGIC LEAP, INC.
Reel/Frame 065982/0614 →
Continuity (10)
Continuation 17994882 · Nov 28, 2020
Continuation 17087359 · Nov 2, 2020
Continuation 16457469 · Jun 28, 2019
Division 15341760 · Nov 2, 2016
Provisional Application 62278779 · Jan 14, 2016
Provisional Application 62278824 · Jan 14, 2016
Provisional Application 62278794 · Jan 14, 2016
Provisional Application 62250934 · Nov 4, 2015
Provisional Application 62250925 · Nov 4, 2015
Related Publication 20240142224A1 · May 2, 2024
References Cited (160)
US 5594563A · Larson · 1997 [cited by applicant]
US 6011581A · Swift et al. · 2000 [cited by applicant]
US 6326996B1 · Brabander · 2001 [cited by examiner]
US 6456339B1 · Surati et al. · 2002 [cited by applicant]
US 6816625B2 · Lewis, Jr. et al. · 2004 [cited by applicant]
US 6850221B1 · Tickle · 2005 [cited by applicant]
US D514570S · Ohta · 2006 [cited by applicant]
US 7414791B2 · Urakawa et al. · 2008 [cited by applicant]
US 8406562B2 · Bassi et al. · 2013 [cited by applicant]
US 8922636B1 · Belden et al. · 2014 [cited by applicant]
US 8950867B2 · Macnamara · 2015 [cited by applicant]
US 8953899B2 · Wu et al. · 2015 [cited by applicant]
US 9081426B2 · Armstrong · 2015 [cited by applicant]
US 9191658B2 · Kato et al. · 2015 [cited by applicant]
US 9215293B2 · Miller · 2015 [cited by applicant]
US D752529S · Loretan et al. · 2016 [cited by applicant]
US 9297945B2 · Ide et al. · 2016 [cited by applicant]
US 9310559B2 · Macnamara · 2016 [cited by applicant]
US 9348143B2 · Gao et al. · 2016 [cited by applicant]
US D758367S · Natsume · 2016 [cited by applicant]
US D759657S · Kujawski et al. · 2016 [cited by applicant]
US 9383587B2 · Balogh · 2016 [cited by applicant]
US 9417452B2 · Schowengerdt et al. · 2016 [cited by applicant]
US 9470906B2 · Kaji et al. · 2016 [cited by applicant]
US 9536492B2 · Jin et al. · 2017 [cited by applicant]
US 9547174B2 · Gao et al. · 2017 [cited by applicant]
US 9671566B2 · Abovitz et al. · 2017 [cited by applicant]
US D794288S · Beers et al. · 2017 [cited by applicant]
US 9740006B2 · Gao · 2017 [cited by applicant]
US 9791700B2 · Schowengerdt · 2017 [cited by applicant]
US D805734S · Fisher et al. · 2017 [cited by applicant]
US 9835864B2 · Morifuji et al. · 2017 [cited by applicant]
US 9851563B2 · Gao et al. · 2017 [cited by applicant]
US 9857591B2 · Welch et al. · 2018 [cited by applicant]
US 9874749B2 · Bradski et al. · 2018 [cited by applicant]
US 10260864B2 · Edwin et al. · 2019 [cited by applicant]
US 10378882B2 · Yeoh et al. · 2019 [cited by applicant]
US 10571251B2 · Edwin et al. · 2020 [cited by applicant]
US 11226193B2 · Yeoh et al. · 2022 [cited by applicant]
US 11536559B2 · Yeoh et al. · 2022 [cited by applicant]
US 20020063807A1 · Margulis · 2002 [cited by applicant]
US 20060028436A1 · Armstrong · 2006 [cited by applicant]
US 20060072206A1 · Tsuyuki et al. · 2006 [cited by applicant]
US 20060077558A1 · Urakawa et al. · 2006 [cited by applicant]
US 20070081123A1 · Lewis · 2007 [cited by applicant]
US 20080062164A1 · Bassi et al. · 2008 [cited by applicant]
US 20080144174A1 · Lucente et al. · 2008 [cited by applicant]
US 20080198240A1 · Duncan et al. · 2008 [cited by applicant]
US 20090028423A1 · Sandstrom et al. · 2009 [cited by applicant]
US 20100214310A1 · Kuwahara et al. · 2010 [cited by applicant]
US 20100265385A1 · Knight · 2010 [cited by examiner]
US 20110261169A1 · Tin · 2011 [cited by applicant]
US 20110279749A1 · Erinjippurath · 2011 [cited by examiner]
US 20120019493A1 · Barnhoefer · 2012 [cited by examiner]
US 20120081360A1 · Uehira et al. · 2012 [cited by applicant]
US 20120113223A1 · Hilliges et al. · 2012 [cited by applicant]
US 20120127062A1 · Bar-Zeev et al. · 2012 [cited by applicant]
US 20120127320A1 · Balogh · 2012 [cited by applicant]
US 20120162549A1 · Gao et al. · 2012 [cited by applicant]
US 20120182334A1 · Ranieri et al. · 2012 [cited by applicant]
US 20120212499A1 · Haddick et al. · 2012 [cited by applicant]
US 20130082922A1 · Miller · 2013 [cited by applicant]
US 20130083404A1 · Takagi et al. · 2013 [cited by applicant]
US 20130117377A1 · Miller · 2013 [cited by applicant]
US 20130125027A1 · Abovitz · 2013 [cited by applicant]
US 20130169683A1 · Perez et al. · 2013 [cited by applicant]
US 20130208234A1 · Lewis · 2013 [cited by applicant]
US 20130235169A1 · Kato et al. · 2013 [cited by applicant]
US 20130242262A1 · Lewis · 2013 [cited by applicant]
US 20130278790A1 · Oh et al. · 2013 [cited by applicant]
US 20130300635A1 · White et al. · 2013 [cited by applicant]
US 20140002587A1 · Aguren · 2014 [cited by applicant]
US 20140003762A1 · Macnamara · 2014 [cited by applicant]
US 20140035959A1 · Lapstun · 2014 [cited by applicant]
US 20140071539A1 · Gao · 2014 [cited by applicant]
US 20140177023A1 · Gao et al. · 2014 [cited by applicant]
US 20140198104A1 · Tanaka · 2014 [cited by examiner]
US 20140205329A1 · Inoue · 2014 [cited by applicant]
US 20140218468A1 · Gao et al. · 2014 [cited by applicant]
US 20140226131A1 · Lopez · 2014 [cited by examiner]
US 20140240842A1 · Nguyen et al. · 2014 [cited by applicant]
US 20140267420A1 · Schowengerdt et al. · 2014 [cited by applicant]
US 20140267771A1 · Lawler · 2014 [cited by applicant]
US 20140306866A1 · Miller et al. · 2014 [cited by applicant]
US 20140313423A1 · Johnson et al. · 2014 [cited by applicant]
US 20140347548A1 · Wu et al. · 2014 [cited by applicant]
US 20140361987A1 · Bickerstaff et al. · 2014 [cited by applicant]
US 20150016777A1 · Abovitz et al. · 2015 [cited by applicant]
US 20150098620A1 · Wu et al. · 2015 [cited by applicant]
US 20150103306A1 · Kaji et al. · 2015 [cited by applicant]
US 20150104101A1 · Bryant et al. · 2015 [cited by applicant]
US 20150168731A1 · Robbins · 2015 [cited by applicant]
US 20150178939A1 · Bradski et al. · 2015 [cited by applicant]
US 20150198808A1 · Morifuji et al. · 2015 [cited by applicant]
US 20150205126A1 · Schowengerdt · 2015 [cited by applicant]
US 20150222883A1 · Welch · 2015 [cited by applicant]
US 20150222884A1 · Cheng · 2015 [cited by applicant]
US 20150241959A1 · Abovitz et al. · 2015 [cited by applicant]
US 20150264299A1 · Leech et al. · 2015 [cited by applicant]
US 20150268415A1 · Schowengerdt et al. · 2015 [cited by applicant]
US 20150286340A1 · Send et al. · 2015 [cited by applicant]
US 20150296203A1 · Lucente et al. · 2015 [cited by applicant]
US 20150302652A1 · Miller et al. · 2015 [cited by applicant]
US 20150309263A2 · Abovitz et al. · 2015 [cited by applicant]
US 20150309316A1 · Osterhout et al. · 2015 [cited by applicant]
US 20150326570A1 · Publicover et al. · 2015 [cited by applicant]
US 20150346490A1 · TeKolste et al. · 2015 [cited by applicant]
US 20150346495A1 · Welch et al. · 2015 [cited by applicant]
US 20160011419A1 · Gao · 2016 [cited by applicant]
US 20160026253A1 · Bradski et al. · 2016 [cited by applicant]
US 20160029883A1 · Cox · 2016 [cited by applicant]
US 20160035305A1 · Jin et al. · 2016 [cited by applicant]
US 20160085301A1 · Lopez · 2016 [cited by applicant]
US 20160131902A1 · Ambrus et al. · 2016 [cited by applicant]
US 20160370605A1 · Ain-Kedem · 2016 [cited by applicant]
US 20170122725A1 · Yeoh et al. · 2017 [cited by applicant]
US 20170124928A1 · Edwin et al. · 2017 [cited by applicant]
US 20190323825A1 · Yeoh et al. · 2019 [cited by applicant]
US 20200225024A1 · Edwin et al. · 2020 [cited by applicant]
US 20210148697A1 · Yeoh et al. · 2021 [cited by applicant]
EP 0579224A1 · 1994 [cited by applicant]
JP H05161166A · 1993 [cited by applicant]
JP H11203986A · 1999 [cited by applicant]
JP 2002014053A · 2002 [cited by applicant]
JP 2005101828A · 2005 [cited by applicant]
JP 2006153914A · 2006 [cited by applicant]
JP 2008191257A · 2008 [cited by applicant]
JP 2008258802A · 2008 [cited by applicant]
JP 2010271565A · 2010 [cited by applicant]
JP 2012098733A · 2012 [cited by applicant]
JP 2013037021A · 2013 [cited by applicant]
JP 2013240057A · 2013 [cited by applicant]
JP 2014199385A · 2014 [cited by applicant]
JP 2015099238A · 2018 [cited by applicant]
WO 2014155133A1 · 2014 [cited by applicant]
WO 2015081313A2 · 2015 [cited by applicant]
WO 2017079329A1 · 2017 [cited by applicant]
WO 2017079333A1 · 2017 [cited by applicant]
JP2023-9319 Office Action mailed Feb. 22, 2024. [cited by applicant]
ARToolKit: https://web.archive.org/web/20051013062315/http://www.hitl.washington.edu:80/artoolkit/documentation/hardware.htm, archived Oct. 13, 2005. [cited by applicant]
Azuma, “A Survey of Augmented Reality,” Teleoperators and Virtual Environments 6, 4 (Aug. 1997), pp. 355-385. https://web.archive.org/web/20010604100006/http://www.cs.unc.edu/ azuma/ARpresence.pdf. [cited by applicant]
Azuma, “Predictive Tracking for Augmented Realty,” TR95-007, Department of Computer Science, UNC-Chapel Hill, NC, Feb. 1995. [cited by applicant]
Bimber, et al., “Spatial Augmented Reality—Merging Real and Virtual Worlds,” 2005 https://web.media.mit.edu/raskar/book/BimberRaskarAugmentedRealityBook.pdf. [cited by applicant]
Chen, et al., “Automatic geometrical calibration for multiprojector-type light field three-dimensional display,” Optical Engineering 53(7), 073107 (Jul. 2014). [cited by applicant]
European Extended Search Report, re EP Application No. 16862899.8, dated Apr. 3, 2019. [cited by applicant]
European Extended Search Report, re EP Application No. 16862902.0, dated Apr. 5, 2019. [cited by applicant]
International Preliminary Report on Patentability for PCT Application No. PCT/US16/60158, dated May 8, 2018. [cited by applicant]
International Preliminary Report on Patentability for PCT Application No. PCT/US16/60162, dated May 8, 2018. [cited by applicant]
International Search Report and Written Opinion for PCT Application No. PCT/US16/60158, dated Jan. 23, 2017. [cited by applicant]
International Search Report and Written Opinion for PCT Application No. PCT/US16/60162, dated Jan. 31, 2017. [cited by applicant]
Jacob, “Eye Tracking in Advanced Interface Design,” Human-Computer Interaction Lab Naval Research Laboratory, Washington, D.C. / paper/ in Virtual Environments and Advanced Interface Design, ed. by W. Barfield and T.A. … [cited by applicant]
KR2018-7015893 Office Action dated Feb. 15, 2023. [cited by applicant]
KR2018-7015896 Office Action dated May 11, 2023. [cited by applicant]
Pogalin, Erik, “Gaze Tracking by Using Factorized Likelihoods Particle Filtering and Stereo Vision,” 3D Data Processing, Visualization, and Transmission, Third International AL Symposium on, IEEE, PI, pp. 57-64, Jun. 1,… [cited by applicant]
Tanriverdi and Jacob, “Interacting With Eye Movements in Virtual Environments,” Department of Electrical Engineering and Computer Science, Tufts University, Medford, MA—paper/Proc. AMC CHI 2000 Human Factors in Computin… [cited by applicant]
Wikipedia: “Light-field camera,” Wikipedia, printed Sep. 17, 2015, in 5 pages. URL: https://en.wikipedia.org/wiki/Light-field_camera. [cited by applicant]
KR2024-7003400 Office Action dated May 13, 2024. [cited by applicant]
EP24215512.5 Extended European Search Report dated Feb. 10, 2025. [cited by applicant]
Itoh, Yuta, et al., “Light-Field Correction for Spatial Calibration of Optical See-Through Head-Mounted Displays,” IEEE Transactions of Visualization and Computer Graphics, IEEE, USA, vol. 21, No. 4, Apr. 18, 2015, pp. … [cited by applicant]
Jones, Adam J., et al., Correction of Geometric Distortions and the Impact of Eye Position in Virtual Reality Displays, 2015 Intl. Conf. on Collaboration Technologies and Systems (CTS), IEEE, Jun. 1, 2015, pp. 77-83. [cited by applicant]