IP Library Granted Patent US 12,205,220
Granted Patent B2
US 12,205,220 · App. 18/077,677 · Granted Jan 21, 2025

Display systems and methods for clipping content to increase viewing comfort

Inventors: Brian David Schwab (Sunrise, FL); Randall E. Hand (Parkland, FL); Björn Nicolaas Servatius Maskamp (Seattle, WA)
Assignee: MAGIC LEAP, INC.
G06T15/30G06T15/20
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,205,220
App. No.
18/077,677
Granted
Jan 21, 2025
Kind
B2
Abstract

Augmented and virtual reality display systems increase viewer comfort by reducing viewer exposure to virtual content that causes undesirable accommodation-vergence mismatches (AVM). The display systems limit displaying content that exceeds an accommodation-vergence mismatch threshold, which may define a volume around the viewer. The volume may be subdivided into two or more zones, including an innermost loss-of-fusion zone (LoF) in which no content is displayed, and one or more outer AVM zones in which the displaying of content may be stopped, or clipped, under certain conditions. For example, content may be clipped if the viewer is verging within an AVM zone and if the content is displayed within the AVM zone for more than a threshold duration. A further possible condition for clipping content is that the user is verging on that content. In addition, the boundaries of the AVM zone and/or the acceptable amount of time that the content is displayed may vary depending upon the type of content being displayed, e.g., whether the content is user-locked content or in-world content.

Claims (43)

1. A head-mounted display system for displaying a virtual object, the display system comprising:

one or more processors; and

one or more computer storage media storing instructions that, when executed by the one or more processors, cause the one or more processors to perform operations comprising:

determining whether a virtual object is to be displayed at a location having an accommodation-vergence mismatch exceeding a threshold value;

determining whether an eye of the user is closed;

refraining from displaying the virtual object after determining that an eye of the user is not closed and that the virtual object is to be displayed at a location having the accommodation-vergence mismatch exceeding the threshold value; and

displaying the virtual object after determining that the eye of the user is closed and that the virtual object is to be displayed at a location having the accommodation-vergence mismatch exceeding the threshold value.

2. The display system of claim 1 , wherein displaying the virtual object after determining that the eye of the user is closed comprises displaying the virtual object to an open eye of the user after determining that an other eye of the user is closed.

3. The display system of claim 1 , wherein displaying the virtual object after determining that the eye of the user is closed comprises displaying the virtual object within a loss-of fusion zone of the user.

4. The display system of claim 1 , comprising an eyepiece configured to output light with variable amounts of wavefront divergence.

5. The display system of claim 4 , wherein the eyepiece comprises a stack of waveguides, wherein at least one of the waveguides of the stack of waveguides is configured to output light with a different amount of wavefront divergence than an other waveguide of the stack of the waveguides.

6. The display system of claim 1 , wherein the operations further comprise: identifying a type of content corresponding to the virtual object; and setting the threshold value based at least in part on the identified type of content.

7. The display system of claim 6 , wherein the type of content comprises user-locked content or in-world content.

8. A method for displaying a virtual object by a head-mounted display system, the method comprising:

determining whether a virtual object is to be displayed at a location having an accommodation-vergence mismatch exceeding a threshold value;

determining whether an eye of the user is closed;

refraining from displaying the virtual object after determining that an eye of the user is not closed and that the virtual object is to be displayed at a location having the accommodation-vergence mismatch exceeding the threshold value; and

displaying the virtual object after determining that the eye of the user is closed and that the virtual object is to be displayed at a location having the accommodation-vergence mismatch exceeding the threshold value.

9. The method of claim 8 , wherein displaying the virtual object after determining that the eye of the user is closed comprises displaying the virtual object to an open eye of the user after determining that an other eye of the user is closed.

10. The method of claim 8 , wherein displaying the virtual object after determining that the eye of the user is closed comprises displaying the virtual object within a loss-of-fusion zone of the user.

11. The method of claim 8 , wherein the display system comprises an eyepiece configured to output light with variable amounts of wavefront divergence.

12. The method of claim 11 , wherein the eyepiece comprising a stack of waveguides, wherein at least one of the waveguides of the stack of waveguides is configured to output light with a different amount of wavefront divergence than an other waveguide of the stack of the waveguides.

13. The method of claim 11 , wherein the operations further comprise:

identifying a type of content corresponding to the virtual object; and

setting the threshold value based at least in part on the identified type of content.

14. The method of claim 8 , wherein the type of content comprises user-locked content or in-world content.

15. A display system for displaying a virtual object in a binocular field of view (FOV) of a user, the display system comprising:

one or more processors; and

one or more computer storage media storing instructions that, when executed by the one or more processors, cause the one or more processors to perform operations comprising:

determining whether the virtual object has crossed a boundary between different zones of a plurality of zones corresponding to different volumes of space in the FOV, wherein the determining whether the virtual object has crossed the boundary between the different zones includes determining whether the virtual object has crossed the boundary between an accommodation-vergence mismatch discomfort zone and an accommodation-vergence comfort zone; and

in response to determining that the virtual object has crossed the boundary between the different zones, switching between (i) a binocular presentation mode in which the virtual object is displayed to both eyes of the user, and (ii) a monocular presentation mode in which the virtual object is only displayed to one eye of the user.

16. The display system of claim 15 , wherein the operations further comprise:

selecting, from among both eyes of the user, a particular eye of the user, wherein the virtual object is only displayed to the particular eye of the user in the monocular presentation mode.

17. The display system of claim 16 , wherein the operations further comprise:

determining which eye of the user is the user's dominant eye, wherein selecting the particular eye of the user comprises the dominant eye to display the virtual object.

18. The display system of claim 16 , wherein the operations further comprise:

determining whether the virtual object is located within a left portion of the user's binocular FOV or a right portion of the user's binocular FOV,

wherein selecting, from among both eyes of the user, the particular eye of the user comprises: selecting, from among both eyes of the user, the particular eye of the user based at least in part on determining whether the virtual object is located within the left portion of the user's binocular FOV or the right portion of the user's binocular FOV.

19. The display system of claim 15 , wherein switching between (i) the binocular presentation mode in which the virtual object is displayed to both eyes of the user, and (ii) the monocular presentation mode in which the virtual object is only displayed to one eye of the user comprises:

switching between (i) a binocular presentation mode in which the virtual object is displayed at a first level of brightness to each eye of the user, and (ii) a monocular presentation mode in which the virtual object is only displayed at a second level of brightness to one eye of the user, the second level of brightness being greater than the first level of brightness in which the other virtual object is only displayed to one eye of the user.

20. The display system of claim 15 , wherein the operations further comprise:

determining whether another virtual object has crossed the boundary between different zones of the binocular FOV of the user; and

in response to determining that the other virtual object has crossed the boundary between different zones of the binocular FOV of the user and independent from determining that the virtual object has crossed the boundary between different zones of the binocular FOV of the user, switching between (i) a binocular presentation mode in which the other virtual object is displayed to both eyes of the user, and (ii) a monocular presentation mode in which the other virtual object is only displayed to one eye of the user.

Assignments (3)
SECURITY INTEREST Recorded Oct 31, 2025
From: MAGIC LEAP, INC.; MENTOR ACQUISITION ONE, LLC; MOLECULAR IMPRINTS, INC.
To: CITIBANK, N.A., AS COLLATERAL AGENT
Reel/Frame 073422/0549 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 7, 2023
From: SCHWAB, BRIAN DAVID; HAND, RANDALL E.; VLASKAMP, BJÖRN NICOLAAS SERVATIUS
To: MAGIC LEAP, INC.
Reel/Frame 062907/0991 →
SECURITY INTEREST Recorded Feb 7, 2023
From: MAGIC LEAP, INC.; MENTOR ACQUISITION ONE, LLC; MOLECULAR IMPRINTS, INC.
To: CITIBANK, N.A., AS COLLATERAL AGENT
Reel/Frame 062681/0065 →
Continuity (4)
Continuation 17132436 · Dec 23, 2020
Division 16353989 · Mar 14, 2019
Provisional Application 62642761 · Mar 14, 2018
Related Publication 20230099062A1 · Mar 30, 2023
References Cited (83)
US 6850221B1 · Tickle · 2005 [cited by applicant]
US 9081426B2 · Armstrong · 2015 [cited by applicant]
US 9215293B2 · Miller · 2015 [cited by applicant]
US 9348143B2 · Gao et al. · 2016 [cited by applicant]
US 9417452B2 · Schowengerdt et al. · 2016 [cited by applicant]
US 9470906B2 · Kaji et al. · 2016 [cited by applicant]
US 9547174B2 · Gao et al. · 2017 [cited by applicant]
US 9671566B2 · Abovitz et al. · 2017 [cited by applicant]
US 9740006B2 · Gao · 2017 [cited by applicant]
US 9760167B2 · Fateh · 2017 [cited by applicant]
US 9791700B2 · Schowengerdt · 2017 [cited by applicant]
US 9851563B2 · Gao et al. · 2017 [cited by applicant]
US 9857591B2 · Welch et al. · 2018 [cited by applicant]
US 9874749B2 · Bradski et al. · 2018 [cited by applicant]
US 10878620B2 · Schwab et al. · 2020 [cited by applicant]
US 11532118B2 · Schwab · 2022 [cited by examiner]
US 20060028436A1 · Armstrong · 2006 [cited by applicant]
US 20070081123A1 · Lewis · 2007 [cited by applicant]
US 20100103077A1 · Sugiyama et al. · 2010 [cited by applicant]
US 20110109720A1 · Smolic et al. · 2011 [cited by applicant]
US 20110228059A1 · Nagai · 2011 [cited by applicant]
US 20120069002A1 · Kuribayashi et al. · 2012 [cited by applicant]
US 20120069296A1 · Li et al. · 2012 [cited by applicant]
US 20120127062A1 · Bar-Zeev et al. · 2012 [cited by applicant]
US 20120162549A1 · Gao et al. · 2012 [cited by applicant]
US 20120320047A1 · Yanagita et al. · 2012 [cited by applicant]
US 20130057574A1 · Shimizu · 2013 [cited by applicant]
US 20130082922A1 · Miller · 2013 [cited by applicant]
US 20130117377A1 · Miller · 2013 [cited by applicant]
US 20130125027A1 · Abovitz · 2013 [cited by applicant]
US 20130208234A1 · Lewis · 2013 [cited by applicant]
US 20130242262A1 · Lewis · 2013 [cited by applicant]
US 20130286004A1 · Mcculloch et al. · 2013 [cited by applicant]
US 20130335301A1 · Wong et al. · 2013 [cited by applicant]
US 20140071539A1 · Gao · 2014 [cited by applicant]
US 20140085442A1 · Ono · 2014 [cited by applicant]
US 20140177023A1 · Gao et al. · 2014 [cited by applicant]
US 20140204176A1 · Boisson et al. · 2014 [cited by applicant]
US 20140218468A1 · Gao et al. · 2014 [cited by applicant]
US 20140267420A1 · Schowengerdt et al. · 2014 [cited by applicant]
US 20140285641A1 · Kato et al. · 2014 [cited by applicant]
US 20140306866A1 · Miller et al. · 2014 [cited by applicant]
US 20140307065A1 · Okano · 2014 [cited by applicant]
US 20140368535A1 · Salter et al. · 2014 [cited by applicant]
US 20150016777A1 · Abovitz et al. · 2015 [cited by applicant]
US 20150103306A1 · Kaji et al. · 2015 [cited by applicant]
US 20150178939A1 · Bradski et al. · 2015 [cited by applicant]
US 20150205126A1 · Schowengerdt · 2015 [cited by applicant]
US 20150222883A1 · Welch · 2015 [cited by applicant]
US 20150222884A1 · Cheng · 2015 [cited by applicant]
US 20150268415A1 · Schowengerdt et al. · 2015 [cited by applicant]
US 20150302652A1 · Miller et al. · 2015 [cited by applicant]
US 20150309263A2 · Abovitz et al. · 2015 [cited by applicant]
US 20150326570A1 · Publicover et al. · 2015 [cited by applicant]
US 20150346490A1 · Tekolste et al. · 2015 [cited by applicant]
US 20150346495A1 · Welch et al. · 2015 [cited by applicant]
US 20160011419A1 · Gao · 2016 [cited by applicant]
US 20160026253A1 · Bradski et al. · 2016 [cited by applicant]
US 20160140728A1 · Aonuma et al. · 2016 [cited by applicant]
US 20160152184A1 · Ogawa et al. · 2016 [cited by applicant]
US 20160316196A1 · Mackenzie et al. · 2016 [cited by applicant]
US 20170237974A1 · Samec et al. · 2017 [cited by applicant]
US 20170262054A1 · Lanman et al. · 2017 [cited by applicant]
US 20170293145A1 · Miller et al. · 2017 [cited by applicant]
US 20170336861A1 · Fateh · 2017 [cited by applicant]
US 20190311527A1 · Schwab et al. · 2019 [cited by applicant]
US 20200186761A1 · Greenberg · 2020 [cited by applicant]
AU 2016233268A1 · 2017 [cited by applicant]
JP 2017500605A · 2018 [cited by applicant]
WO WO2003079272A1 · 2003 [cited by applicant]
WO 2014057618A1 · 2014 [cited by applicant]
WO 2017165848A1 · 2017 [cited by applicant]
WO 2019178409A1 · 2019 [cited by applicant]
ARToolKit: https://web.archive.org/web/20051013062315/http://www.hitl.washington.edu:80/artoolkit/documentation/hardware.htm, archived Oct. 13, 2005. [cited by applicant]
Azuma, “A Survey of Augmented Reality,” Teleoperators and Virtual Environments 6, (Aug. 4, 1997), pp. 355-385. https://web.archive.org/web/20010604100006/http://www.cs.unc.edu/azuma/ARpresence.pdf. [cited by applicant]
Azuma, “Predictive Tracking for Augmented Realty,” TR95-007, Department of Computer Science, UNC-Chapel Hill, NC, Feb. 1995. [cited by applicant]
Bimber, et al., “Spatial Augmented Reality—Merging Real and Virtual Worlds,” 2005 https://web.media.mit.edu/ [cited by applicant]
David M. Hoffman et al., “Vergence-accommodation conflicts hinder visual performance and cause visual fatigue”, Journal of Vision Mar. 2008, vol. 8, 33. doi:https://doi.org/10.1167/8.3.33 (Year: 2008). [cited by applicant]
International Preliminary Report on Patentability for PCT Application No. PCT/US2019/022349, dated Sep. 15, 2020. [cited by applicant]
International Search Report and Written Opinion for PCT Application No. PCT/US2019/022349, dated Jul. 3, 2019. [cited by applicant]
Jacob, “Eye Tracking in Advanced Interface Design,” Human-Computer Interaction Lab Naval Research Laboratory, Washington, D.C. / paper/ in Virtual Environments and Advanced Interface Design, ed. by W. Barfield and T.A. … [cited by applicant]
Tanriverdi and Jacob, “Interacting With Eye Movements in Virtual Environments,” Department of Electrical Engineering and Computer Science, Tufts University, Medford, MA-paper/Proc. ACM CHI 2000 Human Factors in Computin… [cited by applicant]
Willemsen, et al., “Effects of Stereo Viewing Conditions on Distance Perception in Virtual Environment,” Teleoperators and Virtual Environments, Feb. 15, 2005. [cited by applicant]