IP Library › Granted Patent US 12,253,680
Granted Patent B2
US 12,253,680 · App. 18/475,688 · Granted Mar 18, 2025

Imaging modification, display and visualization using augmented and virtual reality eyewear

Inventors: Nastasja U. Robaina (Coconut Grove, FL); Nicole Elizabeth Samec (Fort Lauderdale, FL); Christopher M. Harrises (Nashua, NH); Rony Abovitz (Weston, FL); Mark Baerenrodt (Millbrae, CA); Brian Lloyd Schmidt (Bellevue, WA)
Assignee: Magic Leap, Inc.
G02B27/0172A61B34/25A61B90/36A61B90/37G02B27/01G06F3/011G06F3/012G06F3/013G06F3/017G06F3/0304G06T19/006A61B2017/00203A61B2017/00207A61B2017/00216A61B2034/2048A61B2034/254A61B2034/258A61B2090/365A61B2090/367A61B2090/368A61B2090/371A61B2090/372A61B2090/378A61B2090/502
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,253,680
App. No.
18/475,688
Granted
Mar 18, 2025
Kind
B2
Abstract

A display system can include a head-mounted display configured to project light to an eye of a user to display augmented reality image content to the user. The display system can include one or more user sensors configured to sense the user and can include one or more environmental sensors configured to sense surroundings of the user. The display system can also include processing electronics in communication with the display, the one or more user sensors, and the one or more environmental sensors. The processing electronics can be configured to sense a situation involving user focus, determine user intent for the situation, and alter user perception of a real or virtual object within the vision field of the user based at least in part on the user intent and/or sensed situation involving user focus. The processing electronics can be configured to at least one of enhance or de-emphasize the user perception of the real or virtual object within the vision field of the user.

Claims (44)

1. A display system comprising:

a display, said display configured to project light into a user's eye to display augmented reality image content to a vision field of the user, at least a portion of said display being transparent and disposed at a location in front of the user's eye when a user operates the system such that said transparent portion transmits light from a portion of the environment in front of the user to the user's eye to provide a view of said portion of the environment in front of the user and said display;

one or more sensors;

processing electronics in communication with the display and the one or more sensors; and

memory storing computer-readable instructions that, when executed by the processing electronics, cause the processing electronics to perform operations comprising:

detecting a frequency of change in gaze direction between a first gaze direction of the user and a second gaze direction of the user;

detecting increased user focus on an object within a vision field of the user for the second gaze direction, wherein the object is within a peripheral location of the vision field of the user for the first gaze direction; and

altering user perception of the object within the vision field of the user for the first gaze direction based at least in part on detecting the increased user focus on the object in the second gaze direction and on the frequency of change in gaze direction exceeding a minimum frequency of change in gaze direction, and

wherein altering user perception of the object within the vision field of the user for the first gaze direction comprises generating image content within the vision field of the user for the first gaze direction such that the object appears to move from the peripheral location of the vision field for the first gaze direction to a central location of the vision field for the first gaze direction.

2. The display system of claim 1 , wherein altering user perception of the object within the vision field of the user for the first gaze direction further comprises altering at least one of a contrast, an opacity, color, color saturation, color balance, size, background, brightness, edges, or sharpness of the image content.

3. The display system of claim 1 , wherein the one or more sensors comprises at least one of a depth sensor, a geolocation sensor, a camera, a proximity sensor, a wireless signal sensor, and a global positioning system (GPS) sensor.

4. The display system of claim 1 , wherein the operations comprise detecting a number of occasions that the user directs the user's head, at least one eye, or head and at least one eye toward the second gaze direction, and

wherein altering user perception of the object within the vision field of the user for the first gaze direction is further based on the number of occasions that the user directs the user's head, at least one eye, or head and at least one eye toward the second gaze direction exceeding a minimum number.

5. The display system of claim 4 , wherein detecting the increased user focus on the object is based on sensing the user struggling to view the object within the vision field of the user for the second gaze direction.

6. The display system of claim 5 , wherein sensing the user struggling to view the object comprises detecting fluctuations in accommodation of the user's eye.

7. The display system of claim 5 , wherein sensing the user struggling to view the object comprises detecting fluctuations in vergence of the user's eye.

8. The display system of claim 1 , wherein altering user perception comprises enhancing an appearance of the object that appears to move from the peripheral location to the central location.

9. The display system of claim 8 , wherein enhancing the appearance of the object comprises magnifying the appearance of the object.

10. The display system of claim 8 , wherein a degree of enhancement of the appearance of the object is based on at least one of a distance of the object in the peripheral location from the fovea of the user's eye for the first gaze direction, a minimum distance for two-point distinction, and a spatial resolution of the user's eye.

11. The display system of claim 1 , wherein generating the image content comprises presenting the image content to a depth plane in the central location of the vision field for the first gaze direction that is closer to the user than a depth plane at which the object is located in the peripheral location of the vision field for the first gaze direction.

12. The display system of claim 1 , wherein generating the image content comprises moving the image content to a location corresponding to a projected light location of the user's eye that is in a range of 0 to 10 degrees from the center of the fovea of the user's eye.

13. A computer-implemented method comprising:

detecting, by processing electronics, a frequency of change in a user's gaze direction between a first gaze direction of the user and a second gaze direction of the user;

detecting, by the processing electronics, an increased user focus on an object within a vision field of the user for the second gaze direction, wherein the object is within a peripheral location of the vision field of the user for the first gaze direction; and

altering user perception of the object within the vision field of the user for the first gaze direction based at least in part on detecting the increased user focus on the object in the second gaze direction, and on the frequency of change in gaze direction exceeding a minimum frequency of change in gaze direction, and

wherein altering the user perception of the object within the vision field of the user for the first gaze direction comprises causing a display system to generate and display image content within the user's vision field for the first gaze direction such that the object appears to move from the peripheral location of the vision field for the first gaze direction to a central location of the vision field for the first gaze direction.

14. The method of claim 13 , wherein altering user perception of the object within the vision field of the user for the first gaze direction further comprises altering at least one of a contrast, an opacity, color, color saturation, color balance, size, background, brightness, edges, or sharpness of the image content.

15. The method of claim 13 , comprising sensing, by one or more sensors of the display system, the user and surroundings of the user, wherein detecting the frequency of change in the user's gaze direction is based on measurement signals from at least a first sensor of the one or more sensors.

16. The method of claim 13 , comprising sensing, by one or more sensors, the user and surroundings of the user, wherein detecting the increased user focus on the object within the vision field of the user for the second gaze direction is based on measurement signals from at least a first sensor of the one or more sensors.

17. The method of either claim 15 or 16 , wherein the display system comprises a plurality of sensors, and the plurality of sensors comprises at least one of a depth sensor, a geolocation sensor, a camera, a proximity sensor, a wireless signal sensor, and a global positioning system (GPS) sensor.

18. The method of claim 13 , comprising detecting a number of occasions that the user directs the user's head, at least one eye, or head and at least one eye toward the second gaze direction, and

wherein altering user perception of the object within the vision field of the user for the first gaze direction is further based on the number of occasions that the user directs the user's head, at least one eye, or head and at least one eye toward the second gaze direction exceeding a minimum number.

19. The method of claim 13 , wherein detecting the increased user focus on the object is based on sensing the user struggling to view the object within the vision field of the user for the second gaze direction.

20. The method of claim 19 , wherein sensing the user struggling to view the object comprises detecting fluctuations in accommodation of an eye of the user.

21. The method of claim 19 , wherein sensing the user struggling to view the object comprises detecting fluctuations in vergence of an eye of the user.

22. The method of claim 13 , wherein altering user perception comprises enhancing an appearance of the object that appears to move from the peripheral location to the central location.

23. The method of claim 22 , wherein enhancing the appearance of the object comprises magnifying the appearance of the object.

24. The method of claim 22 , wherein a degree of enhancement of the appearance of the object is based on at least one of a distance of the object in the peripheral location from the fovea of an eye of the user for the first gaze direction, a minimum distance for two-point distinction, and a spatial resolution of the eye of the user.

25. The method of claim 13 , wherein causing the display system to generate and display the image content comprises presenting the image content to a depth plane in the central location of the vision field for the first gaze direction that is closer to the user than a depth plane at which the object is located in the peripheral location of the vision field for the first gaze direction.

26. The method of claim 13 , wherein causing the display system to generate and display the image content comprises moving the image content to a location corresponding to a projected light location of an eye of the user that is in a range of 0 to 10 degrees from the center of the fovea of the eye of the user.

27. The display system of claim 1 , wherein detecting a frequency of change in gaze direction between a first gaze direction of the user and a second gaze direction of the user comprises detecting a repeated change in the gaze direction.

28. The display system of claim 1 , wherein detecting a frequency of change in gaze direction between a first gaze direction of the user and a second gaze direction of the user comprises detecting a frequency of change in gaze direction between a first gaze direction of the user and a second gaze direction of the user with a frequency greater than 1.

29. The method of claim 13 , wherein detecting a frequency of change in gaze direction between a first gaze direction of the user and a second gaze direction of the user comprises detecting a repeated change in the gaze direction.

30. The method of claim 13 , wherein detecting a frequency of change in gaze direction between a first gaze direction of the user and a second gaze direction of the user comprises detecting a frequency of change in gaze direction between a first gaze direction of the user and a second gaze direction of the user with a frequency greater than 1.

Assignments (4)
SECURITY INTEREST Recorded Oct 31, 2025
From: MAGIC LEAP, INC.; MENTOR ACQUISITION ONE, LLC; MOLECULAR IMPRINTS, INC.
To: CITIBANK, N.A., AS COLLATERAL AGENT
Reel/Frame 073422/0549 →
SECURITY INTEREST Recorded Oct 20, 2025
From: MAGIC LEAP, INC.; MENTOR ACQUISITION ONE, LLC; MOLECULAR IMPRINTS, INC.
To: CITIBANK, N.A., AS COLLATERAL AGENT
Reel/Frame 073031/0206 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Oct 20, 2023
From: ROBAINA, NASTASJA U.; SAMEC, NICOLE ELIZABETH; ABOVITZ, RONY; BAERENRODT, MARK; SCHMIDT, BRIAN LLOYD
To: MAGIC LEAP, INC.
Reel/Frame 065289/0601 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Oct 20, 2023
From: HARRISES, CHRISTOPHER M.; TECHNICAL SOLUTIONS, INC.
To: MAGIC LEAP, INC.
Reel/Frame 065289/0638 →
Continuity (7)
Continuation 17644776 · Dec 16, 2021
Continuation 16790576 · Feb 13, 2020
Continuation 15657589 · Jul 24, 2017
Provisional Application 62440332 · Dec 29, 2016
Provisional Application 62396071 · Sep 16, 2016
Provisional Application 62366599 · Jul 25, 2016
Related Publication 20240053612A1 · Feb 15, 2024
References Cited (232)
US 5913079A · Aoyama et al. · 1999 [cited by applicant]
US 6456262B1 · Bell · 2002 [cited by applicant]
US 6850221B1 · Tickle · 2005 [cited by applicant]
US 8223088B1 · Gomez et al. · 2012 [cited by applicant]
US 8681256B2 · Sako et al. · 2014 [cited by applicant]
US 8950867B2 · Macnamara · 2015 [cited by applicant]
US 8988474B2 · Travis · 2015 [cited by applicant]
US 9081426B2 · Armstrong · 2015 [cited by applicant]
US 9096920B1 · Gomez et al. · 2015 [cited by applicant]
US 9210413B2 · Grinberg et al. · 2015 [cited by applicant]
US 9213405B2 · Perez et al. · 2015 [cited by applicant]
US 9215293B2 · Miller · 2015 [cited by applicant]
US 9245388B2 · Poulos et al. · 2016 [cited by applicant]
US 9247779B1 · Aloumanis et al. · 2016 [cited by applicant]
US 9310559B2 · Macnamara · 2016 [cited by applicant]
US 9323325B2 · Perez et al. · 2016 [cited by applicant]
US 9348143B2 · Gao et al. · 2016 [cited by applicant]
US D758367S · Natsume · 2016 [cited by applicant]
US 9417452B2 · Schowengerdt et al. · 2016 [cited by applicant]
US 9470906B2 · Kaji et al. · 2016 [cited by applicant]
US 9547174B2 · Gao et al. · 2017 [cited by applicant]
US 9671566B2 · Abovitz et al. · 2017 [cited by applicant]
US 9740006B2 · Gao · 2017 [cited by applicant]
US 9791700B2 · Schowengerdt et al. · 2017 [cited by applicant]
US 9851563B2 · Gao et al. · 2017 [cited by applicant]
US 9857591B2 · Welch et al. · 2018 [cited by applicant]
US 9874749B2 · Bradski · 2018 [cited by applicant]
US 10241350B1 · Poulsen · 2019 [cited by applicant]
US 10838210B2 · Robaina et al. · 2020 [cited by applicant]
US 20060028436A1 · Armstrong · 2006 [cited by applicant]
US 20060238704A1 · Donnerhacke et al. · 2006 [cited by applicant]
US 20070081123A1 · Lewis · 2007 [cited by applicant]
US 20070132663A1 · Iba et al. · 2007 [cited by applicant]
US 20070248261A1 · Zhou · 2007 [cited by examiner]
US 20110006978A1 · Yuan · 2011 [cited by applicant]
US 20110112756A1 · Winkler et al. · 2011 [cited by applicant]
US 20110134124A1 · Contractor et al. · 2011 [cited by applicant]
US 20110175925A1 · Kane et al. · 2011 [cited by applicant]
US 20120127062A1 · Bar-Zeev et al. · 2012 [cited by applicant]
US 20120154557A1 · Perez et al. · 2012 [cited by applicant]
US 20120162549A1 · Gao et al. · 2012 [cited by applicant]
US 20120212399A1 · Border et al. · 2012 [cited by applicant]
US 20130050432A1 · Perez et al. · 2013 [cited by applicant]
US 20130082922A1 · Miller · 2013 [cited by applicant]
US 20130117377A1 · Miller · 2013 [cited by applicant]
US 20130125027A1 · Abovitz · 2013 [cited by applicant]
US 20130207951A1 · Didyk · 2013 [cited by examiner]
US 20130208234A1 · Lewis · 2013 [cited by applicant]
US 20130242262A1 · Lewis · 2013 [cited by applicant]
US 20130250135A1 · Blum et al. · 2013 [cited by applicant]
US 20130300634A1 · White et al. · 2013 [cited by applicant]
US 20130314793A1 · Robbins et al. · 2013 [cited by applicant]
US 20140002352A1 · Jacob et al. · 2014 [cited by applicant]
US 20140049452A1 · Maltz · 2014 [cited by applicant]
US 20140071539A1 · Gao · 2014 [cited by applicant]
US 20140125558A1 · Miyajima et al. · 2014 [cited by applicant]
US 20140177023A1 · Gao et al. · 2014 [cited by applicant]
US 20140192084A1 · Latta et al. · 2014 [cited by applicant]
US 20140198017A1 · Lamb et al. · 2014 [cited by applicant]
US 20140218468A1 · Gao et al. · 2014 [cited by applicant]
US 20140225514A1 · Pickard · 2014 [cited by applicant]
US 20140267410A1 · Fein et al. · 2014 [cited by applicant]
US 20140267420A1 · Schowengerdt · 2014 [cited by applicant]
US 20140282646A1 · McCoy et al. · 2014 [cited by applicant]
US 20140306866A1 · Miller et al. · 2014 [cited by applicant]
US 20140341473A1 · Lee et al. · 2014 [cited by applicant]
US 20140361971A1 · Sala · 2014 [cited by applicant]
US 20150016777A1 · Abovitz et al. · 2015 [cited by applicant]
US 20150042679A1 · Jarvenpaa · 2015 [cited by examiner]
US 20150103306A1 · Kaji et al. · 2015 [cited by applicant]
US 20150178939A1 · Bradski et al. · 2015 [cited by applicant]
US 20150187330A1 · Yang · 2015 [cited by applicant]
US 20150205126A1 · Schowengerdt · 2015 [cited by applicant]
US 20150215601A1 · Zhou · 2015 [cited by applicant]
US 20150222883A1 · Welch · 2015 [cited by applicant]
US 20150222884A1 · Cheng · 2015 [cited by applicant]
US 20150241707A1 · Schowengerdt · 2015 [cited by examiner]
US 20150268415A1 · Schowengerdt et al. · 2015 [cited by applicant]
US 20150302652A1 · Miller et al. · 2015 [cited by applicant]
US 20150309263A2 · Abovitz et al. · 2015 [cited by applicant]
US 20150317956A1 · Lection · 2015 [cited by examiner]
US 20150326570A1 · Publicover et al. · 2015 [cited by applicant]
US 20150338915A1 · Publicover et al. · 2015 [cited by applicant]
US 20150346490A1 · TeKolste et al. · 2015 [cited by applicant]
US 20150346495A1 · Welch et al. · 2015 [cited by applicant]
US 20160008625A1 · Barclay et al. · 2016 [cited by applicant]
US 20160011419A1 · Gao · 2016 [cited by applicant]
US 20160026253A1 · Bradski et al. · 2016 [cited by applicant]
US 20160055822A1 · Bell · 2016 [cited by applicant]
US 20160085302A1 · Publicover · 2016 [cited by examiner]
US 20160098861A1 · Sisbot et al. · 2016 [cited by applicant]
US 20160109709A1 · Osterhout · 2016 [cited by applicant]
US 20160131905A1 · Takahashi et al. · 2016 [cited by applicant]
US 20160131912A1 · Border · 2016 [cited by examiner]
US 20160161740A1 · Bar-Zeev et al. · 2016 [cited by applicant]
US 20160187651A1 · Border et al. · 2016 [cited by applicant]
US 20160203359A1 · Liechtenstein · 2016 [cited by applicant]
US 20160270656A1 · Samec et al. · 2016 [cited by applicant]
US 20170109936A1 · Powderly et al. · 2017 [cited by applicant]
US 20170206691A1 · Harrises et al. · 2017 [cited by applicant]
US 20170308262A1 · Murase · 2017 [cited by applicant]
US 20170323485A1 · Samec et al. · 2017 [cited by applicant]
US 20170353714A1 · Poulad et al. · 2017 [cited by applicant]
US 20170365101A1 · Samec et al. · 2017 [cited by applicant]
US 20180136486A1 · Macnamara et al. · 2018 [cited by applicant]
US 20180288405A1 · Allen et al. · 2018 [cited by applicant]
US 20190011703A1 · Robaina et al. · 2019 [cited by applicant]
US 20200183171A1 · Robaina · 2020 [cited by applicant]
CN 102540463 · 2012 [cited by applicant]
CN 102566756 · 2012 [cited by applicant]
CN 103975268 · 2014 [cited by applicant]
CN 104919398 · 2015 [cited by applicant]
CN 105009598 · 2015 [cited by applicant]
CN 105283834 · 2016 [cited by applicant]
CN 105518570 · 2016 [cited by applicant]
JP H06335007A · 1994 [cited by applicant]
JP H07261728 · 1995 [cited by applicant]
JP 2001013941 · 2001 [cited by applicant]
JP 2002041234 · 2002 [cited by applicant]
JP 2005059660 · 2005 [cited by applicant]
JP 2009288529 · 2009 [cited by applicant]
JP 2010164782A · 2010 [cited by applicant]
JP 2014504413 · 2014 [cited by applicant]
JP 2014137396 · 2014 [cited by applicant]
JP 2014211651 · 2014 [cited by applicant]
JP 2015153426 · 2015 [cited by applicant]
JP 2016509705 · 2016 [cited by applicant]
JP 2016517036A · 2016 [cited by applicant]
JP 2016519322A · 2016 [cited by applicant]
KR 1020150023792 · 2015 [cited by applicant]
WO WO2012130774A1 · 2012 [cited by examiner]
WO 2013179426A1 · 2013 [cited by applicant]
WO WO2015077766 · 2015 [cited by applicant]
WO WO2015081313 · 2015 [cited by applicant]
WO WO2015161307 · 2015 [cited by applicant]
WO WO2016056162 · 2016 [cited by applicant]
WO WO2016056227 · 2016 [cited by applicant]
WO WO2016062642 · 2016 [cited by applicant]
WO WO2018022523 · 2018 [cited by applicant]
“Augmented Reality,” Wikipedia, Internet Archive (Sep. 18, 2009) (accessed at https://web.archive.org/web/20090918034414/https://en.wikipedia.org/wiki/Augmented_reality), 8 pages. [cited by applicant]
“Autorefractor,” https://en.wikipedia.org/wiki/Autorefractor (accessed via: https://web.archive.org/web/20150930192812/en.wikipedia.org/wiki/Autorefractor), 2015, 2 pages. [cited by applicant]
“Basic Psychological Process—B.Sc. in Counseling Psychology Core Course”, University of Calicut, School of Distance Education, (2011 Admission onwards) in 189 pages. URL: http://www.universityofcalicut.info/syl/CP1B01Ba… [cited by applicant]
“Big Tree Measuring Methods”, Nature Nova Scotia, archived Jun. 6, 2016, in 5 pages. URL: https://web.archive.org/web/20160606110252/http://www.naturelst.net/naturens/files/tree_measure.html. [cited by applicant]
“Eye Chromatism”, Telescope-Optics.net, archived Mar. 10, 2016, in 5 pages. URL: http://web.archive.org/web/20160310131809/http://www.telescopeoptics.net/eye_chromatism.htm. [cited by applicant]
“Eye Intensity Response, Contrast Sensitivity”, Telescope-Optics.net, archived Jul. 13, 2016, in 12 pages. URL: https://web.archive.org/web/20160713100211/https://www.telescopeoptics.net/eye_intensity_response.htm. [cited by applicant]
“Eye Spectral Response”, Telescope-Optics.net, retrieved Dec. 8, 2015, in 7 pages. URL: https://www.telescope-optics.net/eye_spectral_response.htm. [cited by applicant]
“Fiducial Marker Placement”, RadiologyInfo; Reviewed Jan. 26, 2016; printed Oct. 23, 2017 in 5 pages; URL: http://www.radiologyinfo.org/en/info.cfm?pg-fiducial-marker. [cited by applicant]
“How do I calculate total acceleration from the x, y, and z g-force values given by an accelerometer?”, Quora, answered Dec. 28, 2015, accessed Jul. 11, 2017, in 4 pages. URL: https://www.quora.com/How-do-I-calculate-to… [cited by applicant]
“Mako—Robotic-Arm Assisted Surgery”, Stryker, printed Oct. 23, 2017, in 2 pages; URL: https://www.stryker.com/us/en/portfolios/orthopaedics/joint-replacement/mako-robotic-arm-assisted-surgery.html. [cited by applicant]
“Positron emission tomography”, as archived Jun. 17, 2016, in 14 pages. https://web.archive.org/web/20160617062029/http://en.wikipedia.org/wiki/Positron_emission_tomography. [cited by applicant]
“Scientists Literally Stretch Brain to Map Details”, Medgadget, Aug. 1, 2016, as archived Aug. 4, 2017, in 7 pages. URL: https://web.archive.org/web/20170804161858/https://www.medgadget.com/2016/08/scientists-stretch-br… [cited by applicant]
“Super-resolution 3D Microscopy of Whole Cells Opens New Window for Scientists” Jul. 14, 2016 in 3 pages. https://www.medgadget.com/2016/07/super-resolution-3d-microscopy-whole-cells-opens-new-window-scientists.html. [cited by applicant]
“Telescope images are degraded by the blurring effects of the atmosphere and by light pollution,” Chapter 6-3 https://web.archive.org/web/20160726162320/http://www.public.asu.edu/˜atpcs/atpcs/Univ10e/chapter06-03.html, … [cited by applicant]
“The Telescopic Eye”, Telescope-Optics.net, archived Jul. 21, 2016, in 6 pages. URL: https://web.archive.org/web/20160721003510/https://www.telescope-optics.net/eye.htm. [cited by applicant]
Anthony, S., “MIT releases open-source software that reveals invisible motion and detail in video”, Extreme Tech, Feb. 28, 2013, as accessed Aug. 4, 2017, in 8 pages. [cited by applicant]
ARToolKit: https://web.archive.org/web/20051013062315/http://www.hitl.washington.edu:80/artoolkit/documentation/hardware.htm, archived Oct. 13, 2005, 3 pages. [cited by applicant]
Azuma, “A Survey of Augmented Reality,” Teleoperators and Virtual Environments 6, 4 (Aug. 1997), pp. 355-385. https://web.archive.org/web/20010604100006/http://www.cs.unc.edu/˜azuma/ARpresence.pdf. [cited by applicant]
Azuma, “Predictive Tracking for Augmented Realty,” TR95-007, Department of Computer Science, UNC-Chapel Hill, NC, Feb. 1995, 262 pages. [cited by applicant]
Barrilleaux, J., “Experiences and observations in applying augmented reality to live training,” VWSim'99: The Virtual Worlds and Simulation Conference, Society for Computer Simulation International (1999) (accessed at h… [cited by applicant]
Benford, S., et al., “Understanding and Constructing Shared Spaces with Mixed-Reality Boundaries,” ACM Transactions on Computer-Human Interaction, 5(3):185-223, Sep. 1998. [cited by applicant]
Billinghurst, M., et al., “The MagicBook: a transitional AR interface,” Computers & Graphics, 25(5):745-753, Oct. 2001, 14 pages. [cited by applicant]
Bimber, et al., “Spatial Augmented Reality—Merging Real and Virtual Worlds,” 2005 https://web.media.mit.edu/˜raskar/book/BimberRaskarAugmentedRealityBook.pdf, 393 pages. [cited by applicant]
Binocular Visual Field, URL: https://www.med.kindai.ac.jp/optho/english/olaboratory.html, Jun. 2016, in 1 page. [cited by applicant]
Borghino, D., “Hi-tech glasses aim to assist the blind with directions and obstacle detection”, New Atlas, May 22, 2014, in 4 pages. URL: http://newatlas.com/stereoscopic-ultrasound-gps-ai-glasses-blind-assistance/32166… [cited by applicant]
Brunelli, Roberto, “Template Matching Techniques in Computer Vision, Theory and Practice,” pp. 25-28, published 2009. [cited by applicant]
Carmigniani, J., et al., “Augmented reality technologies, systems and applications,” Multimedia Tools and Applications, 51(1):341-377, Dec. 2010. [cited by applicant]
Carter, T. et al., “UltraHaptics: Multi-Point Mid-Air Haptic Feedback for Touch Surfaces”, UIST '13 Proceedings of the 26th Annual ACM Symposium on User Interface Software and Technology, Oct. 2013, in 10 pages. URL: ht… [cited by applicant]
Chadd, L., “Natural Health—Bio-Identical Hormons”, downloaded Oct. 23, 2017 in 7 pages; URL: https://www.lynnchadd.com/services/#thermal-imaging-breast-health. [cited by applicant]
Cornish et al., “Distribution of short-wavelength-sensitive cones in human fetal and postnatal retina: early development of spatial order and density profiles,” Science Direct, Vision Research 44, 2004, in 8 pages. [cited by applicant]
Curawave: “About The Curawave (MRgFUS) Treatment”, Curawave, as archived Aug. 16, 2017, in 4 pages. URL: https://web.archive.org/web/20170816232429/http://USA.uterine-fibroids.org/about-mrgfus/. [cited by applicant]
Cutting, James E., and Peter M. Vishton. “Perceiving layout and knowing distances: The integration, relative potency, and contextual use of different information about depth,” In Perception of space and motion, pp. 69-1… [cited by applicant]
Declaration of Dr. Joseph Gabbard Under 37 C.F.R. § 1.132 including Appendix A (List of Materials Considered) and Appendix B (Curriculum Vitae) and Incorporated Claim Charts CC-A1-A9, CC-B1-B4, and CC-C1-C2, Oct. 7, 202… [cited by applicant]
Edwards-Stewart, A., et al., “Classifying Different Types of Augmented Reality Technology,” Annual Review of CyberTherapy and Telemedicine, vol. 14, pp. 199-202 (Jan. 2016). [cited by applicant]
EKO Devices “Duo electronic stethoscope + EKG—Telemedicine Solutions”, retrieved Oct. 23, 2017 in 2 pages; URL: https://ekodevices.com/. [cited by applicant]
European Extended Search Report, re EP Application No. 17835070.8, dated Mar. 11, 2020, 16 pages. [cited by applicant]
Evena Medical: “Eyes-On Glasses 3.0—Medical Smart Glasses”, Evena Medical Products, archived Mar. 18, 2016 in 6 pages; URL: https://web.archive.org/web/20160318125251/http://evenamed.com/eyes-on-glasses/. [cited by applicant]
Gilliam, C., “Can VR Justify QHD and 4K Displays?”, DA Developers, Feb. 11, 2015, as archived Aug. 4, 2017, in 9 pages. URL: https://web.archive.org/web/20170804164547/https://www.da-developers.com/can-vr-justify-qhd-an… [cited by applicant]
Gilliam, C., “Can VR Justify QHD and 4K Displays?”, XDA Developers, Feb. 11, 2015, https://web.archive.org/web/20150903005229/https://www.xda-developers.com/can-vr-justify-qhd-and-4k-displays/ as archived Sep. 3, 2015 i… [cited by applicant]
Green, M., “Night Vision”, Visual E pert, as archived Aug. 4, 2017, in 8 pages. URL: https://web.archive.org/web/20170804160954/http://www.visualexpert.com/Resources/nightvision.html. [cited by applicant]
Griffith, D., “‘Digital neurotherapeutic’ developed at UC Davis Mind Institute”, Daily Democrat, Jun. 24, 2016, as archived Aug. 3, 2017, in 3 pages. URL: https://web.archive.org/web/20170803232850/http://www.dailydemoc… [cited by applicant]
Harrison, W., “Eye movement targets are released from visual crowding”, Will J Harrison, Mar. 13, 2013, as archived Aug. 4, 2017, in 3 pages. URL: https://web.archive.org/web/20170804165524/http://willjharrison.com/2013… [cited by applicant]
Hoste, L., and Signer, B., “Expressive Control of Indirect Augmented Reality During Live Music Performances,” Proceedings of NIME 2013, 13th International Conference on New Interfaces for Musical Expression (May 2013), … [cited by applicant]
Hua, et al. “Depth-fused multi-focal plane displays enable accurate depth perception,” Proceedings of SPIE vol. 7849 (2010), 10 pages. [cited by applicant]
Human Vision, Physics Including Human Applications, Chapter 20, pp. 440-452, 1978. http://physics.doane.edu/hpp/Resources/Fuller3/pdf/F3Chapter_20A. [cited by applicant]
HyperPhysics, “The Color-Sensitive Cones”, archived Jul. 10, 2016 in 2 pages. URL: https://web.archive.org/web/20160710054648/http://hyperphysics.phy-astr.gsu.edu/hbase/vision/colcon.html. [cited by applicant]
International Preliminary Report on Patentability for PCT Application No. PCT/US2017/043555, issued Jan. 29, 2019, 6 pages. [cited by applicant]
International Search Report and Written Opinion for PCT Application No. PCT/US2017/043555, mailed Oct. 6, 2017, 16 pages. [cited by applicant]
Jacob, “Eye Tracking in Advanced Interface Design,” Human-Computer Interaction Lab Naval Research Laboratory, Washington, D.C. / paper/ in Virtual Environments and Advanced Interface Design, ed. by W. Barfield and T.A. … [cited by applicant]
Jose M. Otero, “Measurement of Accommodation in Dim Light and in Darkness by Means of the Purkinje Images,” J. Opt. Soc. Am., 43, 925-925 (1953). [cited by applicant]
KODAK: “KODAK In-vivo Multispectral System FX-”, published by CareStream Health, Inc. (Jan. 2008) Brochure in 8 pages; URL: http://clab.ajums.ac.ir/_clab/documents/Multrispectral_in-Vivo_Imaging.pdf. [cited by applicant]
Kolb, H., “Part XIII: Facts and Figures concerning the human retina by Helga Kolb”, Webvision, archived Jul. 2, 2016, in 6 pages. URL: https://web.archive.org/web/20160702134518/http://webvision.med.utah.edu/book/part-x… [cited by applicant]
Kramida, G., “Resolving the vergence-accommodation conflict in head-mounted displays,” IEEE Transactions on Visualization and Computer Graphics, 2016, 22(7): 20 pages. [cited by applicant]
Liu, S., et al., “An optical see-through head mounted display with addressable focal planes,” In Mixed and Augmented Reality, ISMAR 2008, 7th IEEE/ACM International Symposium, pp. 33-42, 2008 (“Liu”). [cited by applicant]
Milgram, P. and Kishino, F., “A Taxonomy of Mixed Reality Visual Displays,” IEICE Transactions on Information and Systems, E77-D(12):1321-1329, Dec. 1994. [cited by applicant]
MIT, “MIT Media lab Medical Mirror”, YouTube, published Oct. 1, 2010, in 2 pages. URL: https://www.youtube.com/watch?v=LyWnvAWEbWE. [cited by applicant]
Mrovlje, J. et al., “Distance measuring based on stereoscopic pictures”, 9th International PhD Workshop on Systems and Control: Young Generation Viewpoint, Oct. 2008, in 6 pages. URL: http://dsc.ijs.si/files/papers/S101… [cited by applicant]
Murch, G. “Physiological Principles for the Effective Use of Color”, IEEE CG&A, pp. 49-54, Nov. 1984, https://education.siggraph.org/static/HyperGraph/color/coloreff.htm. [cited by applicant]
Notice of Allowance in Korean Appln. No. 10-2019-7005428, dated Jul. 27, 2022, 4 pages (with English translation). [cited by applicant]
Office Action in Canadian Appln. No. 3031771, dated Aug. 8, 2023, 5 pages. [cited by applicant]
Office Action in Chinese Appln. No. 201780059031.6, dated Mar. 24, 2022, 17 pages (with English translation). [cited by applicant]
Office Action in European Appln. No. 17835070.8, dated Jun. 24, 2022, 8 pages. [cited by applicant]
Office Action in Japanese Appln. No. 2019-503537, dated Jun. 27, 2022, 7 pages (with English translation). [cited by applicant]
Office Action in Japanese Appln. No. 2022-039089, dated Feb. 15, 2023, 8 pages (with English translation). [cited by applicant]
Office Action in Korean Appln. No. 10-2019-7005428, dated Jan. 28, 2022, 18 pages (with English translation). [cited by applicant]
Office Action in Korean Appln. No. 10-2022-7037695, dated Dec. 16, 2022, 23 pages (with English translation). [cited by applicant]
Orlosky, J.,“Adaptive Display of Virtual Content for Improving Usability and Safety in Mixed and Augmented Reality”, Osaka University Knowledge Archive, 2016, 81 pages. [cited by applicant]
Philips expands Benelux home healthcare offering with innovative home medication dispensing service, Philips News center, Feb. 20, 2014. https://www.philips.com/aw/about/news/archive/standard/news/press/2014/20140220-Ph… [cited by applicant]
Puig et al., “Difference in Visual Processing Assessed by Eye Vergence Movements,” PLOS One, Voume 8 Issue 9 (Sep. 2013), 8 pages. [cited by applicant]
Schmalstieg, D. and Höllerer, T., “Augmented Reality: Principles and Practice” (2016), 84 pages. [cited by applicant]
Schmalstieg, D., et al., “The Studierstube Augmented Reality Project,” DBLP, 11(1):333-54, Feb. 2002 (“Studierstube Augmented Reality Project”). [cited by applicant]
Sharma, N. et al., “Automated Medical Image Segmentation Techniques”, J. Med. Phys, 35(1), Jan.-Mar. 2010, pp. 3-14, URL: http://www.ncbi.nlm.nih.gov/pmc/articles/PMC2825001/. [cited by applicant]
Skarbez, R., et al., “Revisiting Milgram and Kishino's Reality-Virtuality Continuum,” Frontiers in Virtual Reality, vol. 2 (Mar. 2021), 8 pages. [cited by applicant]
Snowbrains.com, Illustration Blind Spot, as archived May 18, 2015 in 1 page, https://web.archive.org/web/20150518201835/https://snowbrains.com/wp-content/uploads/2013/07/illustration-blind-spot.gif. [cited by applicant]
Sunnex, “The Role of Blue Light in the Pathogenesis of AMD”, archived Jul. 8, 2016, in 20 pages. https://web.archive.org/web/20160708064257/https://www.sunnexbiotech.com/therapist/bluelightandamd.html. [cited by applicant]
Tanriverdi and Jacob, “Interacting With Eye Movements in Virtual Environments,” Department of Electrical Engineering and Computer Science, Tufts University, Medford, MA—paper/Proc. ACM CHI 2000 Human Factors in Computin… [cited by applicant]
The New York Times, “MIT Computer Program Reveals Invisible Motion in Video | The New York Times”, You Tube, published Feb. 27, 2013, as archived Sep. 8, 2017, in 2 pages. https://web.archive.org/web/20170906180629/http… [cited by applicant]
Ultrahaptics, “A Remarkable Connection with Technology”, retrieved Oct. 31, 2017 in 8 pages. URL: http://ultrahaptics.com/. [cited by applicant]
USPTO Order Granting Reexamination in U.S. Appl. No. 90/019,120, dated Nov. 30, 2022, 41 pages. [cited by applicant]
USPTO Order Granting Reexamination in U.S. Appl. No. 90/019,137, dated Jan. 12, 2023, 21 pages. [cited by applicant]
Wandell, Brian A., “The Photoreceptor Mosaic,” Foundations of Vision, Chapter 3. Stanford University, 1995, in 18 pages. https://foundationsofvision.stanford.edu/chapter-3-the-photoreceptormosaic. [cited by applicant]
Webvision, Graph of Rod and Cone Densities along the Horizontal Meridian, as archived Jul. 22, 2016, in 1 page, https://web.archive.org/web/20160722010221/https://webvision.med.utah.edu/imageswv/Ostergr.jpeg. [cited by applicant]
Webvision, Isodensity Maps of Cone Densities (X1000) in the Human Retina, as archived Jul. 22, 2016, in 1 page, https://web.archive.org/web/20160722041212/http://webvision.med.utah.edu/imageswv/Curciopl.jpeg. [cited by applicant]
Weier, Martin. “Perception-driven rendering: techniques for the efficient visualization of 3D scenes including view-and gaze-contingent approaches,” PhD diss., Universität des Saarlandes, 2019, 44 pages. [cited by applicant]
Wikipedia Blind spot (vision), archived Jun. 9, 2016, in 2 pages. URL: https://web.archive.org/web/20160609224858/https:en.wikipedia.org/wiki/Blind_spot(vision). [cited by applicant]
Wikipedia, “Crowding”, as archived Oct. 17, 2015 in 1 page. https://web.archive.org/web/20151017044343/https://en.wikipedia.org/wiki/Crowding. [cited by applicant]
Wikipedia: “Adaptation”, as archived Nov. 17, 2015, in 6 pages. https://web.archive.org/web/20151117222707/https://en.m.wikipedia.org/wiki/Adaptation_(eye). [cited by applicant]
Wikipedia: “Angular resolution”, Wikipedia, as archived Jul. 17, 2015, in 5 pages. URL: https://web.archive.org/web/20150717175037/https://en.wikipedia.org/wiki/Angular_resolution. [cited by applicant]
Wikipedia: “Peripheral vision”, as archived Jul. 21, 2016, in 6 pages. https://web.archive.org/web/20160721034342/https://en.wikipedia.org/wiki/Peripheral_vision. [cited by applicant]
Wikipedia: “Photopic vision”, Wikipedia, as archived May 6, 2015, in 2 pages. URL: https://web.archive.org/web/20150506191509/https://en.wikipedia.org/wiki/Photopic_vision. [cited by applicant]
Office Action in Japanese Appln. No. 2023-111338, dated May 10, 2024, 8 pages (with English translation). [cited by applicant]
Extended European Search Report in European Appln. No. 24157237.9, dated Mar. 27, 2024, 24 pages. [cited by applicant]
Office Action in Korean Appln. No. 10-2023-7030543, mailed on Jan. 6, 2025, 14 pages (with English translation). [cited by applicant]