Granted Patent B2

US 12,579,755 · App. 17/791,058 · Granted Mar 17, 2026

Inventors: Long Qian (Baltimore, MD); Peter Kazanzides (Baltimore, MD); Mathias Unberath (Baltimore, MD); Tianyu Song (Baltimore, MD)

Assignee: THE JOHNS HOPKINS UNIVERSITY

G06T19/006G02B25/004G06T5/80G06T19/20G06V10/25G06T2219/2004G06T2219/2016

Abstract

Claims (72)

1 . A computer-implemented method for displaying augmented reality (AR) content within an AR device coupled to one or more physical loupe lenses, the method comprising:

obtaining calibration parameters defining a magnified display portion within a display of the AR device, wherein the magnified display portion aligns with a boundary encompassing the one or more physical loupe lenses;

receiving the AR content for display within the AR device; and

rendering the AR content within the display, wherein rendering the AR content comprises:

identifying a magnified portion of the AR content to be displayed within the magnified display portion,

rendering the magnified portion of the AR content within the magnified display portion at a magnification consistent with a magnification of real-world imagery as seen by a user through the one or more physical loupe lenses,

rendering a non-magnified portion of the AR content outside of the magnified display portion, wherein the AR content appears magnified within a view of the user through the one or more physical loupe lenses, and wherein the AR content appears to be non-magnified outside of the view of the user through the one or more physical loupe lenses, and

rendering the AR content within an occluded portion of the display, wherein the occluded portion of the display comprises a boundary of the magnified display portion.

2 . The method of claim 1 , further comprising:

obtaining system calibration parameters, the obtaining the system calibration parameters comprising:

receiving an input calibration image from a calibration camera;

measuring distortion caused by the one or more physical loupe lenses based on the received input calibration image; and

storing system calibration parameters based on the measured distortion,

wherein the rendering the magnified portion of the AR content includes applying the system calibration parameters.

3 . The method of claim 1 , wherein the obtaining calibration parameters comprises:

presenting a calibration interface having loupe boundaries displayed within the AR device;

receiving user input, via the calibration interface, defining the magnified display portion; and

receiving user input defining alignment information.

4 . The method of claim 3 , wherein the alignment information is obtained using at least one of:

a single point active alignment method (SPAAM);

eye-based tracking calibration;

Interaction-Free Display Calibration (INDICA); and

Display-relative Calibration (DRC).

5 . The method of claim 1 , wherein rendering the magnified portion of the AR content comprises projection mapping the AR content to the magnified display portion.

6 . The method of claim 1 , wherein the calibration parameters further define a projective mapping for the magnified display portion and a projective mapping for a non-magnified display portion.

7 . A computer program product comprising a non-transitory computer readable storage medium having program instructions embodied therewith, the program instructions being executable by an augmented reality (AR) device and causing the AR device to perform operations comprising:

obtaining calibration parameters defining a magnified display portion within a display of the AR device, wherein the magnified display portion aligns with a boundary encompassing one or more physical loupe lenses coupled to the AR device;

receiving AR content for display within the AR device; and

rendering the AR content within the display, wherein the rendering the AR content comprises:

identifying a magnified portion of the AR content to be displayed within the magnified display portion,

rendering the magnified portion of the AR content within the magnified display portion at a magnification consistent with a magnification of real-world imagery as seen by a user through the one or more physical loupe lenses,

rendering a non-magnified portion of the AR content outside of the magnified display portion, wherein the AR content appears magnified within a view of the user through the one or more physical loupe lenses, and wherein the AR content appears to be non-magnified outside of the view of the user through the one or more physical loupe lenses, and

rendering the AR content within an occluded portion of the display, wherein the occluded portion of the display comprises a boundary of the magnified display portion.

8 . The computer program product of claim 7 , the operations further comprising: obtaining system calibration parameters, the obtaining the system calibration parameters comprising:

receiving an input calibration image from a calibration camera;

measuring distortion caused by the one or more physical loupe lenses based on the received input calibration image; and

storing system calibration parameters based on the measured distortion, wherein the rendering the magnified portion of the AR content includes applying the system calibration parameters.

9 . The computer program product of claim 7 , wherein the obtaining calibration parameters comprises:

presenting a calibration interface having loupe boundaries displayed within the AR device;

receiving user input, via the calibration interface, defining the magnified display portion; and

receiving user input defining alignment information.

10 . The computer program product of claim 9 , wherein the alignment information is obtained using at least one of:

a single point active alignment method (SPAAM);

eye-based tracking calibration;

Interaction-Free Display Calibration (INDICA); and

Display-relative Calibration (DRC).

11 . The computer program product of claim 7 , wherein rendering the magnified portion of the AR content comprises projection mapping the AR content to the magnified display portion.

12 . The computer program product of claim 7 , wherein the calibration parameters further define a projective mapping for the magnified display portion and a projective mapping for a non-magnified display portion.

13 . A system comprising:

a processor, a computer readable memory, a non-transitory computer readable storage medium associated with an augmented reality (AR) device, and program instructions executable by the AR device to cause the AR device to perform operations comprising:

obtaining calibration parameters defining a magnified display portion within a display of the AR device, wherein the magnified display portion aligns with a boundary encompassing one or more physical loupe lenses coupled to the AR device;

receiving AR content for display within the AR device; and

rendering the AR content within the display, wherein the rendering the AR content comprises:

identifying a magnified portion of the AR content to be displayed within the magnified display portion,

rendering the magnified portion of the AR content within the magnified display portion at a magnification consistent with a magnification of real-world imagery as seen by a user through the one or more physical loupe lenses,

rendering a non-magnified portion of the AR content outside of the magnified display portion, wherein the AR content appears magnified within a view of the user through the one or more physical loupe lenses, and wherein the AR content appears to be non-magnified outside of the view of the user through the one or more physical loupe lenses, and

rendering the AR content within an occluded portion of the display, wherein the occluded portion of the display comprises a boundary of the magnified display portion.

14 . The system of claim 13 , the operations further comprising:

obtaining system calibration parameters, the obtaining the system calibration parameters comprising:

receiving an input calibration image from a calibration camera;

measuring distortion caused by the one or more physical loupe lenses based on the received input calibration image; and

storing system calibration parameters based on the measured distortion,

wherein the rendering the magnified portion of the AR content includes applying the system calibration parameters.

15 . The system of claim 13 , wherein the obtaining calibration parameters comprises: presenting a calibration interface having loupe boundaries displayed within the AR device;

receiving user input, via the calibration interface, defining the magnified display portion; and

receiving user input defining alignment information.

16 . The system of claim 15 , wherein the alignment information is obtained using at least one of:

a single point active alignment method (SPAAM);

eye-based tracking calibration;

Interaction-Free Display Calibration (INDICA); and

Display-relative Calibration (DRC).

17 . The system of claim 13 , wherein rendering the magnified portion of the AR content comprises projection mapping the AR content to the magnified display portion.

Assignments (1)

Continuity (2)

References Cited (19)