Granted Patent B2

US 12,487,466 · App. 17/876,334 · Granted Dec 2, 2025

Inventors: Alexander Sourov (Seattle, WA); John James Robertson (Los Angeles, CA)

Assignee: Snap Inc.

G02B27/027G02B27/017G06T7/70G10L15/22G02B2027/0178G06T2207/30204G10L2015/223

Abstract

Claims (86)

1 . A method of presenting a magnified overlay with an eyewear device comprising a camera and a display, the method comprising:

capturing frames of video data using the camera;

registering a virtual marker associated with a marker location relative to a physical environment, wherein the virtual marker comprises a perimeter;

determining an eyewear device location relative to the marker location based on the captured frames of video data;

defining a frame relative to the display, wherein the frame is associated with a frame size and a frame position; and

presenting on the display the virtual marker at the marker location based on the eyewear device location;

presenting on the display within the frame a pointer aligned with the frame position wherein the pointer comprises a vector pointing toward the marker location,

calculating a range based on the frame position relative to at least one of the marker location or the perimeter; and

selectively presenting on the display within the frame a magnified overlay according to the range, wherein the magnified overlay comprises one or more of the captured frames of video data magnified by a magnification power.

2 . The method of claim 1 , wherein the step of presenting the magnified overlay further comprises:

generating the magnified overlay in accordance with the physical environment and the frame; and

presenting the magnified overlay within the frame as an overlay in a foreground relative to the physical environment.

3 . The method of claim 1 , wherein the eyewear device further comprises a microphone coupled to a speaker and a voice recognition module, and wherein the step of presenting the magnified overlay further comprises:

receiving human speech using the microphone;

converting the received human speech into frames of audio data;

identifying, using the voice recognition module, a command based on the converted frames of audio data; and

selectively presenting the magnified overlay in accordance with the identified command.

4 . The method of claim 1 , wherein the magnification power is selectively configurable, and wherein the step of presenting the magnified overlay further comprises:

detecting a segment traversed by a finger along a touchpad coupled to the eyewear device;

adjusting the magnification power based on the detected segment; and

applying the adjusted magnification power to the magnified overlay.

5 . The method of claim 1 , wherein the step of registering the virtual marker further comprises:

presenting on the display within the frame a reticle aligned with the frame position;

receiving an input when the reticle nearly coincides with a proposed marker location; and

setting the marker location and storing the virtual marker in accordance with the received input.

6 . The method of claim 1 , wherein the virtual marker comprises a target and a virtual object, and wherein presenting the pointer comprises:

presenting on the display the target and the virtual object according to the marker location and the eyewear device location;

presenting the pointer aligned with the frame position, wherein the pointer comprises a visual tether extending from the frame position to the marker location;

sizing the vector in length according to the range; and

presenting the vector in a direction according to a relative orientation between the frame position and the target.

7 . The method of claim 6 , wherein the step of presenting the pointer further comprises:

selectively presenting one or more of the vector, the visual tether, and the range based on at least one of the range, the frame size, or the frame position relative to the perimeter.

8 . The method of claim 1 , wherein the frame size is characterized by a predefined and configurable effective radius, and wherein the step of defining the frame further comprises:

detecting a segment traversed by a finger along a touchpad coupled to the eyewear device; and

adjusting the effective radius based on the detected segment.

9 . An eyewear device for presenting a magnified overlay, comprising:

a processor;

a memory;

a camera;

a display; and

programming in the memory, wherein execution of the programming by the processor configures the eyewear device to perform functions, including functions to:

capture frames of video data using the camera;

register a virtual marker associated with a marker location relative to a physical environment, wherein the virtual marker comprises a perimeter;

determine an eyewear device location relative to the marker location based on the captured frames of video data;

define a frame relative to the display, wherein the frame is associated with a frame size and a frame position; and

present on the display within the frame a pointer aligned with the frame position, wherein the pointer comprises a vector pointing toward the marker location,

calculate a range based on the frame position relative to at least one of the marker location or the perimeter;

selectively present on the display within the frame a magnified overlay according to the range, wherein the magnified overlay comprises one or more of the captured frames of video data magnified by a magnification power.

10 . The eyewear device of claim 9 , wherein the function to present the magnified overlay further comprises functions to:

generate the magnified overlay in accordance with the physical environment and the frame; and

present the magnified overlay within the frame as an overlay in a foreground relative to the physical environment.

11 . The eyewear device of claim 9 , wherein the eyewear device further comprises a microphone coupled to a speaker and a voice recognition module, and wherein the execution of the programming further configures the eyewear device to perform functions to:

receive human speech using the microphone;

convert the received human speech into frames of audio data;

identify, using the voice recognition module, a command based on the converted frames of audio data; and

selectively present the magnified overlay in accordance with the identified command.

12 . The eyewear device of claim 9 , wherein the eyewear device further comprises a touchpad, and wherein the function to present the magnified overlay further comprises functions to:

detect a segment traversed by a finger along the touchpad;

adjust the magnification power based on the detected segment; and

apply the adjusted magnification power to the magnified overlay.

13 . The eyewear device of claim 9 , wherein the function to register the virtual marker further comprises functions to:

present on the display within the frame a reticle aligned with the frame position;

receive an input when the reticle nearly coincides with a proposed marker location; and

set the marker location and store the virtual marker in accordance with the received input.

14 . The eyewear device of claim 9 , wherein the virtual marker comprises a target and a virtual object and wherein the execution of the programming further configures the eyewear device to perform functions to:

present on the display the target and the virtual object according to the marker location and the eyewear device location;

present the pointer aligned with the frame position, wherein the pointer comprises a visual tether extending from the frame position to the marker location;

size the vector in length according to the range; and

present the vector in a direction according to a relative orientation between the frame position and the target.

15 . The eyewear device of claim 14 wherein the function to present the pointer further comprises functions to:

selectively present one or more of the vector, the visual tether, and the range based on at least one of the range, the frame size, or the frame position relative to the perimeter.

16 . A non-transitory computer-readable medium storing program code that, when executed, is operative to cause a processor of an eyewear device to perform the steps of:

capturing frames of video data using a camera coupled to the eyewear device, wherein the eyewear device comprises a display;

registering a virtual marker associated with a marker location relative to a physical environment, wherein the virtual marker comprises a perimeter;

determining an eyewear device location relative to the marker location based on the captured frames of video data;

defining a frame relative to the display, wherein the frame is associated with a frame size and a frame position;

presenting on the display the virtual marker at the marker location based on the eyewear device location;

presenting on the display within the frame a pointer aligned with the frame position, wherein the pointer comprises a vector pointing toward the marker location,

calculating a range based on the frame position relative to at least one of the marker location or the perimeter; and

selectively presenting on the display, within the frame, and as an overlay in a foreground relative to the physical environment, a magnified overlay according to the range, wherein the magnified overlay comprises one or more of the captured frames of video data presented according to a magnification power.

17 . The non-transitory computer-readable medium storing program code of claim 16 , wherein the virtual marker comprises a target and a virtual object, and wherein the program code when executed is operative to cause the processor to perform the further steps of:

presenting on the display the target and the virtual object according to the marker location and the eyewear device location;

presenting the pointer aligned associated with the frame position, wherein the pointer comprises a visual tether extending along the range from the frame position to the marker location;

sizing the vector in length according to the range;

presenting the vector in a direction according to a relative orientation between the frame position and the target; and

selectively presenting one or more of the vector, the visual tether, and the range based on at least one of the range, the frame size, or the frame position relative to the perimeter.

Assignments (1)

Continuity (1)

References Cited (27)