Granted Patent B2

US 12,517,575 · App. 18/065,436 · Granted Jan 6, 2026

Inventors: Jason Todd Spencer (Rocklin, CA); Seth Raphael (Portland, OR); Sofien Bouaziz (Los Gatos, CA)

Assignee: GOOGLE LLC

G06F3/013G06V10/751

Abstract

Claims (68)

1 . A computer-implemented method, comprising:

capturing, by an image sensor of a wearable computing device, a first image of a portion of an ambient environment within a field of view of the image sensor, the first image including a representation of a first object in the ambient environment;

outputting, by a display device of the wearable computing device, the first image in a display area of the display device;

tracking, by at least one gaze tracking device of the wearable computing device, a gaze of a user of the wearable computing device;

detecting a fixation of the gaze in response to detecting movement of the gaze having at least one of a magnitude or a frequency that meets a threshold; and

identifying an object of focus of the fixation of the gaze, including:

in response to detecting the fixation of the gaze within the display area corresponding to the first image, identifying the first image as the object of focus; and

in response to detecting the fixation of the gaze outside of the display area, identifying a second object in the ambient environment, outside of the display area, as the object of focus, and outputting a second image including a representation of the second object in the ambient environment identified as the object of focus.

2 . The computer-implemented method of claim 1 , wherein outputting the second image includes terminating display of the first image.

3 . The computer-implemented method of claim 1 , further comprising:

capturing, by the image sensor of the wearable computing device, a plurality of images of the ambient environment, the plurality of images including the first image and the second image;

outputting, by the display device of the wearable computing device, the plurality of images as the plurality of images are captured; and

maintaining display of an image, of the plurality of images, in response to detecting fixation of the gaze of the user directed at the image.

4 . The computer-implemented method of claim 3 , further comprising:

detecting the fixation of the gaze within the image; and

identifying a pixel area within the image corresponding to the fixation of the gaze.

5 . The computer-implemented method of claim 4 , wherein tracking the gaze includes:

identifying a fixation point of the fixation of the gaze at an intersection of the gaze with the image; and

identifying the pixel area at the intersection of the fixation of the gaze with the image.

6 . The computer-implemented method of claim 4 , further comprising:

analyzing an area surrounding the pixel area;

identifying features within the area surrounding the pixel area based on the analyzing;

recognizing at least one of the features; and

matching the pixel area to a corresponding object in the ambient environment based on the at least one of the features.

7 . The computer-implemented method of claim 6 , wherein recognizing the at least one of the features includes at least one of:

recognizing the at least one of the features based on a search of an external database of previously stored images; or

recognizing the at least one of the features based on known features of the ambient environment.

8 . The computer-implemented method of claim 3 , wherein capturing the plurality of images includes capturing portions of the ambient environment within a field of view of the image sensor as at least one of a position or an orientation of the wearable computing device changes in the ambient environment.

9 . The computer-implemented method of claim 1 , wherein the wearable computing device is a head-mounted wearable computing device including a frame, with the image sensor, the display device, and the at least one gaze tracking device mounted on the frame, the at least one gaze tracking device tracking an eye gaze of a user of the head-mounted wearable computing device, and the display device outputting a plurality of images such that the plurality of images and the ambient environment are visible through a lens portion of the head-mounted wearable computing device.

10 . A computer-implemented method, comprising:

capturing, by an image sensor of a wearable computing device, a plurality of images of an ambient environment;

outputting, in a display area of a display device of the wearable computing device, the plurality of images as the plurality of images are captured;

tracking, by at least one gaze tracking device of the wearable computing device, a gaze of a user of the wearable computing device;

detecting a fixation of the gaze in response to detecting movement of the gaze having at least one of a magnitude or a frequency that meets a threshold; and

identifying an object of focus of the fixation of the gaze, including:

in response to detecting the fixation of the gaze within the display area of an image of the plurality of images, identifying a representation of an object within the image as the object of focus; and

in response to detection of the fixation of the gaze outside of the display area, identifying an object in the ambient environment as the object of focus and outputting an image including the object in the ambient environment identified as the object of focus.

11 . The computer-implemented method of claim 10 , further comprising:

maintaining display of the image including the object in the ambient environment;

detecting fixation of the gaze within the image; and

identifying a pixel area within the image corresponding to the fixation of the gaze.

12 . The computer-implemented method of claim 11 , wherein tracking the gaze includes:

identifying a fixation point of the fixation of the gaze at an intersection of the fixation of the gaze with the image; and

identifying the pixel area at the intersection of the fixation of the gaze with the image.

13 . The computer-implemented method of claim 11 , further comprising:

analyzing an area surrounding the pixel area;

identifying features within the area surrounding the pixel area based on the analyzing;

recognizing at least one of the features; and

matching the pixel area to a corresponding object in the ambient environment based on the at least one of the features.

14 . The computer-implemented method of claim 13 , wherein recognizing the at least one of the features includes at least one of:

recognizing the at least one of the features based on a search of an external database of previously stored images; or

recognizing the at least one of the features based on known features of the ambient environment.

15 . The computer-implemented method of claim 10 , wherein capturing the plurality of images includes capturing portions of the ambient environment within a field of view of the image sensor as at least one of a position or an orientation of the wearable computing device changes in the ambient environment.

16 . The computer-implemented method of claim 10 , wherein the wearable computing device is a head-mounted wearable computing device including a frame, with the image sensor, the display device, and the at least one gaze tracking device mounted on the frame, the at least one gaze tracking device tracking an eye gaze of a user of the head-mounted wearable computing device, and the display device outputting the plurality of images such that the plurality of images and the ambient environment are visible through a lens portion of the head-mounted wearable computing device.

17 . A computer-implemented method, comprising:

capturing an image of a portion of an ambient environment within a field of view of an image sensor of a computing device;

outputting the image in a display area of a display device of the computing device;

tracking a gaze of a user of the computing device;

detecting a fixation of the gaze of the user; and

identifying an object of focus of the fixation of the gaze, including:

in response to detecting the fixation of the gaze within a pixel area of the display area corresponding to the image, identifying the image as the object of focus; and

in response to detection of the fixation of the gaze outside of the display area, identifying an object in the ambient environment, outside of the image, as the object of focus, and replacing the image with a new image including the object in the ambient environment identified as the object of focus.

18 . The computer-implemented method of claim 17 , wherein the computing device is a head mounted wearable computing device including at least one gaze tracking device tracking eye gaze of the user of the computing device, and wherein identifying the image as the object of focus includes:

identifying a fixation point of the fixation of the gaze at an intersection of the gaze with the image;

identifying a pixel area at the intersection of the fixation of the gaze with the image;

analyzing an area surrounding the pixel area;

identifying features within an area surrounding the pixel area; and

matching the pixel area to a corresponding object in the ambient environment based on the at least one of the features.

Assignments (1)

Continuity (2)

References Cited (9)