Granted Patent B2

US 12,405,662 · App. 18/290,673 · Granted Sep 2, 2025

Inventors: Andrea Fanelli (Oakland, CA); Evan David Gitterman (Palo Alto, CA); Nathan Carl Swedlow (Oakland, CA); Alex Brandmeyer (Berkeley, CA); McGregor Steele Joyner (Alameda, CA); Scott Daly (Kalama, WA); Poppy Anne Carrie Crum (Oakland, CA)

Assignee: Dolby Laboratories Licensing Corporation

G06F3/013G06F3/012G06F3/04845G06F3/0487H04S7/303G06F2203/04806

Abstract

Claims (51)

1. A method comprising:

acquiring a set of voltage signals from a set of electrodes arranged in proximity to ears of a user;

based on the set of voltage signals, determining an electro-oculography (EOG) gaze vector in ego-centric coordinates;

determining a head pose of the user in display coordinates, using a sensor device worn by the user;

combining the EOG gaze vector and the head pose to obtain a gaze vector in display coordinates; and

determining a gaze point by calculating an intersection of the gaze vector and an imaging surface having a known position in display coordinates.

2. The method according to claim 1 , further comprising:

calibrating the sensor device to obtain a position of the sensor device in display coordinates.

3. The method according to claim 2 , wherein the calibrating includes:

displaying a graphical element on the imaging surface; and

receiving user input confirming the user is looking at a location on the imaging surface corresponding to the graphical element.

4. The method according to claim 2 , wherein the user-worn sensor device is synchronized with a second sensor device fixedly arranged relative to at least a portion of a display system including the imaging surface.

5. The method according to claim 2 , wherein the calibrating includes:

determining a distance between the user and the imaging surface, optionally using one or several sensors in the display system.

6. The method according claim 1 , further comprising: performing an on-line statistical calibration including statistical analysis of the gaze point over time and knowledge of expected user viewing patterns.

7. The method of claim 1 , further comprising:

receiving image data for display on the imaging surface;

determining a gaze depth based at least in part on depth information associated with the gaze point;

calculating a relative depth associated with a first portion of the image data as the difference between depth information associated with the first portion of the image data and the gaze depth; and

generating modified image data by modifying pixel data associated with the first portion of the image data in accordance with a function of the relative depth associated with the first portion of the image data.

8. The method according to claim 7 , wherein modifying pixel data involves altering one or more of: hue, brightness, gamma, contrast, sharpness, blur, and spatial filtering of the pixel data.

9. The method of claim 1 , further comprising:

receiving image data for display on the imaging surface;

determining a gaze depth based at least in part on depth information associated with the gaze point;

based at least in part on the gaze point and the gaze depth, identifying at least one audio object associated with a current point of interest; and

modifying the audio objects such that the identified audio objects are modified differently than other audio objects.

10. The method according to claim 9 , wherein modifying the identified audio objects involves altering one of: volume, loudness, and frequency distribution of the identified audio objects.

11. The method of claim 1 , further comprising:

receiving image data for display on the imaging surface;

monitoring over time the gaze point on the imaging surface;

determining an average gaze position and a gaze radius;

comparing the gaze radius with a radius threshold; and

in accordance with a determination that the gaze radius is smaller than the radius threshold, applying a zoom of the image data.

12. The method according to claim 11 , further comprising:

determining a smallest distance between the average gaze position and one or more edges of the imaging surface;

comparing the smallest distance with a distance threshold; and

in accordance with a determination that the smallest distance is less than the distance threshold, applying an offset off the image data to increase the smallest distance.

13. The method according to claim 11 , further comprising: low-pass filtering the offset or the zoom before application to the image data.

14. A system comprising:

a set of electrodes arranged in proximity to ears of a user and configured to acquire a set of voltage signals;

an electro-oculography (EOG) processing unit for determining an EOG gaze vector in ego-centric coordinates based on the set of voltage signals;

a user-worn sensor device for determining a head pose of the user in display coordinates;

a processing unit configured to:

combine the EOG gaze vector and the head pose to obtain a gaze vector in display coordinates; and

determine a gaze point by calculating an intersection of the gaze vector and an imaging surface having a known position in display coordinates.

15. A non-transitory computer readable medium storing computer program code configured to perform, when executed on a computer processor, the steps:

acquiring a set of voltage signals from a set of electrodes arranged in proximity to ears of a user;

based on the set of voltage signals, determining an electro-oculography (EOG) gaze vector in ego-centric coordinates;

determining a head pose of the user in display coordinates, using a sensor device worn by the user;

combining the EOG gaze vector and the head pose to obtain a gaze vector in display coordinates; and

determining a gaze point by calculating an intersection of the gaze vector and an imaging surface having a known position in display coordinates.

Assignments (1)

Priority Claims (1)

Continuity (2)

References Cited (54)