IP Library › Granted Patent US 12706072
Granted Patent B2
US 12706072 · App. 18/909,263 · Granted Aug 11, 2026

Using eye tracking for super resolution

Inventors: Jieyu Tan (Zhejiang, CN); Katerina Le Shiffer (San Ramon, CA); Siyuan Li (Zhejiang, CN); Kanghua Zhu (Zhejiang, CN)
Assignee: CISCO TECHNOLOGY, INC.
G09G5/391G02B27/0093G09G2340/0407
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12706072
App. No.
18/909,263
Granted
Aug 11, 2026
Kind
B2
Abstract

A method and system are provided for applying super-resolution to a video and/or shared content in an audio and video communication session. One or more video streams are obtained from one or more computing devices and are displayed on a display device. A layout of the displayed one or more video streams and their respective positions as displayed in the layout is obtained. The location, duration, and frequency of eye-gaze of a user associated to the display device is obtained via an eye-tracking module. A first position in the layout that matches the location of eye-gaze of the user is obtained and the corresponding duration of eye-gaze is compared to a first threshold time. Super-resolution is applied to at least one of the one or more video streams that corresponds to the first position when the duration of eye-gaze is longer than the first threshold time.

Claims (46)

1 . A method, comprising:

obtaining one or more video streams from one or more computing devices and displaying the one or more video streams on a display device;

acquiring a layout of the displayed one or more video streams and a position at which each of the one or more video streams is displayed in the layout;

obtaining, via an eye-tracking module, a location, a duration, and a frequency of eye-gaze of a user associated with the display device;

identifying a first position in the layout that matches the location of eye-gaze of the user and comparing the duration of eye-gaze of the user to a first threshold time;

applying super-resolution using an algorithm to a first video stream of the one or more video streams that corresponds to the first position when the duration of eye-gaze is longer than the first threshold time; and

comparing the duration of eye-gaze to a second threshold time and enabling a zoom-in of contents included in the first video stream when the duration of eye-gaze is longer than the second threshold time.

2 . The method of claim 1 , further comprising: comparing the frequency of eye-gaze to a threshold value and pinning, to the layout, the first video stream of the one or more video streams, that corresponds to the first position, when the frequency of eye-gaze is higher than the threshold value.

3 . The method of claim 2 , further comprising: un-pinning the first video stream that is pinned from the layout when the frequency of eye-gaze is lower than the threshold value.

4 . The method of claim 1 , further comprising: enabling a zoom-out of the contents included in the first video stream when the duration of eye-gaze is lower than the second threshold time.

5 . The method of claim 4 , wherein the contents included in the first video stream is at least one selected from a group including a person and a shared data.

6 . The method of claim 1 , wherein applying super-resolution comprises:

applying a first degree of super-resolution to the first video stream of the one or more video streams and applying a second degree of super-resolution to neighboring video streams of the first video stream in the layout, wherein the second degree of super-resolution is a lower resolution than the first degree of super-resolution.

7 . The method of claim 1 , wherein the one or more video streams comprise a text or an image.

8 . The method of claim 1 , wherein the one or more video streams comprise video of one or more users of the respective one or more computing devices.

9 . The method of claim 1 , wherein obtaining the one or more video streams comprises requesting one or more higher-resolution video streams from the one or more computing devices.

10 . The method of claim 1 , further comprising resizing the layout of the one or more video streams on the display device and remapping, via the eye-tracking module, the location, the duration, and the frequency of eye-gaze of the user associated with the display device.

11 . One or more non-transitory computer readable storage media encoded with instructions that, when executed by a computer processor, cause the computer processor to perform operations including:

acquiring one or more video streams and displaying the one or more video streams on a display device;

acquiring a layout of the displayed one or more video streams and a position at which each of the one or more video streams is displayed in the layout;

obtaining, via an eye-tracking module, a location, a duration, and a frequency of eye-gaze of a user associated with the display device;

identifying a first position in the layout that matches the location of eye-gaze of the user;

comparing the duration of eye-gaze of the user to a first threshold time;

applying super-resolution using an algorithm to a first video stream of the one or more video streams that corresponds to the first position when the duration of eye-gaze is longer than the first threshold time; and

comparing the duration of eye-gaze to a second threshold time and enabling a zoom-in of contents included in the first video stream when the duration of eye-gaze is longer than the second threshold time.

12 . The one or more non-transitory computer readable storage media of claim 11 , further comprising: comparing the frequency of eye-gaze to a threshold value and pinning, to the layout, the first video stream of the one or more video streams, that corresponds to the first position, when the frequency of eye-gaze is higher than the threshold value.

13 . The one or more non-transitory computer readable storage media of claim 12 , further comprising: un-pinning the first video stream that is pinned from the layout when the frequency of eye-gaze is lower than the threshold value.

14 . The one or more non-transitory computer readable storage media of claim 11 , further comprising: enabling a zoom-out of the contents included in the first video stream when the duration of eye-gaze is lower than the second threshold time.

15 . The one or more non-transitory computer readable storage media of claim 14 , wherein the contents included in the first video stream is at least one selected from a group including a person and a shared data.

16 . The one or more non-transitory computer readable storage media of claim 11 , wherein applying super-resolution comprises:

applying a first degree of super-resolution to the first video stream of the one or more video streams and applying a second degree of super-resolution to neighboring video streams of the first video stream in the layout, wherein the second degree of super-resolution is a lower resolution than the first degree of super-resolution.

17 . The one or more non-transitory computer readable storage media of claim 11 , further comprising resizing the layout of the one or more video streams on the display device and remapping, via the eye-tracking module, the location, the duration, and the frequency of eye-gaze of the user associated with the display device.

18 . An apparatus, comprising:

a display device configured to display one or more video streams obtained from one or more computing devices;

an eye-tracking module configured to provide a location, a duration, and a frequency of eye-gaze of a user associated with the display device; and

a processor configured to execute software instructions to:

acquire a layout of the displayed one or more video streams and a position at which each of the one or more video streams is displayed in the layout;

identify a first position in the layout that matches the location of eye-gaze of the user;

compare the duration of eye-gaze of the user to a first threshold time;

apply super-resolution using an algorithm to a first video stream of the one or more video streams corresponding to the first position when the duration of eye-gaze is longer than the first threshold time; and

compare the duration of eye-gaze to a second threshold time and enable a zoom-in of contents included in the first video stream when the duration of eye-gaze is longer than the second threshold time.

19 . The apparatus of claim 18 , wherein the processor is further configured to compare the frequency of eye-gaze to a threshold value and pin, to the layout, the first video stream of the one or more video streams, that corresponds to the first position, when the frequency of eye-gaze is higher than the threshold value.

20 . The apparatus of claim 18 , wherein the processor is further configured to enable a zoom-out of the contents included in the first video stream when the duration of eye-gaze is lower than the second threshold time, wherein the contents included in the first video stream is at least one selected from a group including a person and a shared data.

21 . The method of claim 1 , wherein at least a second video stream of the one or more video streams is not subject to applying super-resolution.

22 . The one or more non-transitory computer readable storage media of claim 11 , wherein at least a second video stream of the one or more video streams is not subject to applying super-resolution.

23 . The apparatus of claim 18 , wherein at least a second video stream of the one or more video streams is not subject to applying super-resolution.