Granted Patent B2

US 12,373,027 · App. 18/344,921 · Granted Jul 29, 2025

Inventors: Rohit Prasad (Lexington, MA); Arushan Rajasekaram (North Bend, WA); Furqan Muhammad Khan (Culver City, CA); Rajiv M Reddy (Bellevue, WA)

Assignee: Amazon Technologies, Inc.

G06F3/013G06F3/0481G06F3/04842G06F3/165G06F3/167G06F9/451G06T7/70G10L15/22G06T2207/30201G10L15/193G10L2015/223

Abstract

Claims (103)

1. A computer-implemented method comprising:

presenting, by a display of a first device, a first graphical user interface (GUI) element;

capturing, by a camera of the first device, first image data representing a user;

processing the first image data to determine a gaze of the user is directed, at a first time, at the display;

determining the first GUI element corresponds to video content corresponding to a first application;

based at least in part on the first GUI element corresponding to the video content, selecting a first time threshold from a plurality of time thresholds, the first time threshold corresponding to playing of video content in response to gaze detection;

capturing, by the camera, second image data;

processing the second image data to determine the gaze of the user is directed, at a second time after the first time, at the display;

based at least in part on the first time and the second time, determining a first duration corresponding to the gaze of the user being directed at the display;

determining that the first duration satisfies the first time threshold;

receiving video data corresponding to the video content; and

in response to the first duration satisfying the first time threshold, causing the first device to present the video content using the video data.

2. The computer-implemented method of claim 1 , further comprising:

capturing, by the camera, third image data;

processing the third image data to determine the gaze of the user is directed, at a third time between the first time and the second time, at the display;

determining a second duration between the first time and the third time; and

based at least in part on the second duration, presenting, by the display, a visual indicator representing detection of the gaze of the user.

3. The computer-implemented method of claim 1 , further comprising, prior to processing the second image data:

sending a request for the video data to a second device associated with the first application;

receiving a first portion of the video data; and

storing, by the first device, the first portion of the video data,

wherein causing the first device to present the video content comprises processing the first portion of the video data.

4. A computer-implemented method comprising:

receiving first content to display on a device;

determining that the first content corresponds to a gaze initiated playback function;

based on the first content, determining a first time threshold to be used to initiate gaze related playback of the first content;

presenting, by a display of a first device, a first graphical user interface (GUI) element corresponding to first content;

capturing, by a camera, first image data representing a user;

processing the first image data to determine a first gaze of the user is directed at the display;

determining a first duration corresponding to the first gaze;

determining that the first duration satisfies the first time threshold; and

in response to the first duration satisfying the first time threshold, causing playback of the first content.

5. The computer-implemented method of claim 4 , wherein the first content corresponds to video content comprising image content and audio content and wherein causing playback of the first content comprises:

in response to the first duration satisfying the first time threshold, causing playback of the image content.

6. The computer-implemented method of claim 5 , further comprising:

receiving a user input corresponding to playback of the audio content; and

in response to the user input, causing playback of the audio content.

7. The computer-implemented method of claim 4 , further comprising:

determining user profile data corresponding to the user;

wherein determining the first time threshold is further based at least in part on the user profile data.

8. The computer-implemented method of claim 4 , further comprising:

determining the first GUI element corresponds to a first application;

determining first data associating the first gaze with the first application;

capturing, by a microphone, audio representing an utterance;

determining audio data representing the utterance; and

causing speech processing to be performed using the audio data and the first data.

9. The computer-implemented method of claim 4 , further comprising:

presenting, by the display, a visual indicator representing detection of the first gaze of the user.

10. The computer-implemented method of claim 4 , further comprising:

determining an identifier associated with the user,

wherein presenting the first GUI element is based at least in part on the identifier.

11. A system, comprising:

at least one processor; and

at least one memory comprising instructions that, when executed by the at least one processor, cause the system to:

present, by a display of a first device, a first graphical user interface (GUI) element corresponding to first content;

capture, by a camera, first image data representing a user;

process the first image data to determine a gaze of the user is directed to at the display;

determine a duration corresponding to the gaze;

determine user profile data corresponding to the user;

determine that the user profile data includes information identifying a first time threshold associated with the user;

based at least in part on the user profile data including the information, select the first time threshold for use in evaluating the duration;

determine the duration satisfies the first time threshold; and

in response to the duration satisfying the first time threshold, cause playback of the first content.

12. The system of claim 11 , wherein the first content corresponds to video content comprising image content and audio content and wherein the instructions that cause the system to cause playback of the first content comprise instructions that, when executed by the at least one processor, further cause the system to:

in response to the duration satisfying the first time threshold, causing playback of the image content.

13. The system of claim 12 , wherein the at least one memory further comprises instructions that, when executed by the at least one processor, further cause the system to:

receive a user input corresponding to playback of the audio content; and

in response to the user input, cause playback of the audio content.

14. The system of claim 11 , wherein the at least one memory further comprises instructions that, when executed by the at least one processor, further cause the system to:

select the first time threshold further based at least in part on the first content.

15. The system of claim 11 , wherein the at least one memory further comprises instructions that, when executed by the at least one processor, further cause the system to:

determine the first GUI element corresponds to a first application;

determine first data associating the gaze with the first application;

capture, by a microphone, audio representing an utterance;

determine audio data representing the utterance; and

cause speech processing to be performed using the audio data and the first data.

16. The system of claim 11 , wherein the at least one memory further comprises instructions that, when executed by the at least one processor, further cause the system to:

present, by the display, a visual indicator representing detection of the gaze of the user.

17. The system of claim 11 , wherein the at least one memory further comprises instructions that, when executed by the at least one processor, further cause the system to:

determine an identifier associated with the user,

wherein presentation of the first GUI element is based at least in part on the identifier.

18. The computer-implemented method of claim 4 , further comprising:

receiving second content to display on a device;

determining that the second content does not correspond to a gaze initiated playback function;

presenting, by the display, a second GUI element corresponding to second content, wherein the first image data was captured while the first GUI element and the second GUI element were presented simultaneously on the display; and

determining to cause playback of the first content rather than the second content based at least in part on the first content corresponding to a gaze initiated playback function and the second content not corresponding to a gaze initiated playback function.

19. The computer-implemented method of claim 4 , further comprising:

receiving second content to display on a device;

determining that the second content corresponds to a gaze initiated playback function;

based on the second content, determining a second time threshold, different than the first time threshold, to be used to initiate gaze related playback of the second content;

presenting, by the display, a second GUI element corresponding to the second content;

capturing, by the camera, second image data representing the user;

processing the second image data to determine a second gaze of the user is directed at the display;

determining a second duration corresponding to the second gaze;

determining that the second duration satisfies the second time threshold; and

in response to the second duration satisfying the second time threshold, causing playback of the second content.

20. The computer-implemented method of claim 19 , wherein:

determining the first time threshold further comprises:

determining that the first content corresponds to a first type of content, and

determining that the first type of content is associated with the first time threshold; and

determining the second time threshold further comprises:

determining that the second content corresponds to a second type of content different than the first type of content, and

determining that the second type of content is associated with the second time threshold.

Assignments (1)

Continuity (1)

References Cited (129)