Granted Patent B2

US 12,524,461 · App. 18/239,401 · Granted Jan 13, 2026

Inventors: Susanto Sen (Karnataka, IN); Charishma Chundi (Andhra Pradesh, IN)

Assignee: Adeia Guides Inc.

G06F16/632G06F16/433G06F16/438G06V10/80G06V40/20G06V40/28G10L15/19G10L15/22G10L15/30G10L2015/223

Abstract

Claims (95)

1 . A computer-implemented method, comprising:

receiving input from a user via a user input interface of a media device;

processing the input to identify a particular pronoun;

identifying a gesture made by the user;

processing an image associated with the gesture to determine a plurality of identities of a plurality of entities, respectively, in the image, displayed on a display of the media device;

determining, based on the plurality of identities, a respective pronoun for each entity of the plurality of entities;

determining, from among the plurality of entities, a particular entity having a pronoun that corresponds to the particular pronoun identified based on the input, wherein the particular entity is displayed on the display of the media device;

querying a database based on the input and the particular entity;

based on querying the database, receiving at least one identifier of at least one content item; and

generating for presentation, using the media device, the at least one identifier of the at least one content item.

2 . The method of claim 1 , wherein the input is a voice input, the particular entity is a second entity, and the method further comprises:

receiving, at the media device, a selection of a first entity currently being displayed on the display of the media device; and

processing the voice input to identify a search operator,

wherein querying the database comprises constructing a search query based on the identified search operator, the first entity and the second entity.

3 . The method of claim 1 , wherein the image is captured by an imaging sensor, and identifying the gesture made by the user comprises determining, based on the image, motion of the user.

4 . The method of claim 1 , wherein the image is captured by an imaging sensor, and determining the particular entity associated with the gesture further comprises:

determining a direction of the gesture; and

determining that the direction of the gesture corresponds to the image, the image depicting a real-world scene proximate to the user.

5 . The method of claim 4 , wherein determining the particular entity associated with the gesture further comprises:

performing image processing of the image to determine a plurality of portions of the image that respectively correspond to the plurality of entities;

based on the direction of the gesture, extrapolating a path of the gesture to a portion of the image; and

determining as the particular entity an entity of the plurality of entities associated with the portion of the image that the path intersects.

6 . The method of claim 4 , wherein the imaging sensor is a first imaging sensor, the image captured by the first imaging sensor is a first image, and the method further comprises:

determining that a second image captured by a second imaging sensor depicts a different perspective of the real-world scene than a perspective of the real-world scene depicted in the first image;

extrapolating a first path from the direction of the gesture in the first image;

extrapolating a second path from the direction of the gesture in the second image;

identifying a point at which the first path crosses the second path; and

determining as the particular entity an entity of the plurality of entities associated with the point of the image at which the first path crosses the second path.

7 . The method of claim 4 , wherein the imaging sensor is a first imaging sensor, the image captured by the first imaging sensor is a first image, and the method further comprises:

determining that a second image, captured by a second imaging sensor that is facing a second direction, depicts an area in which the user made the gesture;

performing image processing of the second image to identify the gesture;

extrapolating a first path from the direction of the gesture in the second image;

calculating, based on a position and an angle of the first imaging sensor and a position and an angle of the second imaging sensor, a second path in the first image corresponding to the first path;

performing image processing of the first image to determine a plurality of portions of the first image that respectively correspond to the plurality of entities; and

determining as the particular entity an entity of the plurality of entities associated with the portion of the image that the second path intersects.

8 . The method of claim 1 , wherein the media device is a first media device, the method further comprising:

generating, for presentation at a second media device proximate to the first media device, a content item,

wherein the image associated with the gesture corresponds to a frame of the content item being presented at the second media device.

9 . The method of claim 8 , wherein the particular entity is a first entity, the content item is a first content item, and the method further comprises:

generating, for presentation at the first media device while the first content item is being presented at the second media device, a second content item,

wherein the input is a voice input, and the voice input comprises a reference to a second entity depicted in a frame of the second media device, the querying of the database being further based on the second entity.

10 . The method of claim 1 , wherein determining, based on the plurality of identities, the respective pronoun for each entity of the plurality of entities comprises:

determining a first pronoun for a first entity of the plurality of entities; and

determining a second pronoun for a second entity of the plurality of entities;

wherein determining the particular entity having a pronoun that corresponds to the particular pronoun is based on the first pronoun for the first entity of the plurality of entities and the second pronoun for a second entity of the plurality of entities.

11 . A computer-implemented system, comprising:

input/output (I/O) circuitry configured to:

receive input from a user via a user input interface of a media device;

control circuitry configured to:

process the input to identify a particular pronoun;

identify a gesture made by the user;

process an image associated with the gesture to determine a plurality of identities of a plurality of entities, respectively, in the image, the image displayed on a display of the media device;

determine, based on the plurality of identities, a respective pronoun for each entity of the plurality of entities;

determine, from among the plurality of entities, a particular entity having a pronoun that corresponds to the particular pronoun identified based on the input, wherein the particular entity is displayed on the display of the media device; and

query a database based on the input and the particular entity,

wherein the I/O circuitry is further configured to:

based on querying the database, receive at least one identifier of at least one content item; and

generate for presentation, using the media device, the at least one identifier of the at least one content item.

12 . The system of claim 11 , wherein:

the input is a voice input, the particular entity is a second entity;

the I/O circuitry is further configured to receive, at the media device, a selection of a first entity currently being displayed on the display of the media device; and

the control circuitry is further configured to:

process the voice input to identify a search operator; and

query the database by constructing a search query based on the identified search operator, the first entity and the second entity.

13 . The system of claim 11 , wherein the image is captured by an imaging sensor, and the control circuitry is configured to identify the gesture made by the user by determining, based on the image, motion of the user.

14 . The system of claim 11 , wherein the image is captured by an imaging sensor, and the control circuitry is configured to determine the particular entity associated with the gesture by:

determining a direction of the gesture; and

determining that the direction of the gesture corresponds to the image, the image depicting a real-world scene proximate to the user.

15 . The system of claim 14 , wherein the control circuitry is configured to determine the particular entity associated with the gesture by:

performing image processing of the image to determine a plurality of portions of the image that respectively correspond to the plurality of entities;

based on the direction of the gesture, extrapolating a path of the gesture to a portion of the image; and

determining as the particular entity an entity of the plurality of entities associated with the portion of the image that the path intersects.

16 . The system of claim 14 , wherein the imaging sensor is a first imaging sensor, the image captured by the first imaging sensor is a first image, and the control circuitry is further configured to:

determine that a second image captured by a second imaging sensor depicts a different perspective of the real-world scene than a perspective of the real-world scene depicted in the first image;

extrapolate a first path from the direction of the gesture in the first image;

extrapolate a second path from the direction of the gesture in the second image;

identify a point at which the first path crosses the second path; and

determine as the particular entity an entity of the plurality of entities associated with the point of the image at which the first path crosses the second path.

17 . The system of claim 14 , wherein the imaging sensor is a first imaging sensor, the image captured by the first imaging sensor is a first image, and the control circuitry is further configured to:

determine that a second image, captured by a second imaging sensor that is facing a second direction, depicts an area in which the user made the gesture;

perform image processing of the second image to identify the gesture;

extrapolate a first path from the direction of the gesture in the second image;

calculate, based on a position and an angle of the first imaging sensor and a position and an angle of the second imaging sensor, a second path in the first image corresponding to the first path;

perform image processing of the first image to determine a plurality of portions of the first image that respectively correspond to the plurality of entities; and

determine as the particular entity an entity of the plurality of entities associated with the portion of the image that the second path intersects.

18 . The system of claim 11 , wherein the media device is a first media device, and the control circuitry is further configured to:

generate, for presentation at a second media device proximate to the first media device, a content item,

wherein the image associated with the gesture corresponds to a frame of the content item being presented at the second media device.

19 . The system of claim 18 , wherein the particular entity is a first entity, the content item is a first content item, and the control circuitry is further configured to:

generate, for presentation at the first media device while the first content item is being presented at the second media device, a second content item,

wherein the input is a voice input, and the voice input comprises a reference to a second entity depicted in a frame of the second media device, the querying of the database being further based on the second entity.

20 . The system of claim 11 , wherein the control circuitry is configured, when determining the respective pronoun for each entity of the plurality of entities, to:

determine a first pronoun for a first entity of the plurality of entities; and

determine a second pronoun for a second entity of the plurality of entities;

wherein determining the particular entity having a pronoun that corresponds to the particular pronoun is based on the first pronoun for the first entity of the plurality of entities and the second pronoun for a second entity of the plurality of entities.

Assignments (2)

Continuity (3)

References Cited (33)