Granted Patent B2

US 12,738,053 · App. 18/071,968 · Granted Sep 15, 2026

Inventors: Shefali Bhatnagar (Noida, IN); Akansha Sharma (Noida, IN); Amit Singh Bhadauria (Noida, IN); Pulkit Agarawal (Noida, IN); Vishnu Kumar Dwivedi (Noida, IN)

Assignee: SAMSUNG ELECTRONICS CO., LTD.

G06V20/20G06F3/0482G06F9/451G06T11/00G06V10/764

Abstract

Claims (99)

1 . A method for providing actionable content in an electronic device, the method comprising:

obtaining, by the electronic device, an image;

extracting, by the electronic device, scene information by analyzing a scene from the obtained image;

identifying, by the electronic device, at least one object from the scene information;

determining, by the electronic device, at least one scene description from the scene information and the at least one object;

determining, by the electronic device, a scene context based on the at least one object and the at least one scene description, wherein the scene context comprises activity related with the at least one object;

determining, by the electronic device, a user context including a temporal context or a network context;

displaying, by the electronic device, an augmented reality (AR) mode providing visual suggestion over the image;

identifying, by the electronic device, a user's input selecting the AR mode;

extracting, by the electronic device, a list of application functions based on the at least one object;

filtering, by the electronic device, application functions from the list of application functions based on the scene context and the user context;

prioritizing, by the electronic device, the filtered application functions based on a historical usage frequency of the filtered application functions;

displaying, by the electronic device, an interactive message corresponding to at least one of the prioritized application functions at a position corresponding to the at least one object;

receiving, by the electronic device, a user's selection on the displayed interactive message;

in response to receiving the user's selection on the displayed interactive message, displaying, by the electronic device and at a position corresponding to the interactive message, an application icon corresponding to an application configured to execute an application function corresponding to the displayed interactive message; and

launching, by the electronic device, the application configured to execute the application function corresponding to the displayed interactive message based on a user's selection from the displayed application icon.

2 . The method of claim 1 , wherein the filtered application functions are arranged in a descending order of priority.

3 . The method of claim 1 , wherein filtering the application functions comprises:

generating, by the electronic device, at least one augmented context based on the scene context and the user context, wherein the at least one augmented context indicates an overall digital representation of the scene with respect to the at least one object in the scene and the user context while viewing the scene; and

filtering, by the electronic device, the application functions from the list of application functions by applying the at least one augmented context to an artificial intelligence (AI) model.

4 . The method of claim 1 , wherein a size of the interactive message is varied based on a priority of the application function executed by the launched application.

5 . A method for providing augmented reality (AR) recommendations to a user by an electronic device, comprising:

displaying, by the electronic device, an image on a screen of the electronic device;

identifying, by the electronic device, at least one real world object in the image;

determining, by the electronic device, at least one function of at least one application installed in the electronic device based on the at least one identified real world object;

augmenting, by the electronic device, the at least one function of the at least one application as an AR recommendation over the image, the AR recommendation comprising an interactive message corresponding to the at least one function and that is displayed at a position corresponding to the at least one real world object,

receiving, by the electronic device, a selection of the displayed interactive message;

in response to receiving the selection of the displayed interactive message, displaying, by the electronic device, an application icon corresponding to the at least one application that is configured to execute at least one function corresponding to the interactive message; and

executing, by the electronic device, the at least one application based on a selection of the displayed application icon.

6 . The method of claim 5 , wherein determining the at least one function of the at least one application installed in the electronic device comprises:

determining, by the electronic device, an identifier of the at least one identified real world object;

matching, by the electronic device, the identifier of the at least one identified real world object with a predefined set of identifiers embedded with the at least one application; and

fetching, by the electronic device, the at least one function of the at least one application based on a match between the identifier of the at least one identified real world object and the predefined set of identifiers.

7 . The method of claim 5 , wherein augmenting the at least one function of the at least one application comprises:

classifying, by the electronic device, the at least one identified real world object to one or more object classes based on an attribute of the at least one identified real world object;

determining, by the electronic device, an intention of each function of the at least one application for each object class;

identifying, by the electronic device, a preferred data type linked to the intention of each function of the at least one application for each object class;

fetching, by the electronic device, data objects linked to the preferred data type;

fetching, by the electronic device, a message content mapped to each function of the at least one application;

generating, by the electronic device, a first list of messages with corresponding functions of the at least one application based on the mapped message content and the data objects for each object class;

determining, by the electronic device, a context of a scene in the image, wherein the context of the scene comprises an activity performed on each object class and an activity performed by each object class;

determining, by the electronic device, a context of the user of the electronic device in real time, wherein the context of the user comprises a temporal context, a spatial context, a network context, and a physiological context of the user;

generating, by the electronic device, at least one augmented context based on the context of the scene, and the context of the user for each object class, wherein the at least one augmented context indicates an overall digital representation of the scene with respect to the at least one real world object in the scene and the context of the user while viewing the scene;

filtering, by the electronic device, a second list of messages with corresponding functions of the at least one application from the first list of messages for each object class by applying the at least one augmented context to an artificial intelligence (AI) model;

determining, by the electronic device, a priority of functions of the at least one application corresponding to the second list of messages based on a historical usage frequency of the functions of the at least one application;

filtering, by the electronic device, a third list of messages with corresponding functions of the at least one application from the second list of messages for each object class based on the priority of the functions of the at least one application corresponding to the second list of messages; and

augmenting, by the electronic device, each message in the third list of messages as the AR recommendation over the image,

wherein the AR recommendation is configured to trigger a function associated with each message,

wherein a size of the AR recommendation is varied based on the priority of the function associated with each message, and

wherein a position of the AR recommendation is rendered based on the position of the at least one real world object in the image.

8 . The method of claim 7 , wherein generating at least one augmented context comprises:

generating, by the electronic device, hot encodings for the context of the scene, the context of the user and each object class; and

determining, by the electronic device, an augmented context vector that represents the at least one augmented context by determining a cross product of the hot encodings.

9 . The method of claim 7 , wherein filtering the second list of messages comprises:

obtaining, by the electronic device, an augmented context vector that represents the at least one augmented context;

determining, by the electronic device, a probability vector for each function of the at least one application by providing the augmented context vector and the first list of messages to the AI model;

identifying, by the electronic device, the functions of the at least one application with the probability vector matches to a threshold condition; and

generating, by the electronic device, the second list of messages with corresponding functions of the at least one application by sorting the identified functions of the at least one application with corresponding messages based on the probability vector.

10 . An electronic device for providing augmented reality (AR) recommendations to a user, comprising:

a screen;

a memory storing instructions;

a processor configured to execute the instructions to:

display an image on the screen,

identify at least one real world object in the image,

determine at least one function of at least one application installed in the electronic device based on the at least one identified real world object;

augment the at least one function of the at least one application as an AR recommendation over the image, the AR recommendation comprising an interactive message corresponding to the at least one function and that is displayed at a position corresponding to the at least one real world object,

receive a selection of the displayed interactive message;

in response to receiving the selection of the displayed interactive message, display an application icon corresponding to the at least one application that is configured to execute at least one function corresponding to the interactive message; and

execute the at least one application based on a selection of the displayed application icon.

11 . The electronic device of claim 10 , wherein the processor is configured to execute the instructions to determine the at least one function of the at least one application installed in the electronic device by:

determining an identifier of the at least one identified real world object;

matching the identifier of the at least one identified real world object with a predefined set of identifiers embedded with the at least one application; and

fetching the at least one function of the at least one application based on a match between the identifier of the at least one identified real world object and the predefined set of identifiers.

12 . The electronic device of claim 10 , wherein the processor is configured to execute the instructions to augment the at least one function of the at least one application by:

classifying the at least one identified real world object to one or more object classes based on an attribute of the at least one identified real world object;

determining an intention of each function of the at least one application for each object class; and

identifying a preferred data type linked to the intention of each function of the at least one application for each object class.

13 . The electronic device of claim 12 , wherein the processor is further configured to execute the instructions to augment the at least one function of the at least one application by:

fetching data objects linked to the preferred data type;

fetching a message content mapped to each function of the at least one application;

generating a first list of messages with corresponding functions of the at least one application based on the mapped message content and the data objects for each object class;

determining a context of a scene in the image, wherein the context of the scene comprises an activity performed on each object class and an activity performed by each object class;

determining a context of the user of the electronic device in real time, wherein the context of the user comprises a temporal context, a spatial context, a network context, and a physiological context of the user;

generating at least one augmented context based on the context of the scene, and the context of the user for each object class, wherein the at least one augmented context indicates an overall digital representation of the scene with respect to the at least one real world object in the scene and the context of the user while viewing the scene;

filtering a second list of messages with corresponding functions of the at least one application from the first list of messages for each object class by applying the at least one augmented context to an artificial intelligence (AI) model;

determining a priority of functions of the at least one application corresponding to the second list of messages based on a historical usage frequency of the functions of the at least one application;

filtering a third list of messages with corresponding functions of the at least one application from the second list of messages for each object class based on the priority of the functions of the at least one application corresponding to the second list of messages; and

augmenting each message in the third list of messages as the AR recommendation over the image,

wherein the AR recommendation is configured to trigger a function associated with each message,

wherein a size of the AR recommendation is varied based on the priority of the function associated with each message, and

wherein a position of the AR recommendation is rendered based on the position of the at least one real world object in the image.

14 . The electronic device of claim 13 , wherein generating at least one augmented context comprises:

generating hot encodings for the context of the scene, the context of the user and each object class; and

determining an augmented context vector that represents the at least one augmented context by determining a cross product of the hot encodings.

15 . The electronic device of claim 13 , wherein filtering the second list of messages comprises:

obtaining an augmented context vector that represents the at least one augmented context;

determining a probability vector for each function of the at least one application by providing the augmented context vector and the first list of messages to the AI model;

identifying the functions of the at least one application with the probability vector matches to a threshold condition; and

generating the second list of messages with corresponding functions of the at least one application by sorting the identified functions of the at least one application with corresponding messages based on the probability vector.

Assignments (1)

Priority Claims (1)

Continuity (2)

References Cited (51)