IP Library › Granted Patent US 11,853,108
Granted Patent B2
US 11,853,108 · App. 16/757,826 · Granted Dec 26, 2023

Electronic apparatus for searching related image and control method therefor

Inventors: Hyeon Mok Ko (Suwon-si, KR); Hyung Rai Oh (Suwon-si, KR); Hong Chul Kim (Suwon-si, KR); Silas Jeon (Suwon-si, KR); In-Chul Hwang (Suwon-si, KR)
Assignee: SAMSUNG ELECTRONICS CO., LTD.
G06F16/583G06F16/532G06F16/535G06F16/5854G06F16/5866G06F18/214G06N20/00
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,853,108
App. No.
16/757,826
Granted
Dec 26, 2023
Kind
B2
Abstract

The disclosure relates to an artificial intelligence (AI) system using a machine learning algorithm such as deep learning and the like, and an application thereof. In particular, there is provided a control method for an electronic apparatus for searching for an image, the method comprising displaying an image comprising at least one object, detecting a user input for selecting an object, recognizing an object displayed at a point at which the user input is detected and acquiring information regarding the recognized object by using a recognition model trained to acquire information regarding an object, displaying a list including the information regarding the object, and based on one piece of information being selected from the information regarding the object included in the list, providing a related image by searching for the related image based on the selected information.

Claims (48)

1. A control method of an electronic apparatus for searching for an image, the method comprising:

displaying an image;

detecting a user input for the image;

obtaining recognition information of an object displayed at a location of the image corresponding to the user input using an object recognition model;

obtaining context information of the object using a context recognition model;

displaying a list based on the recognition information and the context information, the list including one or more selection items each including text for identifying both a type of object based on the recognition information and a context of the object based on the context information;

based on a selection of one of the selection items included in the list, searching for an image related to the image based on the recognition information and the context information of the selected item; and

providing the related image.

2. The method according to claim 1 , wherein the acquiring comprises:

identifying an object region based on a coordinate value of a location at which the user input is detected; and

recognizing the object by inputting the identified object region to the object recognition model.

3. The method according to claim 1 , wherein the recognition information comprises at least one of a type of the object, a color of the object, a size of the object, a name of the object, or a gender of the object, and the context information comprises one or more of a movement of the object, an emotion of the object, or clothes of the object.

4. The method according to claim 1 , wherein the electronic apparatus stores a plurality of images and tag information matched to the plurality of images, and

the searching comprises searching for images having tag information corresponding to the recognition information and the context information from among the plurality of images stored in the electronic apparatus.

5. The method according to claim 4 , further comprising:

based on receiving a user command for generating an album by selecting at least one of the related images, generating an album including at least one related image.

6. The method according to claim 4 , further comprising:

obtaining an image;

obtaining information regarding an object included in the obtained image by inputting the acquired image to a recognition model trained to predict object information; and

storing the obtained information regarding the object as tag information to be matched with the acquired image.

7. The method according to claim 1 , wherein the providing comprises:

generating a query including the recognition information and the context information;

transmitting the query to an external search server;

receiving a search result including a related image corresponding to the recognition information and the context information from the search server based on the query; and

providing the search result.

8. The method according to claim 1 , wherein the searching comprises:

generating a search formula based on information from first and second different objects in the image; and

searching for a related image using the search formula.

9. The method according to claim 1 , wherein the list is displayed in a vicinity of the object.

10. An electronic apparatus comprising:

a display;

a user input;

a processor electrically connected to the display and the user input; and

a memory electrically connected to the processor,

wherein the processor is configured to:

detect in an input from the user input for an image displayed on the display,

obtain recognition information of an object displayed at a location of the image corresponding to the user input using an object recognition model,

obtain context information of the object using a context recognition model;

control the display to display a list based on the recognition information and the context information, the list including one or more selection items each including text for identifying both a type of object based on the recognition information and a context of the object based on the context information, and

based on a selection of one of the selection items included in the list via the user input, search for an image related to the image based on the recognition information and the context information of the selected item, and

provide the related image.

11. The apparatus according to claim 10 , wherein the processor is configured to;

identify an object region based on a coordinate value of a location at which the user input is detected, and

recognize the object by inputting the identified object region to the object recognition model.

12. The apparatus according to claim 10 , wherein the recognition information comprises at least one of a type of the object, a color of the object, a size of the object, a name of the object, or a gender of the object, and the context information comprises one or more of a movement of the object, an emotion of the object, or clothes of the object.

13. The apparatus according to claim 10 , wherein the memory stores a plurality of images and tag information matched to the plurality of images, and

wherein the processor is configured to:

search for images having tag information corresponding to the recognition information and the context information from among the plurality of images stored in the electronic apparatus.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Apr 21, 2020
From: KO, HYEON MOK; OH, HYUNG RAI; KIM, HONG CHUL; JEON, SILAS; HWANG, IN-CHUL
To: SAMSUNG ELECTRONICS CO., LTD.
Reel/Frame 052450/0964 →
Priority Claims (1)
KR 10-2017-0140805 · Oct 27, 2017 · national
Continuity (1)
Related Publication 20210191971A1 · Jun 24, 2021