IP Library › Granted Patent US 12,198,696
Granted Patent B2
US 12,198,696 · App. 17/946,487 · Granted Jan 14, 2025

Electronic device and operation method thereof

Inventors: Jungkeun Cho (Gyeonggi-do, KR); Jaeyung Yeo (Gyeonggi-do, KR)
Assignee: Samsung Electronics Co., Ltd.
G10L15/22G06F3/14G06F3/167G10L15/063G10L15/16G10L2015/223
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,198,696
App. No.
17/946,487
Granted
Jan 14, 2025
Kind
B2
Abstract

An electronic device and method are disclosed. The electronic device includes a display, input circuitry, a memory and a processor. The processor implements the method, including: display on the display a hint view including a hint, identifying a type of the hint, generating execution information for the hint based on the identified type, including a goal and a capsule, storing the hint with the generated execution information, receiving a user input and determining whether the input corresponds to the stored hint, when the input corresponds to the stored hint, execute the goal using the capsule, and displaying response data prompting update associated with the hint.

Claims (63)

1. An electronic device, comprising:

a display;

input circuitry;

a memory; and

a processor operatively connected to the display, the input circuitry and the memory,

wherein the memory stores instructions executable by the processor to cause the electronic device to:

display on the display a hint view, including at least one hint corresponding to a user utterance,

identify a type of the at least one hint displayed in the hint view,

generate execution information for the at least one hint, based on the identified type, wherein the execution information includes at least a goal indicative of a function to be executed, and a capsule indicative of a domain in which the goal is executed;

store the at least one hint with the generated execution information in the memory;

receive a user input through the input circuitry, and determine whether the received user input corresponds to the stored at least one hint;

when the received user input corresponds the at least one hint, execute the goal using the capsule included in the execution information; and

display response data prompting an update on the processing of the user utterance corresponding to the at least one hint.

2. The electronic device of claim 1 ,

wherein the instructions are further executable by the processor to cause the electronic device:

receive the user input including a voice signal.

3. The electronic device of claim 1 , wherein

identifying the type of the at least one hint is further based on whether the at least one hint includes a single utterance command.

4. The electronic device of claim 3 , wherein the instructions are further executable by the processor to cause the electronic device:

when the at least one hint includes a single utterance command, identify that the hint is of an utterance command type,

wherein when the at least one hint is of the utterance command type, the goal and the capsule of the utterance command are generated in the execution information.

5. The electronic device of claim 3 , wherein the instructions are further executable by the processor to cause the electronic device:

when the at least one hint includes a plurality of utterance commands, identify the at least one hint is of a quick command type,

wherein when the at least one hint is of the quick command type, the execution information includes a plurality of goals and a plurality of identified capsules respectively corresponding to the plurality of utterance commands.

6. The electronic device of claim 1 , wherein the instructions are further executable by the processor to cause the electronic device:

in response to receiving an attachment request to display the hint view on the display, store the at least one hint in the memory; and

in response to receiving a detachment request to remove the hint view from the display, delete the at least one hint from the memory.

7. The electronic device of claim 1 , wherein the instructions are further executable by the processor to cause the electronic device:

store the at least one hint included in the hint view in response to an event in which the hint view is focused to be set to the highest priority with respect to a user input among the displayed elements; and

delete the at least one hint included in the hint view in response to another event in which the hint view is unfocused.

8. The electronic device of claim 1 , wherein the instructions are further executable by the processor to cause the electronic device:

determine whether the user input matches the at least one hint, based on text data converted based on the user input.

9. The electronic device of claim 1 , wherein the instructions are further executable by the processor to cause the electronic device:

when the user input corresponds with the at least one hint, change at least one of the capsule and the goal of the at least one hint, using the execution information.

10. The electronic device of claim 1 , wherein the response data includes an icon, which is capable of setting a default capsule executing an operation corresponding to a second user input as the capsule included in the execution information, when the second user input, which is identical to the utterance command included in the hint or capable of being recognized to be identical to the utterance command included in the hint, is received.

11. A method of an electronic device, the method comprising:

displaying, on a display included in the electronic device or operatively connected to the electronic device, a hint view, including at least one hint corresponding to a user utterance;

identifying, via at least one processor, a type of the at least one hint included in the hint view;

generating execution information for the at least one hint based on the identified type, wherein the execution information includes at least a goal indicative of a function to be executed, and a capsule indicative of a domain in which the goal is executed;

storing the at least one hint with the execution information of the hint in a memory;

receiving a user input through input circuitry and determining whether the receiver user input corresponds to the stored at least one hint;

when the received user input corresponds to the at least one hint, executing the goal included using the AI capsule included in the execution information; and

displaying response data prompting an update on the processing of the user utterance corresponding to the at least one hint.

12. The method of claim 11 , wherein the user input includes a voice signal.

13. The method of claim 11 , wherein

identifying the type of the at least one hint is further based on whether the at least one hint includes a single utterance command.

14. The method of claim 13 , further comprising:

when the at least one hint includes a single utterance command, identifying that the hint is of an utterance command type,

wherein when the at least one hint is of the utterance command type, the goal and the capsule of the utterance command are generated in the execution information.

15. The method of claim 13 , further comprising:

when the at least one hint includes a plurality of utterance commands, identifying the at least one hint is of a quick command type,

wherein when the at least one hint is of the quick command type, the execution information includes a plurality of goals and a plurality of identified capsules respectively corresponding to the plurality of utterance commands.

16. The method of claim 11 , further comprising:

in response to receiving an attachment request to display the hint view on the display, storing the at least one hint in the memory; and

in response to receiving a detachment request to remove the hint view from the display, deleting the at least one hint from the memory.

17. The method of claim 11 , further comprising:

storing the at least one hint included in the hint view in response to an event in which the hint view is focused to be set to the highest priority with respect to a user input among the displayed elements; and

deleting the at least one hint included in the hint view in response to another event in which the hint view is unfocused.

18. The method of claim 11 , further comprising:

determining whether the user input matches the at least one hint, based on text data converted based on the user input.

19. The method of claim 11 , further comprising:

when the user input corresponds with the at least one hint, changing at least one of the capsule and the goal of the at least one hint, using the execution information.

20. The method of claim 11 , wherein the response data is data including an icon, which is capable of setting a default capsule executing an operation corresponding to a second user input as the capsule included in the execution information, when the second user input, which is identical to the utterance command included in the hint or capable of being recognized to be identical to the utterance command included in the hint, is received.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Sep 16, 2022
From: CHO, JUNGKEUN; YEO, JAEYUNG
To: SAMSUNG ELECTRONICS CO., LTD.
Reel/Frame 061122/0164 →
Priority Claims (1)
KR 10-2021-0128050 · Sep 28, 2021 · national
Continuity (2)
Continuation PCTKR2022013769 · Sep 15, 2022
Related Publication 20230094274A1 · Mar 30, 2023
References Cited (35)
US 9070366B1 · Mathias et al. · 2015 [cited by applicant]
US 10440167B2 · Choi et al. · 2019 [cited by applicant]
US 10546023B2 · Ni · 2020 [cited by applicant]
US 10547729B2 · Choi et al. · 2020 [cited by applicant]
US 10581824B2 · Ducker et al. · 2020 [cited by applicant]
US 10976997B2 · Kim et al. · 2021 [cited by applicant]
US 11145019B2 · De Cremer · 2021 [cited by examiner]
US 11146670B2 · Choi et al. · 2021 [cited by applicant]
US 11176936B2 · Mathias et al. · 2021 [cited by applicant]
US 11232155B2 · Ni · 2022 [cited by applicant]
US 11303623B2 · Ducker et al. · 2022 [cited by applicant]
US 11494443B2 · Ahn et al. · 2022 [cited by applicant]
US 11650785B1 · Voss · 2023 [cited by examiner]
US 20150312236A1 · Ducker · 2015 [cited by examiner]
US 20170332064A1 · Martineau · 2017 [cited by examiner]
US 20180315425A1 · Mathias et al. · 2018 [cited by applicant]
US 20190394511A1 · Kalaboukis et al. · 2019 [cited by applicant]
US 20200043476A1 · Hwang · 2020 [cited by applicant]
US 20200192684A1 · Woo · 2020 [cited by examiner]
US 20210020168A1 · Dame · 2021 [cited by examiner]
US 20210280180A1 · Skobeltsyn et al. · 2021 [cited by applicant]
US 20210295839A1 · Xu · 2021 [cited by applicant]
US 20220157309A1 · Skobeltsyn et al. · 2022 [cited by applicant]
US 20220399030A1 · Zhou · 2022 [cited by examiner]
KR 1020180096483A · 2018 [cited by applicant]
KR 1020190011031A · 2019 [cited by applicant]
KR 1020190019442A · 2019 [cited by applicant]
KR 1020190092997A · 2019 [cited by applicant]
KR 1020200007882A · 2020 [cited by applicant]
WO 2019221894A1 · 2019 [cited by applicant]
M. Migliardi and M. Gaudina, “Active Personal Information Manager: A System for Human Memory Support, ” 2011 International Conference on Complex, Intelligent, and Software Intensive Systems, Seoul, Korea (South), 2011, … [cited by examiner]
G. Chen and M. Kandemir, “Optimizing embedded applications using programmer-inserted hints,” Proceedings of the ASP-DAC 2005. Asia and South Pacific Design Automation Conference, 2005., Shanghai, China, 2005, pp. 157-16… [cited by examiner]
G. Chen and M. Kandemir, “Optimizing embedded applications using programmer-inserted hints,” Proceedings of the ASP-DAC 2005. Asia and South Pacific Design Automation Conference, 2005., Shanghai, China, 2005, pp. 157-16… [cited by examiner]
M. Migliardi and M. Gaudina, “Active Personal Information Manager: A System for Human Memory Support,” 2011 International Conference on Complex, Intelligent, and Software Intensive Systems, Seoul, Korea (South), 2011, p… [cited by examiner]
International Search Report dated Jan. 12, 2023. [cited by applicant]