IP Library › Granted Patent US 11,508,375
Granted Patent B2
US 11,508,375 · App. 16/903,918 · Granted Nov 22, 2022

Electronic apparatus including control command identification tool generated by using a control command identified by voice recognition identifying a control command corresponding to a user voice and control method thereof

Inventors: Woojei Choi (Suwon-si, KR); Minkyong Kim (Suwon-si, KR)
Assignee: Samsung Electronics Co., Ltd.
G10L15/26G10L15/16G10L15/22G10L17/24
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,508,375
App. No.
16/903,918
Granted
Nov 22, 2022
Kind
B2
Abstract

An electronic apparatus is provided. The electronic apparatus includes a microphone, a transceiver, a memory configured to store a control command identification tool based on a control command identified by a voice recognition server that performs voice recognition processing on a user voice received from the electronic apparatus, and at least one processor configured to, based on the user voice being received through the microphone, acquire user intention information by performing the voice recognition processing on the received user voice, receive status information of external devices related to the acquired user intention information from a device control server, identify a control command for controlling a device to be controlled among the plurality of external devices by applying the acquired user intention information and the received status information of the external devices to the control command identification tool, and transmit the identified control command to the device control server.

Claims (62)

1. An electronic apparatus comprising:

a microphone;

a transceiver;

a memory configured to store a control command identification tool generated by using a control command identified by a voice recognition server that identifies the control command corresponding to a user voice received by the electronic apparatus; and

at least one processor configured to, based on the user voice being received through the microphone:

acquire user intention information by performing voice recognition processing on the user voice received through the microphone,

receive status information of an external device related to the acquired user intention information from a device control server configured to control a plurality of external devices through the transceiver,

identify a control command for controlling a device to be controlled among the plurality of external devices by applying the acquired user intention information and the received status information of the external device to the control command identification tool, and

transmit the identified control command to the device control server through the transceiver,

wherein the control command identification tool comprises a database (DB) in which the user intention information, the status information of the external device, and the control command identified by the voice recognition server are matched with each other,

wherein the at least one processor is further configured to, based on the acquired user intention information and the received status information of the external device being present in the DB, identify a control command as a control command corresponding to the user voice, and

wherein the at least one processor is further configured to:

based on the acquired user intention information and the received status information of the external device not being present in the DB, transmit the user voice to the voice recognition server through the transceiver,

based on the control command identified from the voice recognition server being received based on the user voice, generate a new rule by matching the received control command with the acquired user intention information and the received status information of the external device, and

update the generated new rule in the DB.

2. The electronic apparatus of claim 1 ,

wherein the control command identification tool comprises an artificial intelligence model trained by using the user intention information and the status information of the external device as an input and by using the control command identified by the voice recognition server as an output, and

wherein the at least one processor is further configured to:

input the acquired user intention information and the received status information of the external device into the trained artificial intelligence model, and

identify a control command output from the trained artificial intelligence model as a control command corresponding to the user voice.

3. The electronic apparatus of claim 2 , wherein the at least one processor is further configured to:

based on the control command not being output from the trained artificial intelligence model in which the acquired user intention information and the received status information of the external device are input, transmit the user voice to the voice recognition server through the transceiver, and

based on the control command identified from the voice recognition server being received through the transceiver based on the user voice, retrain the trained artificial intelligence model based on the received status information of the external device and the received control command.

4. The electronic apparatus of claim 1 , wherein the at least one processor is further configured to, based on the device to be controlled and an operation of the device to be controlled being specified only by the acquired user intention information based on the received status information of the external device, identify the control command based on the acquired user intention information without using the control command identification tool.

5. The electronic apparatus of claim 4 ,

wherein the acquired user intention information comprises information on an entity, and

wherein the at least one processor is further configured to, based on status information of a plurality of external devices related to the entity being received from the device control server:

identify that the device to be controlled is not specified only by the acquired user intention information, and

identify the control command by using the control command identification tool.

6. The electronic apparatus of claim 1 ,

wherein the transceiver comprises an infrared (IR) communication module, and

wherein the at least one processor is further configured to, based on the device to be controlled being controllable by an IR control method, transmit the identified control command to the device to be controlled through the IR communication module.

7. A control method of an electronic apparatus comprising:

storing a control command identification tool generated by using a control command identified by a voice recognition server configured to identify the control command corresponding to a user voice received by the electronic apparatus;

based on the user voice being received through a microphone of the electronic apparatus, acquiring user intention information by performing voice recognition processing on the user voice received through the microphone;

receiving status information of an external device related to the acquired user intention information from a device control server configured to control a plurality of external devices;

applying the acquired user intention information and the received status information of the external device to the control command identification tool;

identifying a control command for controlling a device to be controlled among the plurality of external devices; and

transmitting the identified control command to the device control server,

wherein the control command identification tool comprises a database (DB) in which the user intention information, the status information of the external device, and the control command identified by the voice recognition server are matched with each other,

wherein the identifying of the control command comprises, based on the acquired user intention information and the received status information of the external device being present in the DB, identifying a control command as a control command corresponding to the user voice, and

wherein the method further comprises:

based on the acquired user intention information and the received status information of the external device not being present in the DB, transmitting the user voice to the voice recognition server,

based on the control command identified from the voice recognition server being received based on the user voice, generating a new rule by matching the received control command with the acquired user intention information and the received status information of the external device, and

updating the generated new rule in the DB.

8. The method of claim 7 ,

wherein the control command identification tool comprises an artificial intelligence model trained by using the user intention information and the status information of the external device as an input and by using the control command identified by the voice recognition server as an output, and

wherein the identifying of the control command comprises:

inputting the acquired user intention information and the received status information of the external device into the trained artificial intelligence model, and

identifying a control command output from the trained artificial intelligence model as a control command corresponding to the user voice.

9. The method of claim 8 , further comprising:

based on the control command not being output from the trained artificial intelligence model in which the acquired user intention information and the received status information of the external device are input, transmitting the user voice to the voice recognition server, and

based on the control command identified from the voice recognition server being received based on the user voice, retraining the trained artificial intelligence model based on the received status information of the external device and the received control command.

10. The method of claim 7 , further comprising:

based on the device to be controlled and an operation of the device to be controlled being specified only by the acquired user intention information based on the received status information of the external device, identifying the control command based on the acquired user intention information without using the control command identification tool.

11. The method of claim 10 ,

wherein the acquired user intention information comprises information on an entity, and

wherein the method further comprises:

based on status information of a plurality of external devices related to the entity being received from the device control server, identifying that the device to be controlled is not specified only by the acquired user intention information; and

identifying the control command by using the control command identification tool.

12. The method of claim 7 , further comprising:

based on the device to be controlled being controllable by an infrared (IR) control method, transmitting the identified control command to the device to be controlled.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 17, 2020
From: CHOI, WOOJEI; KIM, MINKYONG
To: SAMSUNG ELECTRONICS CO., LTD.
Reel/Frame 052965/0010 →
Priority Claims (1)
KR 10-2020-0012154 · Jan 31, 2020 · national
Continuity (2)
Provisional Application 62870222 · Jul 3, 2019
Related Publication 20210005205A1 · Jan 7, 2021
Cited By (1)
US 12,406,671