Electronic device and method for controlling the electronic device thereof based on determining intent of a user speech in a first language machine translated into a predefined second language

1. An electronic device comprising:

a memory storing at least one instruction; and a processor, operatively coupled to the memory, and configured to control the electronic device by executing the at least one instruction stored in the memory,

wherein the processor is configured by the at least one instruction to:

based on a user's speech being input, acquire a first sentence in a first language corresponding to the user's speech through a speech recognition model corresponding to a language of the user's speech,

acquire a plurality of candidate sentences corresponding to the first sentence and reliability values corresponding to the plurality of candidate sentences through a machine translation model,

determine whether intent in a first-priority sentence is identified by inputting the first-priority sentence having a highest reliability value among the plurality of candidate sentences to the natural language understanding model trained based on a second language,

and based on determining that the intent in the first-priority sentence is not identified, determine whether intent in a second-priority sentence is identified by inputting the second-priority sentence having a reliability value lower than the reliability value of the first-priority sentence among the plurality of candidate sentences to the natural language understanding model trained based on the second language.

2. The electronic device according to claim 1 ,

wherein the memory comprises a first machine translation model and a first natural language understanding model,

wherein, to acquire the plurality of candidate sentences corresponding to the first sentence and reliability values corresponding to the plurality of candidate sentences, the processor is further configured by the at least one instruction to input the first sentence to the first machine translation model,

wherein the processor is further configured by the at least one instruction to acquire a reliability value of the second sentence in response to the input of the first sentence to the first machine translation model, and

wherein the processor is further configured by the at least one instruction to, based on the reliability value being a threshold value or higher, input the second sentence to the first natural language understanding model.

3. The electronic device according to claim 2 , further comprising:

a communication interface configured to communicate with an external server,

wherein the external server comprises a second machine translation model trained to translate a plurality of languages into the second language, and

wherein the processor is further configured by the at least one instruction to:

based on the reliability value being lower than the threshold value, control the communication interface to transmit the first sentence to the external server, and

based on a third sentence in the second language acquired through the second machine translation model being received from the external server, acquire a control instruction corresponding to the third sentence or acquire a response to the third sentence by inputting the third sentence to the first natural language understanding model.

4. The electronic device according to claim 3 , wherein the first machine translation model is retrained based on the first sentence and the third sentence.

5. The electronic device according to claim 3 , further comprising:

wherein the external server comprises a second natural language understanding model trained based on the second language, and

wherein the processor is further configured by the at least one instruction to, based on the reliability value being lower than the threshold value:

control the communication interface to transmit the first sentence to the external server, and

receive a control instruction corresponding to the third sentence or a response to the third sentence acquired through the second machine translation model and the second natural language understanding model from the external server via the communication interface.

6. The electronic device according to claim 1 , further comprising:

a first database storing an input text and a correction text of the machine translation model which are mapped to each other,

wherein the processor is further configured by the at least one instruction to, based on determining that the intent in the second-priority sentence is identified, map at least one text included in the first sentence and at least one text included in the second-priority sentence as the input text and the correction text in the first database.

7. The electronic device according to claim 6 , further comprising:

a second database storing information regarding a proper noun stored in the electronic device,

wherein the processor is further configured by the at least one instruction to, based on the second sentence including the proper noun stored in the second database:

replace at least one text included in the second sentence with the proper noun in the first language stored in the second database, and

acquire a control instruction of the electronic device corresponding to the replaced at least one text included in the second sentence or acquiring a response to the second sentence.

8. The electronic device according to claim 7 , wherein the processor is further configured by the at least one instruction to:

map the at least one text included in the second sentence and the proper noun as an input text and a correction text, and

store the mapping of the at least one text included in the second sentence and the proper noun in the first database.

9. The electronic device according to claim 1 , wherein the machine translation model is trained by performing multi-task learning (MTL) by using the second language as a common parameter.

10. A method for controlling an electronic device, the method comprising:

based on a user's speech being input, acquiring a first sentence in a first language corresponding to the user's speech through a speech recognition model corresponding to a language of the user's speech;

acquiring a plurality of candidate sentences corresponding to the first sentence and reliability values corresponding to the plurality of candidate sentences through a machine translation model;

determining whether intent in a first-priority sentence is identified by inputting the first-priority sentence having a highest reliability value among the plurality of candidate sentences to the natural language understanding model trained based on a second language: and

based on determining that the intent in the first-priority sentence is not identified, determining whether intent in a second-priority sentence is identified by inputting the second priority sentence having a reliability value lower than the reliability value of the first priority sentence among the plurality of candidate sentences to the natural language understanding model trained based on the second language.

11. The method according to claim 10 ,

wherein a memory of the electronic device comprises a first machine translation model and a first natural language understanding model,

wherein the plurality of candidate sentences corresponding to the first sentence and reliability values corresponding to the plurality of candidate sentences comprises:

inputting the first sentence to the first machine translation model,

wherein the method further comprises acquiring a reliability value of the second sentence in response to the inputting of the first sentence to the first machine translation model, and

wherein the method further comprises:

based on the reliability value being a threshold value or higher, inputting the second sentence to the first natural language understanding model.

12. The method according to claim 11 ,

wherein an external server connected to the electronic device comprises a second machine translation model trained to translate a plurality of languages into the second language, and

wherein the method further comprises:

based on the reliability value being lower than the threshold value, transmitting the first sentence to the external server; and

based on a third sentence in the second language acquired through the second machine translation model being received from the external server, acquiring a control instruction corresponding to the third sentence or acquiring a response to the third sentence by inputting the third sentence to the first natural language understanding model.

13. The method according to claim 12 , further comprising:

retraining the first machine translation model based on the first sentence and the third sentence.

14. The method according to claim 12 ,

wherein an external server connected to the electronic device comprises a second natural language understanding model trained based on the second language, and

wherein the acquiring of the control instruction or the acquiring of the response to the second sentence comprises, based on the reliability value being lower than a threshold value:

transmitting the first sentence to an external server; and

receiving a control instruction corresponding to the third sentence or a response to the third sentence acquired through the second machine translation model and the second natural language understanding model from the external server.

15. The method according to claim 10 ,

wherein the electronic device comprises a first database storing an input text and a correction text of the machine translation model which are mapped to each other,

wherein the method further comprises:

based on determining that the intent in the second-priority sentence is identified, mapping at least one text included in the first sentence and at least one text included in the second-priority sentence as the input text and the correction text and storing in the first database.

16. The method according to claim 15 ,

wherein the natural language understanding model comprises a second database storing information regarding a proper noun stored in the electronic device,

wherein the acquiring of the control instruction or the acquiring of the response to the second sentence comprises, based on the second sentence including a proper noun stored in the second database:

replacing at least one text included in the second sentence with the proper noun in the first language stored in the second database; and

acquiring a control instruction of the electronic device corresponding to the replaced at least one text included in the second sentence or acquiring a response to the second sentence.

17. The method according to claim 16 , further comprising:

mapping the at least one text included in the second sentence and the proper noun as an input text and a correction text; and

storing the mapping of the at least one text included in the second sentence and the proper noun in the first database.

18. The method according to claim 10 ,

wherein the machine translation model is trained by performing multi-task learning (MTL) by using the second language as a common parameter.