Concierge device for providing artificial intelligence concierge service, and a control method for same device

1 . A concierge device comprising:

a camera configured to acquire image information related to an utterer located within a preset effective distance from the concierge device;

a microphone configured to receive speech information from the utterer;

a speaker configured to output response information corresponding to the received speech information;

a memory configured to store a plurality of corpora that meet different conditions;

an artificial intelligence processor that includes a natural language processing (NLP) component to recognize the received speech information through natural language understanding and generate a natural language sentence including the response information corresponding to the recognized received speech information; and

a processor configured to control the artificial intelligence processor to detect at least one corpus among the plurality of corpora that matches at least one feature information related to the utterer detected based on the image information, generate the natural language sentence based on the detected at least one corpus, and control the speaker to output the speech information corresponding to the natural language sentence generated by the artificial intelligence processor as the response information, and

wherein the processor is further configured to determine whether a conversation with the utterer has ended, and based on determining that the conversation with the utterer has ended, store, in the memory, information related to the detected at least one corpus, the at least one feature information related to the utterer, the recognized received speech information and the response information as conversation information related to the utterer.

2 . The concierge device of claim 1 , wherein the at least one feature information related to the utterer includes at least one of a gender or an age of the utterer, and

the processor is further configured to detect any one corpus from the plurality of corpora that matches the at least one of the gender or the age of the utterer.

3 . The concierge device of claim 1 , wherein the processor is further configured to delete the conversation information stored in the memory based on a preset time elapsing.

4 . The concierge device of claim 3 , wherein the processor is further configured to initialize the preset time based on feature information identified from a second utterer close to the concierge device matching the at least one feature information included in the conversation information before the preset time elapses.

5 . The concierge device of claim 4 , wherein based on the feature information identified from the second utterer matching the at least one feature information included in the conversation information before the preset time elapses, the processor is further configured to determine a meaning corresponding to second speech information received from the second utterer by further reflecting the recognized received speech information and the response information included in the conversation information according to natural language understanding for the second speech information.

6 . The concierge device of claim 5 , wherein based on the feature information identified from the second utterer matching the at least one feature information included in the conversation information before the preset time elapses, the processor is further configured to update the conversation information to further include the second speech information and the response information for the second speech information.

7 . The concierge device of claim 1 , further comprising a display,

wherein the processor is further configured to control the display to output an image of a preset avatar.

8 . The concierge device of claim 7 , wherein the processor is further configured to detect an eye level of the utterer based on the at least one feature information, and control the display to change a posture of the preset avatar to match the detected eye level.

9 . The concierge device of claim 7 , wherein the processor is further configured to control the display to output an avatar having a different gender or age according to the at least one feature information.

10 . The concierge device of claim 7 , wherein the processor is further configured to determine an emotional state of the preset avatar based on at least one of an emotional state of the utterer determined based on the image information related to the utterer or the response information corresponding to the recognized received speech information, and control the display to output an avatar to have one of a plurality of expressions corresponding to the determined emotional state.

11 . The concierge device of claim 10 , wherein the processor is further configured to determine the emotional state of the avatar by prioritizing the response information corresponding to the recognized received speech information over the determined emotional state.

12 . The concierge device of claim 1 , wherein based on no utterer being within the preset effective distance, the processor is further configured to identify people around the concierge device, identify, as a help requester, a person among the identified people who has been found at least a certain number of times within a preset time, and output a greeting asking whether the identified person needs help.

13 . The concierge device of claim 12 , further comprising a mover configured to move a body of the concierge device, the mover including at least one moving element including a wheel or a caterpillar, and at least one power element for driving the at least one moving element,

wherein based on the person being identified, the processor is further configured to control the mover such that the concierge device approaches the identified person within the preset effective distance, and output the greeting based on the identified person entering within the preset effective distance.

14 . The concierge device of claim 1 , further comprising a signal transmitter and receiver configured to perform communication with a server including a NLP component,

wherein the processor is further configured to transmit, to the server, the detected at least one feature information and the speech information received by the microphone,

wherein the server is configured to receive the at least one feature information and the speech information, recognize the received speech information through natural language understanding, retrieve information corresponding to a result of the recognition, generate a natural language sentence including information corresponding to the result of the recognition based on at least one of a plurality of corpora matching different conditions, based on the at least one feature information received from the concierge device, and transmit, to the concierge device, the generated natural language sentence in response to the received speech information, and

wherein the processor is further configured to control the signal transmitter and receiver to receive the natural language sentence from the server.

15 . The concierge device of claim 14 , further comprising a display,

wherein the server is further configured to transmit information related to a gesture or posture of a preset avatar using tag information related to the natural language sentence generated by the server, and

wherein the processor is further configured to control the display to output the preset avatar taking the gesture or the posture according to the tag information.

16 . A method for controlling a concierge device, the method comprising:

identifying an utterer located within a preset effective distance from the concierge device;

detecting feature information related to the identified utterer;

selecting at least one corpus of a plurality of corpora based on at least one of the detected feature information related to the utterer, wherein the plurality of corpora meet different conditions;

recognizing speech information received from the utterer through natural language understanding;

retrieving information requested by the utterer according to a result of the recognition;

generating a natural language sentence based on the selected at least one corpus, by using the retrieved information;

outputting the generated natural language sentence as response information;

determining whether a conversation with the utterer has ended; and

based on determining that the conversation with the utterer has ended, storing information related to the selected at least one corpus, the at least one of the detected feature information related to the utterer, the recognized speech information and the response information as conversation information related to the utterer.