IP Library Granted Patent US 11,413,764
Granted Patent B2
US 11,413,764 · App. 16/488,797 · Granted Aug 16, 2022

Serving robot and method for receiving customer using the same

Inventors: Hyongguk Kim (Seoul, KR); Jaeyoung Kim (Seoul, KR); Hyoungmi Kim (Seoul, KR); Yujune Jang (Seoul, KR)
Assignee: LG ELECTRONICS INC.
B25J11/0005B25J9/161B25J11/0045B25J13/003B25J13/08B25J19/023B25J19/026G06V40/174G06V40/20
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,413,764
App. No.
16/488,797
Granted
Aug 16, 2022
Kind
B2
Abstract

A serving robot includes a camera to obtain image data including at least one of a facial expression or a gesture, which is associated with food, of a customer, a microphone to obtain voice data including voice of the customer, which is associated with the food and a processor to obtain customer reaction data including at least one of the image data or the voice data, through at least one of the camera or the microphone, estimate a reaction of the customer to the food, from the obtained customer reaction data, and generate or update customer management information corresponding to the customer based on the estimated reaction. The robot estimates the reaction the customer from the customer reaction data through the learning model based on artificial intelligence.

Claims (83)

1. A serving robot comprising:

a camera to obtain image data including at least one of a facial expression or a gesture, which is associated with food, of a customer;

a microphone to obtain voice data including voice of the customer, which is associated with the food; and

a processor configured to:

obtain customer reaction data including at least one of the image data or the voice data, through at least one of the camera or the microphone;

estimate a reaction of the customer to the food, from the obtained customer reaction data; and

generate or update customer management information corresponding to the customer based on the estimated reaction,

wherein the processor is configured to:

obtain customer feature data including at least one of image data including a face of the customer or voice data including a voice of the customer, through at least one of the camera or the microphone;

identify the customer based on the obtained customer feature data;

obtain, from a database, customer management information corresponding to the identified customer;

output an order request message having favorite menu information that is included within the customer management information corresponding to the identified customer, when the customer is identified; and

output an order request message having popular menu information, when the customer is not identified,

wherein the customer management information includes:

at least one item of a name of the customer, a visit history or an ordered food history.

2. The serving robot of claim 1 , wherein the processor is configured to:

recognize emotion of the customer based on the at least one of the facial expression or the gesture of the customer included in the image data; and

estimate the reaction of the customer to the food, based on the recognized emotion.

3. The serving robot of claim 1 , further comprising:

a memory to store a learning model learned by a learning processor,

wherein the processor is configured to:

estimate the reaction of the customer from the customer reaction data, through the learning model stored in the memory.

4. The serving robot of claim 1 , further comprising:

a communication unit to connect with a server,

wherein the processor is configured to:

control the communication unit to transmit the customer reaction data to the server; and

receive, from the server, information on the reaction of the customer based on the customer reaction data.

5. The serving robot of claim 1 , further comprising:

a communication unit to connect with a kitchen communication device disposed in a kitchen,

wherein the processor is configured to:

control the communication unit to transmit at least one of the customer reaction data, information on the estimated reaction, or voice data including a request or an inquiry associated with food, which is obtained from the customer, to the kitchen communication device.

6. The serving robot of claim 5 , further comprising:

at least one of a display and a speaker,

wherein the processor is configured to:

receive response data including at least one of image data or voice data from the kitchen communication device; and

control at least one of the display and the speaker to output the received response data.

7. The serving robot of claim 1 , wherein the customer management information further includes:

at least one item of allergy or dislike food ingredient, preferred taste, hate taste, or a favorite food menu.

8. The serving robot of claim 7 , wherein the processor is configured to:

update at least one of the allergy or dislike food ingredient, the preferred taste, the hate taste, or the favorite food menu, which is the item of the customer management information, based on the estimated reaction.

9. The serving robot of claim 1 , further comprising:

a communication unit to connect with at least one of a server or a kitchen communication device,

wherein the processor is configured to:

obtain order information from the customer, based on the customer management information; and

transmit the obtained order information to the server or the kitchen communication device.

10. The serving robot of claim 9 , wherein the processor is configured to:

receive expected cooking time information for the order information from the server or the kitchen communication device; and

output the received expected cooking time information through at least one of a display or a speaker.

11. The serving robot of claim 9 , wherein the processor is configured to:

register the order information from the customer in an order list including order information of at least one of another customer;

generate expected cooking time information for the order information, based on the order list; and

output the generated expected cooking time information through at least one of a display or a speaker.

12. A method for receiving a customer using a serving robot, the method comprising:

obtaining customer reaction data associated with food;

estimating a reaction of the customer to the food, from the obtained customer reaction data; and

updating customer management information corresponding to the customer based on the estimated reaction,

wherein the customer reaction data includes:

at least one of image data including at least one of a facial expression or a gesture, which is associated with the food, of the customer, or voice data including a voice of the customer, which is associated with the food,

wherein the method further comprises:

 identifying the customer based on at least one of image data including a face of the customer or voice data including a voice of the customer;

 obtaining, from a database, customer management information that corresponds to the identified customer;

 outputting an order request message having favorite menu information that is included within the customer management information corresponding to the identified customer, when the customer is identified; and

 outputting an order request message having popular menu information, when the customer is not identified,

 wherein the customer management information includes:

 information on at least one item of a name of the customer, a visit history, or an ordered food history.

13. The method of claim 12 , wherein the estimating of the reaction of the customer to the food includes:

recognizing emotion of the customer based on the at least one of the facial expression or the gesture of the customer included in the image data; and

estimating the reaction of the customer to the food, based on the recognized emotion.

14. The method of claim 12 , further comprising:

transmitting at least one of the customer reaction data, information on the estimated reaction, or voice data including a request or an inquiry associated with the food, which is obtained from the customer, to a kitchen communication device connected with the serving robot.

15. The method of claim 14 , further comprising:

receiving response data including at least one of image data or voice data from the kitchen communication device; and

outputting the received response data through the serving robot or a table communication device corresponding to a table of the customer.

16. The method of claim 12 , wherein the customer management information includes:

information on at least one item of allergy or dislike food ingredient, preferred taste, hate taste, or a favorite menu, and

wherein the updating of the customer management information includes:

updating information on at least one item of the allergy or dislike food ingredient, the preferred taste, the hate taste, or the favorite menu, based on the estimated reaction.

17. The method of claim 12 , further comprising:

obtaining order information from the customer based on the customer management information; and

transmitting the obtained order information to a server or a kitchen communication device connected with the serving robot.

18. The method of claim 17 , further comprising:

receiving expected cooking time information for the order information from the server or the kitchen communication device; and

outputting the received expected cooking time information through the serving robot or a table communication device corresponding to a table of the customer.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Aug 26, 2019
From: KIM, HYONGGUK; KIM, JAEYOUNG; KIM, HYOUNGMI; JANG, YUJUNE
To: LG ELECTRONICS INC.
Reel/Frame 050168/0277 →
Continuity (1)
Related Publication 20210354305A1 · Nov 18, 2021