IP Library › Granted Patent US 10,710,244
Granted Patent B2
US 10,710,244 · App. 15/627,506 · Granted Jul 14, 2020

Robot control using gestures

Inventors: Yinian Mao (Beijing, CN); Xinmin Liu (Beijing, CN)
Assignee: Beijing Airlango Technology Co., Ltd.
B25J9/1697B25J9/1656G06F3/017G06F3/0304G06K9/00288G06K9/00355G06K9/00375G06K9/3233G06K9/6269G05B2219/23021G05B2219/35444
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,710,244
App. No.
15/627,506
Granted
Jul 14, 2020
Kind
B2
Abstract

A method and a device for operating a robot are provided. According to an example of the method, information of a first gesture is acquired from a group of gestures of an operator, each gesture from the group of gestures corresponding to an operation instruction from a group of operation instructions. A first operation instruction from the group of operation instructions is obtained based on the acquired information of the first gesture, the first operation corresponding to the first gesture. The first operation instruction is executed.

Claims (87)

1. A method of operating a robot, comprising:

acquiring information of a first gesture from a group of gestures, each gesture from the group of gestures corresponding to an operation instruction from a group of operation instructions;

obtaining, based on the acquired information of the first gesture, a first operation instruction from the group of operation instructions, the first operation instruction corresponding to the first gesture; and

executing the first operation instruction;

wherein acquiring the information of the first gesture comprises:

capturing an image using a camera;

identifying a Region Of Interest (ROI) from the captured image;

determining whether the ROI includes an operator of the robot; and

acquiring, using the captured image, the information of the first gesture after it is determined that the ROI includes the operator.

2. The method according to claim 1 , wherein determining whether the ROI includes the operator comprises:

calculating a similarity between face feature information of the ROI and pre-configured face feature information of the operator; and

determining that the ROI includes the operator when the calculated similarity is greater than a predetermined threshold.

3. The method according to claim 1 , wherein the image is a first image, the method further comprises capturing a second image using the camera, the determining whether the ROI includes the operator comprises:

acquiring, based on the second image, information of a second gesture;

determining whether the second gesture matches a pre-configured gesture password; and

determining that the ROI of the first image includes the operator when the second gesture matches the pre-configured gesture password.

4. The method according to claim 3 , wherein the pre-configured gesture password is associated with a posture or gesture of the operator, the method further comprises:

executing the first operation instruction from the group of operation instructions when the second gesture matches the pre-configured gesture password.

5. The method according to claim 1 , wherein acquiring the information of the first gesture comprises:

identifying, from the captured image, a region associated with the operator;

analyzing the identified region to obtain a first feature vector;

calculating, for each gesture from the group of gestures, a similarity between the first feature vector and a feature vector of that gesture; and

selecting the gesture having the highest similarity as the first gesture to be acquired.

6. The method according to claim 1 , wherein

the first gesture is a static gesture; and

the information of the first gesture comprises information of a status of a limb.

7. The method according to claim 1 , wherein

the first gesture is a dynamic gesture; and

the information of the first gesture comprises information of a movement of a limb.

8. A robot, comprising:

a processor; and

a non-transitory storage medium storing machine-executable instructions,

wherein by executing the machine-executable instructions, the processor is caused to:

acquire information of a first gesture from a group of gestures, each gesture from the group of gestures corresponding to an operation instruction from a group of operation instructions;

obtain, based on the acquired information of the first gesture, a first operation instruction from the group of operation instructions, the first operation instruction corresponding to the first gesture; and

execute the first operation instruction;

wherein when the processor is caused to acquire the information of the first gesture, the machine-executable instructions further cause the processor to:

capture an image using a camera;

identify a Region Of Interest (ROI) from the captured image;

determine whether the ROI includes an operator of the robot; and

acquire, using the captured image, the information of the first gesture after it is determined that the ROI includes the operator.

9. The method according to claim 1 , wherein the group of operation instructions comprise an action instruction, a photographing instruction, an automatic-charging instruction, a return-to-origin instruction, a take-off instruction, and a landing instruction.

10. The robot according to claim 8 , wherein when the processor is caused to determine whether the ROI includes the operator, the machine-executable instructions further cause the processor to:

calculate a similarity between face feature information of the ROI and pre-configured face feature information of the operator; and

determine that the ROI includes the operator when the calculated similarity is greater than a predetermined threshold.

11. The robot according to claim 8 , wherein the image is a first image, when the processor is caused to determine whether the ROI includes the operator, the machine-executable instructions further cause the processor to:

capture a second image using the camera;

acquire, based on the second image, information of a second gesture;

determine whether the second gesture matches a pre-configured gesture password; and

determine that the ROI of the first image includes the operator when the second gesture matches the pre-configured gesture password.

12. The robot according to claim 11 , wherein the pre-configured gesture password includes a posture or gesture of the operator, the machine-executable instructions further cause the processor to:

execute the first operation instruction from the group of operation instructions when the second gesture matches the pre-configured gesture password.

13. The robot according to claim 8 , wherein when the processor is caused to acquire the information of the first gesture, the machine-executable instructions further cause the processor to:

identify, from the captured image, a region associated with the operator;

analyze the identified region to obtain a first feature vector;

calculate, for each gesture from the group of gestures, a similarity between the first feature vector and a feature vector of that gesture; and

select the gesture having the highest similarity as the first gesture to be acquired.

14. The robot according to claim 8 , wherein

the first gesture is a static gesture; and

the information of the first gesture comprises information of a status of a limb.

15. The robot according to claim 8 , wherein

the first gesture is a dynamic gesture; and

the information of the first gesture comprises information of a movement of a limb.

16. The robot according to claim 8 , wherein the group of operation instructions comprise an action instruction, a photographing instruction, an automatic-charging instruction, a return-to-origin instruction, a take-off instruction, and a landing instruction.

17. An apparatus for operating a robot, comprising:

a processor; and

a non-transitory storage medium storing machine-executable instructions,

wherein by executing the machine-executable instructions, the processor is caused to:

receive an image captured using a camera of the robot;

acquire, based on the received image, information of a first gesture from a group of gestures, each gesture from the group of gestures corresponding to an operation instruction from a group of operation instructions;

obtain, based on the acquired information of the first gesture, a first operation instruction from the group of operation instructions, the first operation instruction corresponding to the first gesture; and

transmit the first operation instruction to the robot such that the robot executes the first operation instruction;

wherein when the processor is caused to acquire the information of the first gesture, the machine-executable instructions further cause the processor to:

capture an image using a camera;

identify a Region Of Interest (ROI) from the captured image;

determine whether the ROI includes an operator of the robot; and

acquire, using the captured image, the information of the first gesture after it is determined that the ROI includes the operator.

18. A method of operating a robot, comprising:

receiving, at a server, an image captured using a camera of the robot;

acquiring, at the server and based on the received image, information of a first gesture from a group of gestures, each gesture from the group of gestures corresponding to an operation instruction from a group of operation instructions;

obtaining, at the server and based on the acquired information of the first gesture, a first operation instruction from the group of operation instructions, the first operation instruction corresponding to the first gesture; and

transmitting, from the server to the robot, the first operation instruction such that the robot executes the first operation instruction;

wherein acquiring the information of the first gesture comprises:

capturing an image using a camera;

identifying a Region Of Interest (ROI) from the captured image;

determining whether the ROI includes an operator of the robot; and

acquiring, using the captured image, the information of the first gesture after it is determined that the ROI includes the operator.

Priority Claims (1)
CN 2016 1 0509806 · Jun 30, 2016 · national
Continuity (1)
Related Publication 20180001480A1 · Jan 4, 2018