IP Library Granted Patent US 11,276,402
Granted Patent B2
US 11,276,402 · App. 16/678,267 · Granted Mar 15, 2022

Method for waking up robot and robot thereof

Inventor: Lei Luo (Shenzhen, CN)
Assignee: CLOUDMINDS ROBOTICS CO., LTD.
G10L15/22B25J9/1697B25J13/003G05B19/4155G06F3/013G06K9/00664G10L15/1822G10L15/24G05B2219/36017G06F3/167G10L15/25G10L2015/088G10L2015/223H04L12/2803H04L12/2821
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,276,402
App. No.
16/678,267
Granted
Mar 15, 2022
Kind
B2
Abstract

A method for waking up a robot includes: acquiring sight range information when a voice command issuer issues a voice command; if the sight range information of the voice command issuer when issuing the voice command is acquired, determining, based on the sight range information, whether the voice command issuer gazes the robot when the voice command is issued; and determining that the robot is called if the voice command issuer gazes the robot.

Claims (130)

1. A method for waking up a robot, applied to a robot, wherein the method comprises:

acquiring sight range information when a voice command issuer issues a voice command;

if the sight range information of the voice command issuer when issuing the voice command is acquired, determining, based on the sight range information, whether the voice command issuer gazes the robot when the voice command is issued; and

determining that the robot is called if the voice command issuer gazes the robot;

interrogating the voice command issuer if the sight range information of the voice command issuer when issuing the voice command is not acquired;

wherein the acquiring sight range information when a voice command issuer issues a voice command comprises:

acquiring voice command information, the voice command information comprising time information of the voice command and information of the voice command issuer;

broadcasting the voice command information;

determining whether a robot that hears the same voice command exists;

enabling the robot that hears the same voice command to join a same candidate group if the robot that hears the same voice command exists;

acquiring visual information and position information of the robot complying with the time information;

acquiring, based on the visual information and the position information of the robot, the sight range information of the voice command issuer when issuing the voice command; and

if the sight range information of the voice command issuer when issuing the voice command exists, broadcasting, in the candidate group, the sight range information of the voice command issuer when issuing the voice command.

2. The method according to claim 1 , wherein the voice command information further comprises a sound pressure level of the voice command, and the acquiring voice command information comprises:

monitoring a voice signal;

parsing out a wake-up word in the voice signal;

determining a sound pressure level of the voice signal;

if the wake-up word indicates a call and the sound pressure level of the voice signal exceeds a predetermined threshold, recording a start time point and an end time point of the voice signal as the time information of the voice command, and recording the sound pressure level of the voice signal as the sound pressure level of the voice command;

identifying a voice signal issuer based on the voice signal; and

recording the voice signal issuer as the information of the voice command issuer.

3. The method according to claim 2 , wherein the wake-up word is a pronoun indicating the call; and

the interrogating the voice command issuer comprises:

determining whether the wake-up word is a pronoun indicating a single form;

determining a robot with a highest sound pressure level in the candidate group if the wake-up word is the pronoun indicating the single form, such that the robot with the highest sound pressure level interrogates whether the voice command issuer calls the robot with the highest sound pressure level;

responding to the voice command issuer if the voice command issuer calls the robot with the highest sound pressure level;

otherwise, in a scenario where the voice command issuer issues a new voice command, acquiring sight range information of the voice command issuer when issuing the new voice command;

determining robots with a highest sound pressure level and a second highest sound pressure level in the candidate group if the wake-up word is not the pronoun indicating the single form, such that the robots with the highest sound pressure level and the second highest sound pressure level interrogate whether the voice command issuer calls only the robots with the highest sound pressure level and the second highest sound pressure level;

responding to the voice command issuer if the voice command issuer calls only the robots with the highest sound pressure level and the second highest sound pressure level;

otherwise, in a scenario where the voice command issuer issues a new voice command, acquiring sight range information of the voice command issuer when issuing the new voice command;

broadcasting, in the candidate group, the sight range information of the voice command issuer when issuing the new voice command; and

the method further comprises:

if the sight range information of the voice command issuer when issuing the new voice command is acquired, determining, based on the sight range information, whether the voice command issuer gazes the robot when the voice command is issued; and

determining that the robot is called if the voice command issuer gazes the robot; and

responding to the voice command issuer.

4. The method according to claim 1 , wherein the voice command information further comprises a sound pressure level of the voice command, and the acquiring voice command information comprises:

monitoring a voice signal;

parsing out a wake-up word in the voice signal;

determining a sound pressure level of the voice signal;

if the wake-up word indicates a call and the sound pressure level of the voice signal exceeds a predetermined threshold, recording a start time point and an end time point of the wake-up word as the time information of the voice command, and recording the sound pressure level of the voice signal as the sound pressure level of the voice command; and

identifying a voice signal issuer based on the voice signal, and recording the voice signal issuer as the information of the voice command issuer.

5. The method according to claim 1 , wherein the acquiring, based on the visual information and the position information of the robot, the sight range information of the voice command issuer when issuing the voice command comprises:

equally dividing the time information of the voice command into a plurality of time points;

at each time point, determining an angle between the face of the voice command issuer and the robot based on the visual information at the time point;

acquiring a sight direction at the time point based on the position information of the robot at the time point and the angle; and

using the acquired sight direction and a time point corresponding to the sight direction as the sight range information.

6. The method according to claim 5 , wherein the determining, based on the sight range information, whether the voice command issuer gazes the robot when the voice command is issued comprises:

acquiring the position information of the robot within a predetermined time period;

determining whether the position information complying with the sight direction of the robot at a time point of the sight range information exists; and

determining that the voice command issuer gazes the robot if the position information of the robot complying with the sight direction at a time point of the sight range information exists.

7. The method according to claim 1 , wherein the acquiring, based on the visual information and the position information of the robot, the sight range information of the voice command issuer when issuing the voice command comprises:

equally dividing the time information of the voice command into a plurality of time points;

at each time point, determining an angle between the face of the voice command issuer and the robot based on the visual information at the time point;

acquiring a sight direction at the time point based on the position information of the robot issuer at the time point and the angle;

acquiring a sight interval and a time range corresponding to the sight interval based on the acquired sight direction and the time point corresponding to the sight direction; and

using the sight interval and the time range as the sight range information.

8. The method according to claim 7 , wherein the determining, based on the sight range information, whether the voice command issuer gazes the robot when the voice command is issued comprises:

acquiring the position information of the robot within a predetermined time period;

determining whether the position information of the robot at the time range of the sight range information complies with the sight interval; and

determining that the voice command issuer gazes the robot if the position information of the robot at the time range of the sight range information complies with the sight interval.

9. A robot, comprising:

at least one processor; and

a memory communicably connected to the at least one processor; wherein

the memory stores instructions executable by the at least one processor, wherein, the instructions, when being executed by the at least one processor, cause the at least one processor to perform the steps of:

acquiring sight range information when a voice command issuer issues a voice command;

if the sight range information of the voice command issuer when issuing the voice command is acquired, determining, based on the sight range information, whether the voice command issuer gazes the robot when the voice command is issued; and

determining that the robot is called if the voice command issuer gazes the robot;

interrogating the voice command issuer if the sight range information of the voice command issuer when issuing the voice command is not acquired;

wherein the acquiring sight range information when a voice command issuer issues a voice command comprises:

acquiring voice command information, the voice command information comprising time information of the voice command and information of the voice command issuer;

broadcasting the voice command information;

determining whether a robot that hears the same voice command exists;

enabling the robot that hears the same voice command to join a same candidate group if the robot that hears the same voice command exists;

acquiring visual information and position information of the robot complying with the time information;

acquiring, based on the visual information and the position information of the robot, the sight range information of the voice command issuer when issuing the voice command; and

if the sight range information of the voice command issuer when issuing the voice command exists, broadcasting, in the candidate group, the sight range information of the voice command issuer when issuing the voice command.

10. The robot according to claim 9 , wherein the voice command information further comprises a sound pressure level of the voice command, and the acquiring voice command information comprises:

monitoring a voice signal;

parsing out a wake-up word in the voice signal;

determining a sound pressure level of the voice signal;

if the wake-up word indicates a call and the sound pressure level of the voice signal exceeds a predetermined threshold, recording a start time point and an end time point of the voice signal as the time information of the voice command, and recording the sound pressure level of the voice signal as the sound pressure level of the voice command;

identifying a voice signal issuer based on the voice signal; and

recording the voice signal issuer as the information of the voice command issuer.

11. The robot according to claim 10 , wherein the wake-up word is a pronoun indicating the call; and

the interrogating the voice command issuer comprises:

determining whether the wake-up word is a pronoun indicating a single form;

determining a robot with a highest sound pressure level in the candidate group if the wake-up word is the pronoun indicating the single form, such that the robot with the highest sound pressure level interrogates whether the voice command issuer calls the robot with the highest sound pressure level;

responding to the voice command issuer if the voice command issuer calls the robot with the highest sound pressure level;

otherwise, in a scenario where the voice command issuer issues a new voice command, acquiring sight range information of the voice command issuer when issuing the new voice command;

determining robots with a highest sound pressure level and a second highest sound pressure level in the candidate group if the wake-up word is not the pronoun indicating the single form, such that the robots with the highest sound pressure level and the second highest sound pressure level interrogate whether the voice command issuer calls only the robots with the highest sound pressure level and the second highest sound pressure level;

responding to the voice command issuer if the voice command issuer calls only the robots with the highest sound pressure level and the second highest sound pressure level;

otherwise, in a scenario where the voice command issuer issues a new voice command, acquiring sight range information of the voice command issuer when issuing the new voice command;

broadcasting, in the candidate group, the sight range information of the voice command issuer when issuing the new voice command; and

wherein the instructions, when being executed by the at least one processor, cause the at least one processor further to perform the steps of:

if the sight range information of the voice command issuer when issuing the new voice command is acquired, determining, based on the sight range information, whether the voice command issuer gazes the robot when the voice command is issued; and

determining that the robot is called if the voice command issuer gazes the robot; and

responding to the voice command issuer.

12. The robot according to claim 9 , wherein the voice command information further comprises a sound pressure level of the voice command, and the acquiring voice command information comprises:

monitoring a voice signal;

parsing out a wake-up word in the voice signal;

determining a sound pressure level of the voice signal;

if the wake-up word indicates a call and the sound pressure level of the voice signal exceeds a predetermined threshold, recording a start time point and an end time point of the wake-up word as the time information of the voice command, and recording the sound pressure level of the voice signal as the sound pressure level of the voice command; and

identifying a voice signal issuer based on the voice signal, and recording the voice signal issuer as the information of the voice command issuer.

13. The robot according to claim 9 , wherein the acquiring, based on the visual information and the position information of the robot, the sight range information of the voice command issuer when issuing the voice command comprises:

equally dividing the time information of the voice command into a plurality of time points;

at each time point, determining an angle between the face of the voice command issuer and the robot based on the visual information at the time point;

acquiring a sight direction at the time point based on the position information of the robot at the time point and the angle; and

using the acquired sight direction and a time point corresponding to the sight direction as the sight range information.

14. The robot according to claim 13 , wherein the determining, based on the sight range information, whether the voice command issuer gazes the robot when the voice command is issued comprises:

acquiring the position information of the robot within a predetermined time period;

determining whether the position information complying with the sight direction of the robot at a time point of the sight range information exists; and

determining that the voice command issuer gazes the robot if the position information of the robot complying with the sight direction at a time point of the sight range information exists.

15. The robot according to claim 9 , wherein the acquiring, based on the visual information and the position information of the robot, the sight range information of the voice command issuer when issuing the voice command comprises:

equally dividing the time information of the voice command into a plurality of time points;

at each time point, determining an angle between the face of the voice command issuer and the robot based on the visual information at the time point;

acquiring a sight direction at the time point based on the position information of the robot issuer at the time point and the angle;

acquiring a sight interval and a time range corresponding to the sight interval based on the acquired sight direction and the time point corresponding to the sight direction; and

using the sight interval and the time range as the sight range information.

16. A non-transitory computer-readable storage medium, wherein the computer-readable storage medium stores computer-executable instructions, which, when being executed by a robot, cause the robot to perform the steps of:

acquiring sight range information when a voice command issuer issues a voice command;

if the sight range information of the voice command issuer when issuing the voice command is acquired, determining, based on the sight range information, whether the voice command issuer gazes the robot when the voice command is issued; and

determining that the robot is called if the voice command issuer gazes the robot;

interrogating the voice command issuer if the sight range information of the voice command issuer when issuing the voice command is not acquired;

wherein the acquiring sight range information when a voice command issuer issues a voice command comprises:

acquiring voice command information, the voice command information comprising time information of the voice command and information of the voice command issuer;

broadcasting the voice command information;

determining whether a robot that hears the same voice command exists;

enabling the robot that hears the same voice command to join a same candidate group if the robot that hears the same voice command exists;

acquiring visual information and position information of the robot complying with the time information;

acquiring, based on the visual information and the position information of the robot, the sight range information of the voice command issuer when issuing the voice command; and

if the sight range information of the voice command issuer when issuing the voice command exists, broadcasting, in the candidate group, the sight range information of the voice command issuer when issuing the voice command.

Assignments (4)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 23, 2026
From: DATAA NEW TECHNOLOGY CO., LTD.
To: CHONGQING XINGJIE SHUXING TECHNOLOGY PARTNERSHIP ENTERPRISE (LIMITED PARTNERSHIP)
Reel/Frame 074153/0658 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Aug 18, 2025
From: CLOUDMINDS ROBOTICS CO., LTD.
To: DATAA NEW TECHNOLOGY CO., LTD.
Reel/Frame 072052/0055 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 17, 2021
From: CLOUDMINDS (SHENZHEN) ROBOTICS SYSTEMS CO., LTD.
To: CLOUDMINDS ROBOTICS CO., LTD.
Reel/Frame 055620/0650 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Nov 13, 2019
From: LUO, LEI
To: CLOUDMINDS (SHENZHEN) ROBOTICS SYSTEMS CO., LTD.
Reel/Frame 050999/0379 →
Continuity (2)
Continuation PCTCN2017083424 · May 8, 2017
Related Publication 20200090653A1 · Mar 19, 2020
Cited By (1)
US 12,198,678