IP Library Granted Patent US 11,633,863
Granted Patent B2
US 11,633,863 · App. 15/947,714 · Granted Apr 25, 2023

Condition-based robot audio techniques

Inventors: Jason Wolford (Berkeley, CA); Ben Gabaldon (Oakland, CA); Jordan Rivas (San Francisco, CA); Brian Min (Concord, CA)
B25J11/001B25J9/162B25J13/003G10L15/26
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,633,863
App. No.
15/947,714
Granted
Apr 25, 2023
Kind
B2
Abstract

Methods, systems, and apparatus, including computer programs encoded on computer storage media, for audio augmentation of physical robot sounds. A robot can determine that a first physically moveable component of the robot is to be actuated and in response, obtain a conditional state of the robot. The robot can obtain an audio object that generates an audio enhancement for the first physically moveable component being actuated, the audio enhancement having one or more characteristics that match the obtained conditional state of the robot. The robot can output the audio enhancement while actuating the first physically moveable component.

Claims (76)

1. A robot comprising:

one or more physically moveable components;

one or more speakers;

one or more processors; and

one or more storage devices storing instructions that are operable, when executed by the one or more processors, to cause the robot to perform operations comprising:

determining that a first physically moveable component of the one or more physically moveable components is to be actuated;

obtaining a conditional state of the robot, which conditional state matches a sentiment of music playing in an environment around the robot;

obtaining an audio object that generates a non-text-to-speech audio enhancement for the first physically moveable component being actuated, the audio enhancement having a blend of one or more characteristics that match the obtained conditional state and the audio enhancement simulates a sound produced by actuating the first physically moveable component;

initiating actuation of the first physically moveable component; and

providing the audio object as input to the one or more speakers to output the audio enhancement for the first physically moveable component at least partially concurrently with actuating the first physically moveable component.

2. The robot of claim 1 , wherein the conditional state comprises an emotion state of the robot.

3. The robot of claim 2 , wherein the emotion state of the robot includes a happy state, a sad state, an excited state, a sneaky state, an angry state, or a tired state.

4. The robot of claim 1 , wherein the audio object has one or more associated parameters, and wherein outputting the audio enhancement comprises modifying a sound generated by the audio object according to the one or more associated parameters.

5. The robot of claim 4 , wherein the one or more associated audio parameters include a volume, a pitch, a duration, a frequency, or a sound effect.

6. The robot of claim 5 , wherein the sound effect includes a vibrato effect, an echo effect, a noise cancelling effect, or a maintenance-required effect.

7. The robot of claim 3 , wherein a conditional parameter of the conditional state is a current speed of the robot, and further comprising: determining the current speed of the robot; and modifying the audio enhancement generated by the audio object according to the current speed of the robot.

8. The robot of claim 1 , wherein the operations further comprise: obtaining a second audio object for generating text-to-speech audio output having one or more characteristics that match the obtained conditional state; and providing the second audio output as input to the one or more speakers to output text-to-speech audio having the one or more characteristics that match the obtained conditional state.

9. A method comprising:

determining that a first physically moveable component of the one or more physically moveable components is to be actuated;

obtaining a conditional state of the robot, which conditional state matches a sentiment of music playing in an environment around the robot;

obtaining an audio object that generates a non-text-to-speech audio enhancement for the first physically moveable component being actuated, the audio enhancement having a blend of one or more characteristics that match the obtained conditional state and the audio enhancement simulates a sound produced by actuating the first physically moveable component;

initiating actuation of the first physically moveable component; and

providing the audio object as input to the one or more speakers to output the audio enhancement for the first physically moveable component at least partially concurrently with actuating the first physically moveable component.

10. The method of claim 9 , wherein the conditional state comprises an emotion state of the robot.

11. The method of claim 10 , wherein the emotion state of the robot includes a happy state, a sad state, an excited state, a sneaky state, an angry state, or a tired state.

12. The method of claim 9 , wherein the audio object has one or more associated parameters, and wherein outputting the audio enhancement comprises modifying a sound generated by the audio object according to the one or more associated parameters.

13. The method of claim 12 , wherein the one or more associated audio parameters include a volume, a pitch, a duration, a frequency, or a sound effect.

14. The method of claim 13 , wherein the sound effect includes a vibrato effect, an echo effect, a noise cancelling effect, or a maintenance-required effect.

15. The method of claim 11 , wherein a conditional parameter of the conditional state is a current speed of the robot, and further comprising: determining the current speed of the robot; and modifying the audio enhancement generated by the audio object according to the current speed of the robot.

16. The method of claim 9 , wherein the operations further comprise: obtaining a second audio object for generating text-to-speech audio output having one or more characteristics that match the obtained conditional state; and providing the second audio output as input to the one or more speakers to output text-to-speech audio having the one or more characteristics that match the obtained conditional state.

17. A non-transitory computer readable medium encoded with a computer program, the program comprising

instructions that are operable, when executed by a robot comprising one or more processors, to cause the robot to perform the method of:

determining that a first physically moveable component of one or more physically moveable components is to be actuated;

obtaining a conditional state of the robot, which conditional state matches a sentiment of music playing in an environment around the robot;

obtaining an audio object that generates a non-text-to speech audio enhancement for the first physically moveable component being actuated, the audio enhancement having a blend of one or more characteristics that match the obtained conditional state and the audio enhancement simulates a sound produced by actuating the first physically moveable component;

initiating actuation of the first physically moveable component; and

providing the audio object as input to one or more speakers to output the audio enhancement for the first physically moveable component at least partially concurrently with actuating the first physically moveable component.

18. A robot comprising:

one or more speakers;

one or more processors; and

one or more storage devices storing instructions that are operable, when executed by the one or more processors, to cause the robot to perform operations comprising:

determining that a selected action includes outputting non-text-to-speech audio output using the one or more speakers;

obtaining a conditional state of the robot, which conditional state matches a sentiment of music playing in an environment around the robot;

obtaining an audio object for generating non-text-to-speech audio output having a blend of one or more characteristics that match the obtained conditional state and the audio enhancement simulates a sound produced by actuating the first physically moveable component; and

providing the audio object as input to the one or more speakers to output audio having the one or more characteristics that match the obtained conditional state.

19. A robot comprising:

one or more physically moveable components;

one or more speakers;

one or more processors; and

one or more storage devices storing instructions that are operable, when executed by the one or more processors, to cause the robot to perform operations comprising:

determining that a first physically moveable component of the one or more physically moveable components is to be actuated;

obtaining a conditional state of the robot from at least one touch detection subsystem, which conditional state matches a sentiment of a touch from at least one person in an environment around the robot;

obtaining an audio object that generates a non-text-to-speech audio enhancement for the first physically moveable component being actuated, the audio enhancement having a blend of one or more characteristics that match the obtained conditional state and the audio enhancement simulates a sound produced by actuating the first physically moveable component;

initiating actuation of the first physically moveable component; and

providing the audio object as input to the one or more speakers to output the audio enhancement for the first physically moveable component at least partially concurrently with actuating the first physically moveable component.

20. A method comprising:

determining that a first physically moveable component of the one or more physically moveable components is to be actuated;

obtaining a conditional state of a robot from at least one touch detection subsystem, which conditional state matches a sentiment of a touch from at least one person in an environment around the robot;

obtaining an audio object that generates a non-text-to-speech audio enhancement for the first physically moveable component being actuated, the audio enhancement having a blend of one or more characteristics that match the obtained conditional state and the audio enhancement simulates a sound produced by actuating the first physically moveable component;

initiating actuation of the first physically moveable component; and

providing the audio object as input to the one or more speakers to output the audio enhancement for the first physically moveable component at least partially concurrently with actuating the first physically moveable component.

21. A non-transitory computer readable medium encoded with a computer program, the program comprising

instructions that are operable, when executed by a robot comprising one or more processors, to cause the robot to perform the method of:

determining that a first physically moveable component of one or more physically moveable components is to be actuated;

obtaining a conditional state of the robot from at least one touch detection subsystem, which conditional state matches a sentiment of a touch from a person in an environment around the robot;

obtaining an audio object that generates a non-text-to speech audio enhancement for the first physically moveable component being actuated, the audio enhancement having a blend of one or more characteristics that match the obtained conditional state and the audio enhancement simulates a sound produced by actuating the first physically moveable component;

initiating actuation of the first physically moveable component; and

providing the audio object as input to one or more speakers to output the audio enhancement for the first physically moveable component at least partially concurrently with actuating the first physically moveable component.

22. A robot comprising:

one or more speakers;

one or more processors; and

one or more storage devices storing instructions that are operable, when executed by the one or more processors, to cause the robot to perform operations comprising:

determining that a selected action includes outputting non-text-to-speech audio output using the one or more speakers;

obtaining a conditional state of the robot from at least one touch detection subsystem, which conditional state matches a sentiment of a touch from a person in an environment around the robot;

obtaining an audio object for generating non-text-to-speech audio output having a blend of one or more characteristics that match the obtained conditional state and an audio enhancement that simulates a sound produced by actuating a first physically moveable component on the robot; and

providing the audio object as input to the one or more speakers to output audio having the one or more characteristics that match the obtained conditional state.

Assignments (6)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Apr 28, 2022
From: DIGITAL DREAM LABS, LLC
To: DIGITAL DREAM LABS, INC.
Reel/Frame 059819/0720 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 24, 2020
From: DSI ASSIGNMENTS, LLC
To: DIGITAL DREAM LABS, LLC
Reel/Frame 052211/0235 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 19, 2020
From: ANKI, INC.
To: DSI ASSIGNMENTS, LLC
Reel/Frame 052190/0487 →
RELEASE OF SECURITY INTEREST Recorded Aug 13, 2019
From: FISH & RICHARDSON
To: ANKI, INC
Reel/Frame 050034/0151 →
LIEN Recorded Jun 3, 2019
From: ANKI, INC.
To: FISH & RICHARDSON P.C.
Reel/Frame 049342/0887 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 1, 2018
From: WOLFORD, JASON; GABALDON, BEN; RIVAS, JORDAN; MIN, BRIAN
To: ANKI, INC.
Reel/Frame 045970/0074 →
Continuity (1)
Related Publication 20190308327A1 · Oct 10, 2019