Spatial acoustic filtering by a mobile robot
This specification relates to robots and audio processing in robots. In general, one innovative aspect of the subject matter described in this specification can be embodied in a robot that includes: a body and one or more physically moveable components; a plurality of microphones and one or more other sensor subsystems; one or more processors; and one or more storage devices storing instructions that are operable, when executed by the one or more processors, to cause the robot to perform operations. The operations can include: receiving one or more sensor inputs from the one or more other sensor subsystems; determining a predicted direction of a detected sound emitter based on the one or more sensor inputs of the one or more other sensor subsystems; calculating a spatial filter based on the predicted direction; obtaining, by the plurality of microphones, respective audio inputs; and processing the respective audio inputs according to the calculated spatial filter.
1. A robot comprising:
a body and one or more physically moveable components;
a plurality of microphones and one or more other sensor subsystems;
one or more processors; and
one or more storage devices storing instructions that are operable, when executed by the one or more processors, to cause the robot to perform operations comprising:
receiving one or more sensor inputs from the one or more other sensor subsystems;
determining a predicted direction of a detected sound emitter based on the one or more sensor inputs of the one or more other sensor subsystems;
calculating a spatial filter based on the predicted direction;
obtaining, by the plurality of microphones, respective audio inputs; and
processing the respective audio inputs according to the calculated spatial filter.
2. The robot of claim 1 , wherein the operations further comprise:
computing a location of a sound source based on the processed audio inputs;
determining a first orientation of the robot relative to the location of the sound source;
performing, by the robot, one or more movement actions;
calculating a second orientation of the robot relative to the location of the sound source based on the one or more movement actions;
computing a difference between the first orientation and the second orientation;
updating the spatial filter based on the computed difference between the first orientation and the second orientation; and
processing the respective audio inputs according to the updated spatial filter.
3. The robot of claim 2 , wherein the operations further comprise:
changing the position of the plurality of microphones based on one or more factors including input received by the plurality of microphones, input received by the one or more other sensor subsystems, and the computed difference between the first orientation and the second orientation.
4. The robot of claim 1 , wherein the one or more other sensor subsystems comprise a vision subsystem, and wherein determining a predicted direction of a detected sound emitter based on the one or more sensor inputs of the one or more other sensor subsystems comprises:
determining a direction of an object using the sensor inputs of the vision subsystem; and
computing the predicted direction of the detected sound emitter based on the direction of the object.
5. The robot of claim 4 , wherein determining the direction of the object comprises determining a direction of a face or mouth movement.
6. The robot of claim 1 , wherein determining a predicted direction of a detected sound based on the one or more sensor inputs of the one or more other sensor subsystems comprises:
receiving one or more sensor inputs from an auxiliary sensor in the environment of the robot;
determining a direction of the auxiliary sensor; and
computing the predicted direction of the detected sound based on the direction of the auxiliary sensor.
7. The robot of claim 6 , wherein the auxiliary sensor is a home security sensor configured to detect a door opening, glass breaking, or motion.
8. The robot of claim 1 , wherein determining a predicted direction of a detected sound based on the one or more sensor inputs of the one or more other sensor subsystems comprises:
obtaining a representation of walls in an environment of the robot; and
computing the predicted direction of the detected sound based on the representation of walls in the environment of the robot.
9. The robot of claim 1 , wherein the operations further comprise:
determining, based on a map corresponding to an area accessible to the robot, a list of directions from which a probability of receiving an audio input is lower than a certain threshold value; and
calculating a spatial filter based on the determined list of directions.
10. The robot of claim 9 , wherein the operations further comprise:
selecting on the map a test location;
moving the robot to the test location;
emitting a test signal while at the test location;
receiving an audio input corresponding to a reflection of the test signal;
calculating signal-processing information based on the received audio input; and
storing the test location and the calculated signal-processing information in a database; and
processing the respective audio inputs according to the stored signal-processing information.
11. The robot of claim 10 , wherein the signal processing information includes a spatial filter and a transfer function that takes as input an audio signal and computes a distorted audio signal whose amount of distortion corresponds to the test location.
12. The robot of claim 1 , determining a predicted direction of a detected sound emitter based on the one or more sensor inputs of the one or more other sensor subsystems further comprises determining a relative location of the detected sound emitter.