Environmental feature-specific actions for robot navigation
Systems and methods are described for reacting to a feature in an environment of a robot based on a classification of the feature. A system can detect the feature in the environment using a first sensor on the robot. For example, the system can detect the feature using a feature detection system based on sensor data from a camera. The system can detect a mover in the environment using a second sensor on the robot. For example, the system can detect the mover using a mover detection system based on sensor data from a lidar sensor. The system can fuse the data from detecting the feature and detecting the mover to produce fused data. The system can classify the feature based on the fused data and react to the feature based on classifying the feature.
1 . A method for operating a robot, comprising:
detecting, by a feature detector of the robot, using image sensor data obtained from a first sensor on the robot, a feature in an environment about the robot;
detecting, by a mover detector of the robot, using lidar sensor data obtained from a second sensor on the robot, a mover in the environment about the robot;
fusing, by a computing system of the robot, first data associated with the feature and second data associated with the mover to produce fused data based on an output generated by the feature detector and an output generated by the mover detector, wherein a portion of the environment is classified into a classification of a set of moving feature classifications and non-moving feature classifications based on at least a portion of the fused data; and
based on the classification, reacting to the portion of the environment.
2 . The method of claim 1 , wherein reacting to the portion of the environment comprises communicating with one or more of:
the mover;
a computing device; or
another robot.
3 . The method of claim 1 , wherein reacting to the portion of the environment comprises communicating an electronic communication to one or more of the mover or a computing system associated with the mover.
4 . The method of claim 1 , wherein reacting to the portion of the environment comprises implementing one or more physical gestures by the robot towards the mover.
5 . The method of claim 1 , wherein reacting to the portion of the environment comprises instructing display of a user interface oriented towards the mover.
6 . The method of claim 1 , wherein reacting to the portion of the environment comprises communicating an alert to the mover.
7 . The method of claim 6 , wherein the alert comprises an audible alert, a visual alert, or a physical alert.
8 . The method of claim 1 , wherein the set of moving feature classifications and non-moving feature classifications comprises a human classification and one or more non-human classifications.
9 . The method of claim 1 , wherein the portion of the environment comprises:
a mover located in a route of the robot;
a mover located within a threshold distance of a route of the robot; or
a mover projected to be located in a route of the robot within a threshold period of time.
10 . The method of claim 1 , wherein detecting the feature comprises determining whether the feature is an animal using a vision-based animal detection system.
11 . The method of claim 1 , wherein reacting to the portion of the environment comprises determining a route of the robot in the environment.
12 . The method of claim 1 , wherein reacting to the portion of the environment comprises adjusting a route of the robot in the environment.
13 . The method of claim 1 , wherein reacting to the portion of the environment comprises adjusting a navigational behavior of the robot.
14 . The method of claim 13 , wherein adjusting the navigational behavior comprises:
adjusting a speed of the robot based on one or more of a location, velocity, or direction of the mover; or
avoiding a space relative to the mover.
15 . The method of claim 1 , wherein the portion of the environment is classified based on output of a machine learning model, wherein the machine learning model is trained to classify the portion of the environment into one or more classifications.
16 . The method of claim 15 , further comprising:
obtaining training data; and
retraining the machine learning model using the training data.
17 . The method of claim 15 , further comprising:
implementing the machine learning model; and
obtaining the output of the machine learning model based on implementing the machine learning model.
18 . The method of claim 1 , further comprising causing display of a user interface via a user computing device, wherein the user interface comprises information based on the classification.
19 . The method of claim 1 , further comprising causing display of a user interface via a user computing device, wherein the user interface indicates the classification.
20 . The method of claim 1 , wherein the first data and the second data have different data types.
21 . A legged robot comprising:
a plurality of legs;
memory storing computer-executable instructions; and
a processor in communication with the memory, wherein the processor is configured to execute the computer-executable instructions, wherein execution of the computer-executable instructions causes the processor to:
detect, by a feature detector of the legged robot, using image sensor data obtained from a first sensor on the legged robot, a feature in an environment about the legged robot;
detect, by a mover detector of the legged robot, using lidar sensor data obtained from a second sensor on the legged robot, a mover in the environment about the legged robot;
fuse first data associated with the feature and second data associated with the mover to produce fused data based on an output generated by the feature detector and an output generated by the mover detector, wherein a portion of the environment is classified into a classification of a set of moving feature classifications and non-moving feature classifications based on at least a portion of the fused data; and
based on the classification, react to the portion of the environment.
22 . The legged robot of claim 21 , wherein the processor and computer-executable instructions are configured to react to the portion of the environment by performing an action.
23 . The legged robot of claim 22 , wherein the processor and computer-executable instructions are configured to determine the action based on the classification.
24 . The legged robot of claim 21 , wherein the classification is a human classification.
25 . The legged robot of claim 21 , wherein the output generated by the feature detector is different from the image sensor data, and wherein the output generated by the mover detector is different from the lidar sensor data.
26 . A computing system comprising:
data processing hardware; and
memory hardware in communication with the data processing hardware, the memory hardware storing instructions, wherein execution of the instructions by the data processing hardware causes the data processing hardware to:
detect, by a feature detector of a legged robot, using image sensor data obtained from a first sensor on the legged robot, a feature in an environment about the legged robot;
detect, by a mover detector of the legged robot, using lidar sensor data obtained from a second sensor on the legged robot, a mover in the environment about the legged robot;
fuse first data associated with the feature and second data associated with the mover to produce fused data based on an output generated by the feature detector and an output generated by the mover detector, wherein portion of the environment is classified into a classification of a set of moving feature classifications and non-moving feature classifications based on at least a portion of the fused data; and
based on the classification, react to the portion of the environment.
27 . The computing system of claim 26 , wherein the classification identifies the feature as a human, wherein the data processing hardware and the instructions are configured to react to the portion of the environment by physically interacting with the human based on the classification identifying the feature as the human.
28 . The computing system of claim 26 , wherein the classification identifies the feature as a human, wherein the data processing hardware and the instructions are configured to react to the portion of the environment by instructing the legged robot to navigate away from the human based on the classification identifying the feature as the human.
29 . The computing system of claim 26 , wherein the set of moving feature classifications and non-moving feature classifications are associated with a set of navigational behaviors.
30 . The computing system of claim 29 , wherein a first classification of the set of moving feature classifications and non-moving feature classifications is associated with a first navigational behavior, wherein a second classification of the set of moving feature classifications and non-moving feature classifications is associated with a second navigational behavior, wherein the second navigational behavior is qualitatively different from the first navigational behavior.