Positioning a robot sensor for object classification
In one embodiment, a method includes receiving, from a first sensor on a robot, first sensor data indicative of an environment of the robot. The method also includes identifying, based on the first sensor data, an object of an object type in the environment of the robot, where the object type is associated with a classifier that takes sensor data from a predetermined pose relative to the object as input. The method further includes causing the robot to position a second sensor on the robot at the predetermined pose relative to the object. The method additionally includes receiving, from the second sensor, second sensor data indicative of the object while the second sensor is positioned at the predetermined pose relative to the object. The method further includes determining, by inputting the second sensor data into the classifier, a property of the object.
1. A method comprising:
receiving, from a first sensor on a robot, first sensor data indicative of an environment of the robot;
identifying, based on the first sensor data, an object in the environment of the robot;
selecting, based on identifying an object type of the object, a predetermined pose from a plurality of predetermined poses corresponding to a plurality of object types;
selecting a classifier associated with the object type and the selected predetermined pose, wherein the classifier comprises a machine learned model trained based on sensor data representing objects captured from the selected predetermined pose;
causing the robot to position a second sensor on the robot at the selected predetermined pose relative to the object, wherein the second sensor is located on an appendage of the robot;
receiving, from the second sensor, second sensor data indicative of the object while the second sensor is positioned at the selected predetermined pose relative to the object; and
determining, by inputting the second sensor data into the classifier, a property of the object; and
controlling the robot based on the determined property of the object.
2. The method of claim 1 , wherein the appendage is proximate to an end effector of the robot, and wherein controlling the robot based on the determined property of the object comprises controlling the robot to position the end effector.
3. The method of claim 1 , further comprising:
determining, by inputting the second sensor data into a detector, a location of the object, wherein the detector is configured to take images of objects from the selected predetermined pose as input; and
controlling the robot to interact with the object based on the determined location of the object.
4. The method of claim 1 , wherein the classifier is a manipulability classifier that is configured to output whether the object is manipulatable by the robot, and wherein the method further comprises:
when the manipulability classifier indicates that the object is manipulatable by the robot, controlling the robot to manipulate the object; and
when the manipulability classifier indicates that the object is not manipulatable by the robot, controlling the robot to leave the environment without touching the object.
5. The method of claim 4 , wherein the manipulability classifier is configured to output a manipulability score, and wherein the method further comprises determining whether the object is manipulatable by the robot by comparing the manipulability score to a threshold score.
6. The method of claim 5 , further comprising adjusting the threshold score based on the environment of the robot.
7. The method of claim 5 , further comprising receiving the threshold score via a user interface of a mobile computing device.
8. The method of claim 1 , wherein the classifier is a machine learned model trained based on images captured by the robot or by a similar robot.
9. The method of claim 1 , further comprising:
causing the robot to position the second sensor at the selected predetermined pose relative to a different object;
receiving, from the second sensor, sensor data indicative of the different object while the second sensor is positioned at the selected predetermined pose relative to the different object; and
using the sensor data to train the classifier.
10. The method of claim 1 , wherein the object is a container, and wherein the selected predetermined pose of the second sensor is a downward facing pose at a corresponding predetermined height above the container.
11. The method of claim 10 , further comprising:
causing the robot to position the second sensor at the corresponding predetermined height above a different container at a plurality of different horizontal positions;
receiving sensor data from the second sensor when the second sensor is positioned at each of the plurality of different horizontal positions; and
using the sensor data to train the classifier.
12. The method of claim 10 , wherein the appendage is proximate to a robotic gripping device, and wherein the method further causes:
determining to pick up the container with the robotic gripping device based on the property of the container;
horizontally positioning the robotic gripping device based on the second sensor data; and
controlling the robotic gripping device to pick up the container.
13. The method of claim 1 , wherein the classifier takes the first sensor data as additional input.
14. The method of claim 1 , further comprising determining a bounding box around the object based on the first sensor data, wherein causing the robot to position the second sensor at the selected predetermined pose relative to the object is based on the bounding box.
15. The method of claim 1 , wherein the object is a food container, wherein the classifier is a removability classifier that is configured to output whether the food container is removable by the robot, and wherein the method further comprises:
when the removability classifier indicates that the food container is removable by the robot, controlling the robot to pick up and remove the food container from the environment.
16. The method in claim 1 , wherein the classifier is a second classifier, wherein identifying the object is performed by inputting the first sensor data into a first classifier.
17. A robot comprising:
an appendage;
a first sensor;
a second sensor located on the appendage; and
a control system configured to:
receive, from the first sensor, first sensor data indicative of an environment of the robot;
identify, based on the first sensor data, an object in the environment of the robot;
select, based on identifying an object type of the object, a predetermined pose from a plurality of predetermined poses corresponding to a plurality of object types:
select a classifier associated with the object type and the selected predetermined pose, wherein the classifier comprises a machine learned model trained based on sensor data representing objects captured from the selected predetermined pose;
cause the robot to position the second sensor at the selected predetermined pose relative to the object;
receive, from the second sensor, second sensor data indicative of the object while the second sensor is positioned at the selected predetermined pose relative to the object; and
determine, by inputting the second sensor data into the classifier, a property of the object; and
control the robot based on the determined property of the object.
18. The robot of claim 17 , wherein the appendage is proximate to a robotic gripping device, and wherein the control system is configured to control the robot based on the determined property of the object by controlling the robotic gripping device based on the property of the object.
19. The robot of claim 17 , further comprising a robotic head, wherein the first sensor is coupled to the robotic head, and wherein the control system is further configured to determine the property of the object by inputting the first sensor data received from the first sensor coupled to the robotic head into the classifier.
20. A non-transitory computer readable medium having stored therein instructions executable by one or more processors to cause the one or more processors to perform functions comprising:
receiving, from a first sensor on a robot, first sensor data indicative of an environment of the robot;
identifying, based on the first sensor data, an object in the environment of the robot;
selecting, based on identifying an object type of the object, a predetermined pose from a plurality of predetermined poses corresponding to a plurality of object types;
selecting a classifier associated with the object type and the selected predetermined pose, wherein the classifier comprises a machine learned model trained based on sensor data representing objects captured from the selected predetermined pose;
causing the robot to position a second sensor on the robot at the selected predetermined pose relative to the object, wherein the second sensor is located on an appendage of the robot;
receiving, from the second sensor, second sensor data indicative of the object while the second sensor is positioned at the selected predetermined pose relative to the object; and
determining, by inputting the second sensor data into the classifier, a property of the object; and
controlling the robot based on the determined property of the object.