Software compensated robotics
A software compensated robotic system makes use of recurrent neural networks and image processing to control operation and/or movement of an end effector. Images are used to compensate for variations in the response of the robotic system to command signals. This compensation allows for the use of components having lower reproducibility, precision and/or accuracy that would otherwise be practical.
1 . A robotic system comprising:
an end effector;
a movement generation device configured to move the end effector;
a camera configured to generate images of the end effector; and
a computer vision software pipeline configured to:
generate first control signals, the first control signals being configured to generate an expected movement of the end effector;
receive a first image of a portion of an environment including the end effector, the first image showing a response of the robotic system to the first control signals;
change a state in the computer vision software pipeline responsive to the first image and the expected movement;
generate second control signals for the end effector;
compensate the second control signals to produce compensated control signals, the compensation being responsive to the changed state in the computer vision software pipeline, the compensation being configured to reduce a difference between the expected movement and a movement of the end effector indicated by the image; and
move the end effector based on the compensated control signals.
2 . The robotic system of claim 1 wherein the expected movement is to move the end effector to a position relative to an object in the image.
3 . The robotic system of claim 2 wherein the expected movement is to grip the object.
4 . The robotic system of claim 1 wherein the computer vision software pipeline includes at least one neural network.
5 . The robotic system of claim 4 wherein the at least one neural network includes a recurrent neural network.
6 . The robotic system of claim 1 wherein the computer vision software pipeline comprises:
a perception block configured to receive the image and generate an image processing output representative of a state of the end effector within the image;
a policy block configured to generate the second signals based on at least the expected movement of the end effector; and
a compensation block configured to compensate the second control signals to produce the compensated control signals.
7 . The robotic system of claim 1 wherein the movement generation device includes at least one component selected from a group consisting of: a DC motor; a hydraulic device; a synthetic muscle; a pneumonic device; a piezoelectric device; a linear actuator; a rotational actuator; an AC motor; an electromagnetic driver; a stepper motor; a servo; and a tendon.
8 . The robotic system of claim 1 wherein the computer vision software pipeline comprises one or more non-transitory computer-readable storage media storing computer-executable instructions for causing the robotic system to provide control signals to the movement generation device, the control signals being configured to reach a goal in a movement of the end effector.
9 . The robotic system of claim 1 wherein the end effector has a spatial pose adjustable, at least in part, by the movement generation device, and wherein the expected movement includes a target pose for the end effector.
10 . The robotic system of claim 1 , further comprising:
a robotic arm movable in three-dimensional space, wherein the end effector is coupled to the robotic arm and wherein the movement generation device is configured to move the end effector by: i) moving the robotic arm; and/or ii) adjusting a spatial pose of the end effector.
11 . A method of calibrating a robotic system, the method comprising:
generating first control signals, the first control signals being configured to generate an expected movement of an end effector of the robotic system;
receiving a first image of a portion of an environment including the end effector, the first image showing a response of the robotic system to the first control signals;
changing a state in a computer vision software pipeline responsive to the first image and the expected movement;
generating second control signals for the end effector; compensating the second control signals to produce compensated control signals, the compensation being responsive to the changed state in the computer vision software pipeline, the compensation being configured to reduce a difference between the expected movement and a movement of the end effector indicated by the image; and
moving the end effector based on the compensated control signals.
12 . The method of claim 11 wherein the first image of the portion of the environment further includes an object in the environment, and wherein generating first control signals for the end effector includes generating first control signals to cause the end effector to move to a position relative to the object.
13 . The method of claim 12 wherein generating first control signals to cause the end effector to move to a position relative to the object incudes generating first control signals to cause the end effector to grip the object.
14 . The method of claim 11 wherein the computer vision software pipeline includes a recurrent neural network.
15 . A computer vision software pipeline comprising one or more non-transitory computer-readable storage media storing computer-executable instructions for causing a robotic system to perform operations comprising:
generating first control signals, the first control signals being configured to generate an expected movement of an end effector of the robotic system;
receiving a first image of a portion of an environment including the end effector, the first image showing a response of the robotic system to the first control signals;
changing a state in a computer vision software pipeline responsive to the first image and the expected movement;
generating second control signals for the end effector;
compensating the second control signals to produce compensated control signals, the compensation being responsive to the changed state in the computer vision software pipeline, the compensation being configured to reduce a difference between the expected movement and a movement of the end effector indicated by the image; and
moving the end effector based on the compensated control signals.
16 . The computer vision software pipeline of claim 15 wherein the first image of the portion of the environment further includes an object in the environment, and wherein generating first control signals for the end effector includes generating first control signals to cause the end effector to grip the object.
17 . The computer vision software pipeline of claim 15 wherein the computer-executable instructions for causing the robotic system to perform operations includes at least one recurrent neural network.