Controlling robotic devices to perform tasks
A method for controlling a robotic device to perform a task may include determining a force measurement corresponding to a section of sensors coupled with a digit of a plurality of digits of a robotic device. The force measurement may include a magnitude of a force and a position of a centroid of the force in the section that are determined based on contact with an object corresponding to a time. The method may further include determining a digit position of the digit corresponding to the time, and moving the digit, based on a prediction, to stabilize the object to perform a task. A machine learning model can generate the prediction based on the force measurement and the digit position. Other aspects are also described and claimed.
1 . A method for controlling a robotic device to perform a task, comprising:
determining a force measurement corresponding to a section of sensors coupled with a digit of a plurality of digits of a robotic device, wherein the force measurement includes a magnitude of a force and a position of a centroid of the force in the section that are determined based on contact with an object corresponding to a time;
determining a digit position of the digit corresponding to the time; and
moving the digit, based on a prediction, to stabilize the object to perform a task, wherein a machine learning model generates the prediction based on the force measurement and the digit position.
2 . The method of claim 1 , wherein the machine learning model is trained based on force measurements from a section of sensors coupled with a digit of a demonstration device utilized to perform the task with a demonstration object.
3 . The method of claim 1 , wherein the machine learning model comprises a plurality of encoders, an activation layer, a fully connected layer, and a decoder.
4 . The method of claim 1 , wherein the machine learning model operates on a vector comprising a concatenation of features extracted from force measurements, digit positions, and target data.
5 . The method of claim 1 , wherein the prediction moves multiple digits of the plurality of digits in contact with the object to achieve an equilibrium of the object.
6 . The method of claim 1 , wherein the digit is a robotic thumb, and wherein the prediction moves a robotic finger having a section of sensors and the robotic thumb in contact with the object to achieve an equilibrium of the object.
7 . The method of claim 1 , wherein the machine learning model generates the prediction based on force measurements and digit positions corresponding to a plurality of times determined by a sampling frequency.
8 . The method of claim 1 , wherein the machine learning model generates the prediction based on a plurality of force measurements and a plurality of digit positions corresponding to a plurality of digits in contact with the object.
9 . The method of claim 1 , wherein the digit is a robotic finger or thumb having at least one section of force sensors.
10 . The method of claim 1 , wherein the section is wrapped in three dimensions over the digit, and wherein the position of the centroid includes X, Y, Z coordinates.
11 . The method of claim 1 , wherein the force measurement comprises a force vector that is normal to the section to indicate a normal force.
12 . The method of claim 1 , wherein the force measurement comprises a force vector that is tangential to the section to indicate a shear force.
13 . The method of claim 1 , further comprising:
utilizing a decrease of force in a frame of the section to detect slip of the object in contact with the digit.
14 . The method of claim 1 , wherein the digit position comprises one or more angles corresponding to one or more joints of the digit.
15 . The method of claim 1 , wherein the magnitude of the force and the position of the centroid are determined based on an instantaneous distribution of forces among sensors of the section.
16 . The method of claim 1 , wherein the position of the centroid is determined based on a local coordinate frame of the section.
17 . The method of claim 1 , wherein the digit is moved to achieve a target pose of the object specified by at least one of cartesian positions, Euler angles, or quaternions.
18 . The method of claim 1 , wherein the digit is moved to achieve a target image of the object comprising an RGB-D image.
19 . The method of claim 1 , wherein the digit is moved to achieve a target force profile comprising an array of force vectors or a single force vector and a corresponding centroid location.
20 . A system utilized to perform a task with an object, comprising:
a robotic device having a plurality of digits, each digit having a section of sensors; and
one or more processors executing instructions stored in memory to:
determine a force measurement corresponding to a section of a digit of the plurality of digits, wherein the force measurement includes a magnitude of a force and a position of a centroid of the force in the section that are determined based on contact with an object corresponding to a time;
determine a digit position of the digit corresponding to the time; and
move the digit, based on a prediction, to stabilize the object to perform a task, wherein a machine learning model generates the prediction based on the force measurement and the digit position.