Granted Patent B2

US 12,564,953 · App. 18/025,928 · Granted Mar 3, 2026

Inventors: Edward Adelson (Winchester, MA); Branden Romero (Brighton, MA); Filipe Fernandes Veiga (Cambridge, MA); Shaoxiong Wang (Cambridge, MA); Chen Wang (Shanghai, CN)

Assignee: Massachusetts Institute of Technology

B25J9/1664B25J9/161B25J9/1612B25J9/163B25J9/1653B25J13/084

Abstract

Claims (86)

1 . A method for operating a robot, the method comprising:

causing the robot to grasp an object with an effector, the effector including a tactile sensor providing measurements representative of physical interaction of the effector and the object;

causing the robot to perform a set of manipulations of the object using the effector;

acquiring measurements from the tactile sensor during said manipulations;

forming a data representation of physical characteristics of the object from the acquired measurements; and

controlling a further motion of the object according to the data representation of the physical characteristics and a characteristic of a desired motion of the object,

wherein

causing the robot to perform the set of manipulations of the object comprises causing a set of dynamic manipulations of said object,

acquiring the measurements from the tactile sensor during said manipulations comprises acquiring dynamic measurements based at least in part on kinematic properties of said object, and

forming the data representation of the physical characteristics of the object comprises forming said representation to encode the kinematic properties of said object.

2 . The method of claim 1 wherein performing the set of manipulations comprises performing at least one of a tilting manipulation of the object and a swinging manipulation of the object.

3 . The method of claim 2 , wherein performing the set of manipulations comprises performing the tilting manipulation of the object and performing the swinging manipulation of the object.

4 . The method of claim 1 , wherein controlling the further motion of the object comprises performing a swing-up motion of the object.

5 . The method of claim 1 , wherein the tactile sensor provides measurements of characteristics of a physical interaction between the effector and the object over an area of contact between the effector and the object.

6 . The method of claim 5 , wherein the measurements of characteristics of the physical interaction comprise measurements of displacement and/or force at a plurality of locations in an area of contact between the effector and the object.

7 . The method of claim 5 , wherein acquiring the measurements from the tactile sensor comprises acquiring a time sequence of measurements of the characteristics over the area of contact.

8 . The method of claim 1 , wherein forming the data representation of the physical characteristics of the object comprises using a first machine-learned transformation configured with values of configuration parameters.

9 . The method of claim 8 , wherein the values of the configuration parameters are determined using a plurality of objects manipulated using known controls of motion of said objects.

10 . The method of claim 9 , further comprising determining the configuration parameters using a plurality of objects using the known controls of motion.

11 . The method of claim 10 , wherein determining the configuration parameters includes:

manipulating each of object of the plurality of objects with the effector;

acquiring measurements from the tactile sensor during said manipulations;

using the acquired measurements and known characteristics of the objects of the plurality of objects to determine the configuration parameters.

12 . The method of claim 8 , wherein using the first machine-learned transformation comprises using a first artificial neural network trained using sensor measurements acquired during manipulation of the objects with known controls of motion.

13 . The method of claim 12 , wherein the first artificial neural network comprises a convolutional neural network.

14 . The method of claim 12 , wherein the data representation of physical characteristics of the object comprises a numerical embedding vector produced as output for the first artificial neural network representing said characteristics.

15 . The method of claim 1 , wherein controlling the further motion of the object according to the data representation of the physical characteristics and the characteristics of a desired motion of the object comprises using a computation model.

16 . The method of claim 15 , wherein the computational model comprises a second artificial neural network.

17 . The method of claim 15 , wherein the computational model implements a mapping of a control of motion to an achieved motion for an object based on the data representation of physical characteristics of the object.

18 . A machine-implemented controller for a robot configured to implement all the steps of:

causing the robot to grasp an object with an effector, the effector including a tactile sensor providing measurements representative of physical interaction of the effector and the object;

causing the robot to perform a set of manipulations of the object using the effector;

acquiring measurements from the tactile sensor during said manipulations;

forming a data representation of physical characteristics of the object from the acquired measurements; and

controlling a further motion of the object according to the data representation of the physical characteristics and a characteristic of a desired motion of the object

wherein

causing the robot to perform the set of manipulations of the object comprises causing a set of dynamic manipulations of said object,

acquiring the measurements from the tactile sensor during said manipulations comprises acquiring dynamic measurements based at least in part on kinematic properties of said object, and

forming the data representation of the physical characteristics of the object comprises forming said representation to encode the kinematic properties of said object.

19 . A non-transitory machine-readable medium comprising instructions stored thereon, wherein the instructions when executed by a machine-implemented controller cause the controller to perform all the steps of:

causing the robot to grasp an object with an effector, the effector including a tactile sensor providing measurements representative of physical interaction of the effector and the object;

causing the robot to perform a set of manipulations of the object using the effector;

acquiring measurements from the tactile sensor during said manipulations;

forming a data representation of physical characteristics of the object from the acquired measurements; and

controlling a further motion of the object according to the data representation of the physical characteristics and a characteristic of a desired motion of the object

wherein

causing the robot to perform the set of manipulations of the object comprises causing a set of dynamic manipulations of said object,

acquiring the measurements from the tactile sensor during said manipulations comprises acquiring dynamic measurements based at least in part on kinematic properties of said object, and

forming the data representation of the physical characteristics of the object comprises forming said representation to encode the kinematic properties of said object.

20 . A robot comprising:

an effector having a tactile sensor for providing measurements representative of physical interaction of the effector and objects grasped by the effector;

a controller configured to implement all the steps of:

causing the robot to grasp an object with the effector, including causing the tactile sensor to provide measurements representative of physical interaction of the effector and the object;

causing the robot to perform a set of manipulations of the object using the effector;

acquiring measurements from the tactile sensor during said manipulations;

forming a data representation of physical characteristics of the object from the acquired measurements; and

controlling a further motion of the object according to the data representation of the physical characteristics and a characteristic of a desired motion of the object

wherein

causing the robot to perform the set of manipulations of the object comprises causing a set of dynamic manipulations of said object,

acquiring the measurements from the tactile sensor during said manipulations comprises acquiring dynamic measurements based at least in part on kinematic properties of said object, and

forming the data representation of the physical characteristics of the object comprises forming said representation to encode the kinematic properties of said object.

21 . The method of claim 2 , wherein

acquiring the measurements from the tactile sensor comprises acquiring said measurements during the performing of the at least one of the tilting manipulation and the swinging manipulation, and

forming the data representation of the physical characteristics of the object comprises forming a data representation using dynamic measurements acquired during said tilting or swinging manipulation.

22 . The method of claim 1 , wherein the kinematic properties include a moment of inertial of said object.

23 . The method of claim 1 , wherein

controlling the further motion comprises the robot controlling a dynamic manipulation of said object wherein a response of the object to said robot control depends on kinematic properties of said object.

24 . The method of claim 1 , wherein acquiring the measurements from the tactile sensor during said manipulations comprises:

acquiring a time sequence of measurements of displacement and/or force at a plurality of locations in an area of contact between the effector and the object during the performance of the set of dynamic manipulations; and

forming the data representation comprises processing said time sequence of measurements to form said data representation.

25 . The method of claim 24 , wherein processing the time sequence of measurements comprises:

providing the sequence of measurements as input to a first machine-learned transformation configured with values of first configuration parameters; and

forming the data representation as a numerical embedding vector;

wherein controlling the further motion of the object comprises controlling said motion according to the numerical embedding vector and the characteristics of the desired motion of the object.

26 . The method of claim 25 , wherein controlling the further motion of the object comprises:

wherein controlling the further motion of the object comprises providing the numerical embedding vector and a data representation of the characteristics of the desired motion of the object as input to a second machine-learned transformation configured with values a second configuration parameters, and

using an output of the second machine-learned transformation to control limbs and/or effectors of the robot.

27 . A method for operating a robot, the method comprising:

causing the robot to grasp an object with an effector, the effector including a tactile sensor providing measurements representative of physical interaction of the effector and the object;

causing the robot to perform a set of manipulations of the object using the effector;

acquiring measurements from the tactile sensor during said manipulations;

forming a data representation of physical characteristics of the object from the acquired measurements; and

controlling a further motion of the object according to the data representation of the physical characteristics and a characteristic of a desired motion of the object,

wherein

causing the robot to grasp the object comprises causing the robot to grasp an elongated object extending beyond the effector, and

acquiring the measurements from the tactile sensor during the manipulations comprises acquiring measurements dependent on mass distribution of the elongated object.

Assignments (1)

Continuity (2)

References Cited (18)