IP Library › Granted Patent US 12,384,039
Granted Patent B2
US 12,384,039 · App. 17/845,698 · Granted Aug 12, 2025

Learning to acquire and adapt contact-rich manipulation skills with motion primitives

Inventors: Wenzhao Lian (Fremont, CA); Stefan Schaal (Mountain View, CA); Zheng Wu (Albany, CA)
Assignee: Intrinsic Innovation LLC
B25J9/1687B25J9/1661B25J9/1664G05B2219/39001
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,384,039
App. No.
17/845,698
Granted
Aug 12, 2025
Kind
B2
Abstract

A computer-implemented method comprising, receiving data representing a successful trajectory for an insertion task using a robot to insert a connector into a receptacle, performing a parameter optimization process for the robot to perform the insertion task. This parameter optimization includes defining an objective function that measures a similarity of a current trajectory generated with a current set of parameters to the successful trajectory and repeatedly modifying the current set of parameters and evaluating the modified set of parameters according to the objective function until generating a final set of parameters.

Claims (46)

1. A computer-implemented method comprising:

receiving data representing a successful trajectory for an insertion task using a robot to insert a connector into a receptacle;

performing a parameter optimization process for the robot to perform the insertion task including:

defining an objective function that measures a similarity of a current trajectory generated with a current set of parameters to the successful trajectory, and

repeatedly modifying the current set of parameters and evaluating the modified set of parameters according to the objective function until generating a final set of parameters;

adding the final set of parameters to a task library;

receiving a new insertion task;

selecting the final set of parameters based on the new insertion task; and

adapting the new insertion task using the final set of parameters.

2. The method of claim 1 , wherein performing the parameter optimization process comprises generating a motion primitive for performing the insertion task.

3. The method of claim 2 , further comprising using the one or more generated motion primitives to perform an overall task.

4. The method of claim 1 , wherein the data representing the successful trajectory for the insertion task was generated from a human demonstration.

5. The method of claim 4 , wherein performing the parameter optimization process modifies parameters of the robot to more closely match the human demonstration.

6. The method of claim 1 , wherein performing the parameter optimization process comprises adjusting values for pose, velocity, and force of the robot at each of a plurality of time steps of the current trajectory.

7. The method of claim 1 , wherein selecting the final set of parameters based on the new insertion task comprises computing a measure of similarity between the new insertion task and a plurality of insertion tasks represented in the task library.

8. The method of claim 7 , wherein the measure of similarity is based on a measure of similarity of receptacle shape.

9. The method of claim 1 , wherein adapting the new insertion task using the final set of parameters comprises constraining an initial search space for optimizing the new insertion task for the successful trajectory.

10. A system comprising:

one or more computers and one or more storage devices storing instructions that are operable, when executed by the one or more computers, to cause the one or more computers to perform operations comprising:

receiving data representing a successful trajectory for an insertion task using a robot to insert a connector into a receptacle;

performing a parameter optimization process for the robot to perform the insertion task including:

defining an objective function that measures a similarity of a current trajectory generated with a current set of parameters to the successful trajectory, and

repeatedly modifying the current set of parameters and evaluating the modified set of parameters according to the objective function until generating a final set of parameters;

adding the final set of parameters to a task library;

receiving a new insertion task;

selecting the final set of parameters based on the new insertion task; and

adapting the new insertion task using the final set of parameters.

11. The system of claim 10 , wherein performing the parameter optimization process comprises generating a motion primitive for performing the insertion task.

12. The system of claim 11 , wherein the operations further comprise using the one or more generated motion primitives to perform an overall task.

13. The system of claim 12 , wherein the data representing the successful trajectory for the insertion task was generated from a human demonstration.

14. The system of claim 13 , wherein performing the parameter optimization process modifies parameters of the robot to more closely match the human demonstration.

15. The system of claim 10 , wherein performing the parameter optimization process comprises adjusting values for pose, velocity, and force of the robot at each of a plurality of time steps of the current trajectory.

16. The system of claim 10 , wherein selecting the final set of parameters based on the new insertion task comprises computing a measure of similarity between the new insertion task and a plurality of insertion tasks represented in the task library.

17. The system of claim 16 , wherein the measure of similarity is based on a measure of similarity of receptacle shape.

18. The system of claim 10 , wherein adapting the new insertion task using the final set of parameters comprises constraining an initial search space for optimizing the new insertion task for the successful trajectory.

19. One or more non-transitory computer storage media encoded with computer program instructions that when executed by one or more computers cause the one or more computers to perform operations comprising:

receiving data representing a successful trajectory for an insertion task using a robot to insert a connector into a receptacle;

performing a parameter optimization process for the robot to perform the insertion task including:

defining an objective function that measures a similarity of a current trajectory generated with a current set of parameters to the successful trajectory, and

repeatedly modifying the current set of parameters and evaluating the modified set of parameters according to the objective function until generating a final set of parameters;

adding the final set of parameters to a task library;

receiving a new insertion task;

selecting the final set of parameters based on the new insertion task; and

adapting the new insertion task using the final set of parameters.

20. The computer storage media of claim 19 , wherein performing the parameter optimization process comprises adjusting values for pose, velocity, and force of the robot at each of a plurality of time steps of the current trajectory.

21. The computer storage media of claim 19 , wherein the data representing the successful trajectory for the insertion task was generated from a human demonstration.

Assignments (2)
CORRECTIVE ASSIGNMENT TO CORRECT THE THE RECEIVING PARTY'S NAME AND ADDRESS PREVIOUSLY RECORDED AT REEL: 060743 FRAME: 0779. ASSIGNOR(S) HEREBY CONFIRMS THE ASSIGNMENT. Recorded Aug 11, 2022
From: LIAN, WENZHAO; SCHAAL, STEFAN; WU, ZHENG
To: INTRINSIC INNOVATION LLC
Reel/Frame 061159/0963 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Aug 8, 2022
From: LIAN, WENZHAO; SCHAAL, STEFAN; WU, ZHENG
To: X DEVELOPMENT LLC
Reel/Frame 060743/0779 →
Continuity (2)
Provisional Application 63212481 · Jun 18, 2021
Related Publication 20220402140A1 · Dec 22, 2022
References Cited (58)
US 9840007B1 · Kuffner · 2017 [cited by examiner]
US 10668623B2 · Zhang · 2020 [cited by examiner]
US 20150336268A1 · Payton · 2015 [cited by examiner]
US 20220088780A1 · Abdul-hadi · 2022 [cited by examiner]
US 20220184808A1 · Zheng · 2022 [cited by examiner]
US 20220317659A1 · van Baar · 2022 [cited by examiner]
Alt et al., “Robot program parameter inference via differentiable shadow program inversion.” Submitted on Jul. 2022, arXiv:2103.14452v2, 7 pages. [cited by applicant]
Arkin et al., “An efficiently computable metric for comparing polygonal shapes” Cornell University Operations Research and Industrial Engineering, Jun. 1989, 22 pages. [cited by applicant]
Bohg et al., “Data-driven grasp synthesis—a survey.” Submitted on Apr. 2016, arXiv:1309.2660v2, 21 pages. [cited by applicant]
Chu et al., “Learning affordance segmentation for real-world robotic manipulation via synthetic images.” IEEE Robotics and Automation Letters 4.2, Jan. 2019, 8 pages. [cited by applicant]
Davchev et al., “Residual learning from demonstration: Adapting dynamic movement primitives for contact-rich insertion tasks” Submitted on Oct. 2020, arXiv:2008.07682v2, 11 pages. [cited by applicant]
Ester et al., “A density-based algorithm for discovering clusters in large spatial databases with noise.” kdd. vol. 96. No. 34., Aug. 1996, 226-231. [cited by applicant]
Fang et al., “Learning task-oriented grasping for tool manipulation from simulated self-supervision.” Submitted on Jun. 2018, arXiv:1806.09266v1, 11 pages. [cited by applicant]
Fischler et al., “Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography.” Communications of the ACM 24.6, Jun. 1981, 381-395. [cited by applicant]
Florence et al., “Dense object nets: Learning dense visual object descriptors by and for robotic manipulation.” Submitted on Jun. 2018, arXiv:1806.08756v2, 12 pages. [cited by applicant]
Graham et al., “Submanifold sparse convolutional networks.” Submitted on Jun. 2017, arXiv:1706.01307v1, 10 pages. [cited by applicant]
Gu et al., “Deep reinforcement learning for robotic manipulation with asynchronous off-policy updates.” Submitted on Nov. 2016, arXiv:1610.00633v2, 9 pages. [cited by applicant]
Haarnoja et al., “Composable deep reinforcement learning for robotic manipulation” Submitted on Mar. 2018, arXiv:1803.06773v1, 10 pages. [cited by applicant]
Jiang et al., “Pointgroup: Dual-set point grouping for 3d instance segmentation.” Proceedings of the IEEE/CVF conference on computer vision and Pattern recognition, 2020, 4867-4876. [cited by applicant]
Johannsmeier et al., “A framework for robot manipulation: Skill formalism, meta learning and adaptive control.” Submitted on May 2018, arXiv:1805.08576v1, 8 pages. [cited by applicant]
Kalashnikov et al., “Scalable deep reinforcement learning for vision-based robotic manipulation.” Conference on Robot Learning. PMLR, Oct. 2018, 23 pages. [cited by applicant]
Lee et al., “Making sense of vision and touch: Self-supervised learning of multimodal representations for contact-rich tasks.” Submitted on Mar. 2019, arXiv:1810.10191v2, 8 pages. [cited by applicant]
Li et al., “Learning hierarchical control for robust in-hand manipulation.” Submitted on Oct. 2019, arXiv:1910.10985v1, 8 pages. [cited by applicant]
Lian et al., “Benchmarking off-the-shelf solutions to robotic assembly tasks” Submitted on Mar. 2021, arXiv:2103.05140v1, 7 pages. [cited by applicant]
Liang et al., “PointNetGPD: Detecting grasp configurations from point sets” Submitted on Feb. 2019, arXiv:1809.06267v4, 7 pages. [cited by applicant]
Luo et al., “Robust multi-modal policies for industrial assembly via reinforcement learning and demonstrations: A large-scale study.” Submitted on Jul. 2021, arXiv:2103.11512v4, 10 pages. [cited by applicant]
Mahler et al., “Dex-net 2.0: Deep learning to plan robust grasps with synthetic point clouds and analytic grasp metrics.” Submitted on Aug. 2017, arXiv:1703.09312v3, 12 pages. [cited by applicant]
Mahler et al., “Learning ambidextrous robot grasping policies.” Science Robotics 4.26, 2019, 12 pages. [cited by applicant]
Manuelli et al., “kPAM: Keypoint affordances for category-level robotic manipulation.” Submitted on Oct. 2019, arXiv:1903.06684v2, 26 pages. [cited by applicant]
Miller et al., “Graspit! a versatile simulator for robotic grasping.” IEEE Robotics & Automation Magazine 11.4, Dec. 2004, 110-122. [cited by applicant]
Monica et al., “Point cloud projective analysis for part-based grasp planning.” IEEE Robotics and Automation Letters 5.3, Jun. 2020, 8 pages. [cited by applicant]
Montesano et al., “Leaming object affordances: from sensory-motor coordination to imitation” IEEE Transactions on Robotics, 24(1), Feb. 2008, 15-26. [cited by applicant]
Morgan et al., “Vision-driven compliant manipulation for reliable, high-precision assembly tasks.” Submitted on Jun. 2021, arXiv:2106.14070v1, 13 pages. [cited by applicant]
Murali et al., “6-dof grasping for target-driven object manipulation in clutter.” Submitted on May 2020, arXiv:1912.03628v2, 7 pages. [cited by applicant]
Ni et al., “Pointnet++ grasping: Learning an end-to-end spatial grasp generation algorithm from sparse point clouds.” 2020 IEEE International Conference on Robotics and Automation (ICRA). IEEE, May 2020, 7 pages. [cited by applicant]
Patten et al., “DGCM-net: dense geometrical correspondence matching network for incremental experience-based robotic grasping” Submitted on Jan. 2020, arXiv:2001.05279v1, 19 pages. [cited by applicant]
Peng et al., “Sim-to-real transfer of robotic control with dynamics randomization.” Submitted on Mar. 2018, arXiv:1710.06537v3, 8 pages. [cited by applicant]
Perrone et al., “Learning search spaces for bayesian optimization: Another view of hyperparameter transfer learning” Submitted on Sep. 2019, arXiv:1909.12552v1, 17 pages. [cited by applicant]
PyBullet.org [online], “Pybullet a python module for physics simulation for games” Apr. 2021, retrieved on Aug. 7, 2023, retrieved from URL <https://pybullet.org/wordpress/index.php/2021/04/15/pybullet-in-a-colab/>, 2 p… [cited by applicant]
Qi et al., “Pointnet: Deep learning on point sets for 3d classification and segmentation.” Proceedings of the IEEE conference on computer vision and pattern recognition, 2017, 652-660. [cited by applicant]
Qin et al., “Keto: Learning keypoint representations for tool manipulation.” Submitted on Oct. 2019, arXiv:1910.11977v2, 8 pages. [cited by applicant]
Rakelly et al., “Efficient off-policy meta reinforcement learning via probabilistic context variables” In International conference on machine learning, pp. 5331-5340. PMLR, May 2019, 10 pages. [cited by applicant]
Schoettler et al., “Deep reinforcement learning for industrial insertion tasks with visual inputs and natural rewards” Submitted on Aug. 2019, arXiv:1906.05841v2, 10 pages. [cited by applicant]
Schoettler et al., “Meta-reinforcement learning for robotic industrial insertion tasks.” Submitted on May 2020, arXiv:2004.14404v2, 9 pages. [cited by applicant]
Suárez-Ruiz et al., “A framework for fine robotic assembly.” Submitted on Sep. 2015, arXiv:1509.04806v1, 8 pages. [cited by applicant]
Ten Pas et al., “Grasp pose detection in point clouds.” Submitted on Jun. 2017, arXiv:1706.09911v1, 17 pages. [cited by applicant]
Tobin et al., “Domain randomization for transferring deep neural networks from simulation to the real world.” Submitted on Mar. 2017, arXiv:1703.06907v1, 8 pages. [cited by applicant]
Tremblay et al., “Deep object pose estimation for semantic robotic grasping of household objects.” Submitted on Sep. 2018, arXiv:1809.10790v1, 11 pages. [cited by applicant]
Umeyama, “Least-squares estimation of transformation parameters between two point patterns.” IEEE Transactions on Pattern Analysis & Machine Intelligence 13.04, Apr. 1991, 376-380. [cited by applicant]
Vecerik et al., “Leveraging demonstrations for deep reinforcement learning on robotics problems with sparse rewards” Submitted on Oct. 2018, arXiv:1707.08817v2, 10 pages. [cited by applicant]
Voigt et al., “Multi-Level Structure vs. End-to-End-Learning in High-Performance Tactile Robotic Manipulation.” CoRL. 2020, 11 pages. [cited by applicant]
Vuong et al., “Learning sequences of manipulation primitives for robotic assembly” Submitted on Mar. 2021, arXiv:2011.00778v2, 7 pages. [cited by applicant]
Wang et al., “Normalized object coordinate space for category-level 6d object pose and size estimation.” Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2019, 2642-2651. [cited by applicant]
Wen et al., “SE(3)-tracknet Data-driven 6d pose tracking by calibrating image residuals in synthetic domains” Submitted on Jul. 2020, arXiv:2007.13866v1, 7 pages. [cited by applicant]
Wu et al., “Learning dense rewards for contact rich manipulation tasks” Submitted on Nov. 2020, arXiv:2011.08458v1, 8 pages. [cited by applicant]
Yang et al., “Learning multi-object dense descriptor for autonomous goal-conditioned grasping.” IEEE Robotics and Automation Letters 6.2, Feb. 2021, 4109-4116. [cited by applicant]
Zeng et al., “Multi-view self-supervised deep learning for 6d pose estimation in the amazon picking challenge.” Submitted on May 2017, arXiv:1609.09475v3, 8 pages. [cited by applicant]
Zhao et al., “Towards robotic assembly by predicting robust, precise and task-oriented grasps.” Conference on Robot Learning. PMLR, Oct. 2021, 11 pages. [cited by applicant]