IP Library › Granted Patent US 12,194,638
Granted Patent B2
US 12,194,638 · App. 18/465,101 · Granted Jan 14, 2025

Robotic system control method and controller

Inventors: Rosen Nikolaev Diankov (Tokyo, JP); Yoshiki Kanemoto (Tokyo, JP); Denys Kanunikov (Tokyo, JP)
Assignee: Mujin, Inc.
B25J9/1664B25J9/1661B25J9/1697G01S17/89G06T7/73
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,194,638
App. No.
18/465,101
Filed
Sep 11, 2023
Granted
Jan 14, 2025
Kind
B2
Examiner
JEN, MINGJEN
Art Unit
3657
USPC
700/245
Abstract

The present disclosure provides a control method of a robotic system. The control method includes: deriving an approach location at which the end effector grips an operation object; deriving a scan location for scanning an identifier of the operation object; and based on the approach location and the scan location, creating or deriving a control sequence to instruct the robot to execute the control sequence. The control sequence includes (1) gripping the operation object from a start location; (2) scanning an identifier of the operation object with a scanner located between the start location and a task location; (3) temporarily releasing the operation object from the end effector and regripping the operation object by the end effector to be shifted, at a shift location, when a predetermined condition is satisfied; and (4) moving the operation object to the task location.

Claims (35)

1. A non-transitory computer-readable medium storing processor instructions, the processor instructions, when executed by one or more processors, causing a robotic system to:

compute one or more control sequences that, when executed by a robot, transfers an operation object from a starting location to a task location, wherein the computed one or more control sequences cause the robot to:

transfer the operation object from the starting location based on gripping the operation object at a first grip location of the operation object;

regrip the operation object at a second grip location of the operation object at a location between the starting location and the task location; and

transfer the operation object from the location to the task location based on gripping the operation object at the second grip location of the operation object.

2. The non-transitory computer-readable medium of claim 1 , wherein the system is further caused to, prior to regripping the operation object, perform a scan of the operation object at a location between the starting location and the task location.

3. The non-transitory computer-readable medium of claim 2 , wherein the first grip location is computed to facilitate one or more scanning poses of the operation object while the operation object is transferred between the starting location and the task location.

4. The non-transitory computer-readable medium of claim 2 , wherein the regrip is based on the scan of the operation object at the location between the starting location and the task location.

5. The non-transitory computer-readable medium of claim 1 , wherein the system is further caused to, prior to regripping the operation object and subsequent to initially displacing the operation object from the starting location, obtain a measurement descriptive of the operation object.

6. The non-transitory computer-readable medium of claim 5 , wherein the measurement includes a dimension of the operation object.

7. The non-transitory computer-readable medium of claim 5 , wherein the regrip is based on the measurement descriptive of the operation object.

8. The non-transitory computer-readable medium of claim 1 , wherein the second grip location is derived subsequent to displacing the operation object from the starting location.

9. The non-transitory computer-readable medium of claim 8 , wherein the second grip location is derived dynamically based on information obtained while the operation object is in motion between the starting location and the task location.

10. The non-transitory computer-readable medium of claim 9 , wherein the information includes a physical measurement of the operation object.

11. The non-transitory computer-readable medium of claim 1 , wherein the robotic system is further caused to detect the operation object with a confidence measure, wherein the confidence measure is associated with an initial pose of the operation object at the starting location.

12. The non-transitory computer-readable medium of claim 11 , wherein the robotic system is further caused to determine, based on the confidence measure, one or more scanning poses for the operation object.

13. The non-transitory computer-readable medium of claim 12 , wherein the first grip location is identified based on the initial pose and the one or more scanning poses.

14. The non-transitory computer-readable medium of claim 11 , wherein the robotic system is further caused to determine a placement pose based on the confidence measure, wherein the placement pose is the pose the operation object has at the task location.

15. The non-transitory computer-readable medium of claim 14 , wherein the second grip location is identified based on the placement pose so that the second grip location is compatible with the placement pose.

16. The non-transitory computer-readable medium of claim 14 , wherein the robotic system is further caused to:

determine whether the first grip location is compatible with the placement pose; and

in response to a determination that the first grip location is not compatible with the placement pose, identify the second grip location.

17. A method for controlling a robotic system, the method comprising:

computing one or more control sequences that, when executed by a robot, transfers an operation object from a starting location to a task location, wherein the computed one or more control sequences cause the robot to:

transfer the operation object from the starting location based on gripping the operation object at a first grip location of the operation object;

regrip the operation object at a second grip location of the operation object at a location between the starting location and the task location; and

transfer the operation object from the location to the task location based on gripping the operation object at the second grip location of the operation object.

18. The method of claim 17 , further comprising detecting the operation object with a confidence measure, wherein the confidence measure is associated with an initial pose of the operation object at the starting location.

19. A system, comprising:

a communication circuit configured to communicate with at least a robot; and

one or more processors coupled to the communication circuit, wherein the one or more processors are configured to execute a control method, the control method including computing one or more control sequences that, when executed by the robot, transfers an operation object from a starting location to a task location, wherein the computed one or more control sequences cause the robot to:

transfer the operation object from the starting location based on gripping the operation object at a first grip location of the operation object;

regrip the operation object at a second grip location of the operation object at a location between the starting location and the task location; and

transfer the operation object from the location to the task location based on gripping the operation object at the second grip location of the operation object.

20. The system of claim 19 , wherein the control method further comprises detecting the operation object with a confidence measure, wherein the confidence measure is associated with an initial pose of the operation object at the starting location.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Dec 3, 2024
From: DIANKOV, ROSEN NIKOLAEV; KANEMOTO, YOSHIKI; KANUNIKOV, DENYS
To: MUJIN, INC.
Reel/Frame 069473/0385 →
Priority Claims (2)
JP 2019-118678 · Jun 26, 2019 · national
JP 2019-213029 · Nov 26, 2019 · national
Continuity (7)
Continuation 17806432 · Jun 10, 2022
Continuation 17110205 · Dec 2, 2020
Continuation 16752582 · Jan 24, 2020
Continuation In Part 16743313 · Jan 15, 2020
Continuation 16546209 · Aug 20, 2019
Continuation 16258120 · Jan 25, 2019
Related Publication 20240075619A1 · Mar 7, 2024
References Cited (87)
US 4831549A · Red et al. · 1989 [cited by applicant]
US 5789890A · Genov et al. · 1998 [cited by applicant]
US 10335963B2 · Nakazato et al. · 2019 [cited by applicant]
US 10350755B2 · Wagner et al. · 2019 [cited by applicant]
US 10456915B1 · Diankov · 2019 [cited by applicant]
US 10569416B1 · Diankov · 2020 [cited by applicant]
US 10569417B1 · Diankov · 2020 [cited by applicant]
US 10576631B1 · Diankov · 2020 [cited by examiner]
US 10596701B1 · Diankov · 2020 [cited by examiner]
US 11413753B2 · Diankov et al. · 2022 [cited by applicant]
US 11638993B2 · Diankov · 2023 [cited by applicant]
US 11772267B2 · Diankov et al. · 2023 [cited by applicant]
US 20040093119A1 · Gunnarsson et al. · 2004 [cited by applicant]
US 20040167667A1 · Goncalves et al. · 2004 [cited by applicant]
US 20050182518A1 · Karlsson · 2005 [cited by applicant]
US 20060181236A1 · Brogardh · 2006 [cited by applicant]
US 20110153076A1 · Noro · 2011 [cited by applicant]
US 20110235054A1 · Koike et al. · 2011 [cited by applicant]
US 20120143372A1 · Roh · 2012 [cited by applicant]
US 20130110280A1 · Folk · 2013 [cited by applicant]
US 20130343640A1 · Buehler · 2013 [cited by applicant]
US 20140347473A1 · Wolff et al. · 2014 [cited by applicant]
US 20150352717A1 · Mundt et al. · 2015 [cited by applicant]
US 20160297068A1 · Thibodeau et al. · 2016 [cited by applicant]
US 20160379370A1 · Nakazato et al. · 2016 [cited by applicant]
US 20170165836A1 · Kanazawa · 2017 [cited by applicant]
US 20170326739A1 · Nakazato et al. · 2017 [cited by applicant]
US 20170364076A1 · Keshmiri et al. · 2017 [cited by applicant]
US 20200238517A1 · Diankov · 2020 [cited by applicant]
US 20200238519A1 · Diankov et al. · 2020 [cited by applicant]
US 20210154839A1 · Diankov et al. · 2021 [cited by applicant]
CN 1747816A · 2006 [cited by applicant]
CN 205466149U · 2016 [cited by applicant]
CN 107263485A · 2017 [cited by applicant]
DE 10249786A1 · 2004 [cited by applicant]
DE 102014107143A1 · 2014 [cited by applicant]
DE 102017005882A1 · 2018 [cited by applicant]
DE 102017123877A1 · 2019 [cited by applicant]
JP H05217014A · 1993 [cited by applicant]
JP 05285803A · 1993 [cited by applicant]
JP 2013022705A · 2013 [cited by applicant]
JP 2013078825A · 2013 [cited by applicant]
JP 2013086914A · 2013 [cited by applicant]
JP 2013198943A · 2013 [cited by applicant]
JP 2017058130A · 2017 [cited by applicant]
JP 2017185578A · 2017 [cited by applicant]
JP 2018167950A · 2018 [cited by applicant]
JP 2018202550A · 2018 [cited by applicant]
Supplemental Notice of Allowability mailed Jan. 28, 2020 for U.S. Appl. No. 16/546,226, filed Aug. 20, 2019, Inventor: Rosen Nikolaev Diankov, 2 pages. [cited by applicant]
Non-Final Office Action mailed Apr. 4, 2019 for U.S. Appl. No. 16/258,120, filed Jan. 25, 2019, Inventor: Rosen Nikolaev Diankov, 18 pages. [cited by applicant]
U.S. Patent Office Issue Notification mailed Mar. 4, 2020 for U.S. Appl. No. 16/546,226, filed Aug. 20, 2019, Inventor: Rosen Nikolaev Diankov. [cited by applicant]
Notice of Allowance mailed Nov. 7, 2019 for U.S. Appl. No. 16/546,226, filed Aug. 20, 2019, Inventor: Rosen Nikolaev Diankov, 16 pages. [cited by applicant]
Interview Summary mailed Apr. 25, 2019 for U.S. Appl. No. 16/258,120, filed Jan. 25, 2019, Inventor: Rosen Nikolaev Diankov, 3 pages. [cited by applicant]
U.S. Patent Office Examiner-Initiated Interview Summary mailed Sep. 23, 2019 for U.S. Appl. No. 16/546,248, filed Aug. 20, 2019, 2 pages. [cited by applicant]
Interview Summary mailed Aug. 21, 2019 for U.S. Appl. No. 16/258,120, filed Jan. 25, 2019, Inventor: Rosen Nikolaev Diankov, 3 pages. [cited by applicant]
Notice of Allowance mailed Jul. 29, 2019 for U.S. Appl. No. 16/258,120, filed Jan. 25, 2019, Inventor: Rosen Nikolaev Diankov, 6 pages. [cited by applicant]
U.S. Patent Office Issue Notification mailed Oct. 9, 2019 for U.S. Appl. No. 16/258,120, filed Jan. 25, 2019, Inventor: Rosen Nikolaev Diankov. [cited by applicant]
Notice of Allowance mailed Nov. 6, 2019 for U.S. Appl. No. 16/546,209, filed Aug. 20, 2019, Inventor: Rosen Nikolaev Diankov, 21 pages. [cited by applicant]
U.S. Patent Office Issue Notification mailed Feb. 5, 2020 for U.S. Appl. No. 16/546,209, filed Aug. 20, 2019, Inventor: Rosen Nikolaev Diankov. [cited by applicant]
Notice of Allowance mailed Oct. 17, 2019 for U.S. Appl. No. 16/546,236, filed Aug. 20, 2019, Inventor: Rosen Nikolaev Diankov, 17 pages. [cited by applicant]
U.S. Patent Office Issue Notification mailed Feb. 5, 2020 for U.S. Appl. No. 16/546,236, filed Aug. 20, 2019, Inventor: Rosen Nikolaev Diankov. [cited by applicant]
Notice of Allowance mailed Oct. 25, 2019 for U.S. Appl. No. 16/546,248, filed Aug. 20, 2019, Inventor: Rosen Nikolaev Diankov, 16 pages. [cited by applicant]
U.S. Patent Office Issue Notification mailed Feb. 12, 2020 for U.S. Appl. No. 16/546,248, filed Aug. 20, 2019, Inventor: Rosen Nikolaev Diankov. [cited by applicant]
Non-Final Office Action mailed Aug. 6, 2020 for U.S. Appl. No. 16/743,313, filed Jan. 15, 2020, Inventor: Rosen Nikolaev Diankov, 17 pages. [cited by applicant]
Office Action for German patent application No. 102019130046.0, mailed May 27, 2020, translation and original, 11 pages. [cited by applicant]
Office Action received for Chinese patent application No. 201910337803.9, issued Dec. 26, 2019, 5 pages. [cited by applicant]
Office Action for Japanese patent application No. 2019-080213, mailed Jan. 17, 2020, translation and original, 6 pages. [cited by applicant]
Decision to Grant for Japanese patent application No. 2019-080213, mailed Mar. 12, 2020, 3 pages. [cited by applicant]
German Patent and Trademark Office, Decision to Grant received for German patent application No. 102019130046.0, mailed Oct. 6, 2020, translation and original, 6 pages. [cited by applicant]
Notice of Allowance mailed Oct. 23, 2020 for U.S. Appl. No. 16/743,313, filed Jan. 15, 2020, Inventor: Rosen Nikolaev Diankov, 11 pages. [cited by applicant]
Notice to Grant for Chinese patent application No. 201910337803.9, issued Jun. 4, 2020, 4 pages. [cited by applicant]
Notice of Allowance mailed Aug. 19, 2020 for U.S. Appl. No. 16/752,582, filed Jan. 24, 2020, First Inventor: Rosen Nikolaev Diankov, 26 pages. [cited by applicant]
Notice of Allowance mailed Mar. 31, 2022 for U.S. Appl. No. 17/110,205, filed Dec. 2, 2020, First Inventor: Rosen Nikolaev Diankov, 11 pages. [cited by applicant]
Notice of Allowance mailed Apr. 15, 2022 for U.S. Appl. No. 17/110,205, filed Dec. 2, 2020, First Inventor: Rosen Nikolaev Diankov, 8 pages. [cited by applicant]
Non-Final Office Action mailed Aug. 23, 2022 for U.S. Appl. No. 17/157,975, filed Jan. 25, 2021, Inventor: Rosen Diankov, 23 pages. [cited by applicant]
Notice of Allowance mailed Dec. 20, 2022 for U.S. Appl. No. 17/157,975, filed Jan. 25, 2021, Inventor: Rosen Diankov, 10 pages. [cited by applicant]
Office Action issued for German patent application No. DE 102019009198.1, mailed Apr. 1, 2022, 10 pages. [cited by applicant]
Notice of Allowance issued for U.S. Appl. No. 17/110,205, mailed Mar. 31, 2022, 22 pages. [cited by applicant]
CNIPA Notice to Grant mailed Jul. 12, 2021 for Chinese patent application No. 202010834548.1. [cited by applicant]
CNIPA Office Action mailed May 20, 2021 for Chinese patent application No. 202010834548.1, Applicant: MUJIN, Inc., 7 pages. [cited by applicant]
Japanese Patent Office, Office Action mailed Feb. 15, 2023 for patent application No. 2020-068723, Applicant: MUJIN, Inc., 3 pages. [cited by applicant]
Japanese Patent Office, Notice to Grant mailed Mar. 13, 2023 for patent application No. 2020- 068723, Applicant: MUJIN, Inc., 3 pages. [cited by applicant]
Notice of Allowance received for U.S. Appl. No. 17/806,432, mailed May 25, 2023, first named Inventor: Rosen Nikolaev Diankov, 11 pages. [cited by applicant]
Non-Final Office Action received for U.S. Appl. No. 18/306,949, filed Apr. 25, 2023, mailed Dec. 29, 2023, first named Inventor: Rosen Diankov, 11 pages. [cited by applicant]
JPO Notice of Reasons for Refusal mailed Jul. 12, 2024 for Japanese patent application No. 2023-077708, 7 pages. [cited by applicant]
German Patent Office Decision of Grant mailed Jun. 28, 2024 for German patent application No. 102019009198.1, 5 pages. [cited by applicant]
Notice of Allowance mailed Jul. 24, 2024 for U.S. Appl. No. 18/306,949, filed Apr. 25, 2023, 6 pages. [cited by applicant]