IP Library › Granted Patent US 11,794,346
Granted Patent B2
US 11,794,346 · App. 17/152,723 · Granted Oct 24, 2023

Robotic system with error detection and dynamic packing mechanism

Inventors: Rosen Nikolaev Diankov (Tokyo, JP); Denys Kanunikov (Tokyo, JP)
Assignee: MUJIN, Inc.
B25J9/1687B25J9/1661B25J9/1664B25J9/1676B25J9/1697
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,794,346
App. No.
17/152,723
Granted
Oct 24, 2023
Kind
B2
Abstract

A method for operating a robotic system includes determining a discretized object model based on source sensor data; comparing the discretized object model to a packing plan or to master data; determining a discretized platform model based on destination sensor data; determining height measures based on the destination sensor data; comparing the discretized platform model and/or the height measures to an expected platform model and/or expected height measures; and determining one or more errors by (i) determining at least one source matching error by identifying one or more disparities between (a) the discretized object model and (b) the packing plan or the master data or (ii) determining at least one destination matching error by identifying one or more disparities between (a) the discretized platform model or the height measures and (b) the expected platform model or the expected height measures, respectively.

Claims (77)

1. A method for operating a robotic system, the method comprising:

before placing a target object at a placement location for the target object specified in a packing plan, determining:

(i) a source matching error based on identifying a disparity between

(a) an object model that represents the target object and that is discretized according to a unit cell, and (b) the packing plan or master data, or

(ii) a destination matching error based on identifying a disparity between (a) a platform model that represents a placement area and that is discretized according to a further unit cell, or a height measure representing a maximum height within a portion of the placement area and (b) an expected platform model or an expected height measure, respectively; and

in response, at least in part, to determining the source matching error or the destination matching error, adjusting the placement location for the target object or for another object.

2. The method of claim 1 , wherein:

determining includes identifying a disparity between the object model and the master data; and

further comprising:

determining a master data error representing the master data is wrong.

3. The method of claim 1 , wherein:

the packing plan specifies a sequence in which objects including the target object are to arrive at a start location;

determining the source matching error includes identifying a disparity between the object model and a sequence-specified object model; and

further comprising:

determining an arrival sequence error representing that the target object arrived at the start location out of the sequence specified by the packing plan.

4. The method of claim 1 , wherein:

determining the destination matching error includes identifying a disparity between the platform model and the expected platform model; and

further comprising:

determining a placement accessibility error, an unexpected placement error, and/or a placement area error representing that the placement area differs from an expected placement area.

5. The method of claim 4 , further comprising:

identifying a placement accessibility error representing a wall of a container, cage, or car track at a task location associated with the placement area is not fully opened.

6. The method of claim 1 , wherein determining includes:

identifying the destination matching error based on identifying a disparity between (a) the platform model or the height measure and (b) the expected platform model or the expected height measure, respectively; and

further comprising:

identifying an unexpected placement error representing an object of previously placed objects at a task location associated with the placement area shifted, fell, and/or was displaced.

7. The method of claim 1 , wherein determining includes:

identifying the destination matching error based on identifying a disparity between (a) the platform model or the height measure and (b) the expected platform model or the expected height measure, respectively; and

further comprising:

identifying a placement area error representing an object of previously placed objects at a task location associated with the placement area was misplaced.

8. The method of claim 1 , wherein:

the expected platform model and/or the expected height measure are specified by the packing plan;

determining includes identifying the destination matching error based on identifying a disparity between (a) the platform model or the height measure and (b) the expected platform model or the expected height measure, respectively; and

further comprising:

identifying a placement area error representing an object not included in the packing plan is positioned at a task location associated with the placement area.

9. The method of claim 1 , further comprising:

in response to determining the destination matching error, sequentially evaluating for:

a placement accessibility error representing a wall of a container, cage, or car track at a task location associated with the placement area is not fully opened;

an unexpected placement error representing that one or more of previously placed objects at the task location shifted, fell, and/or was displaced;

a placement area error representing that one or more of the previously placed objects at the task location was misplaced; and

a placement area error representing (a) one or more of the previously placed objects, (b) the container, cage, or car track, and/or (c) an object not included in the packing plan but present at the task location is/are contributing to a disparity between the height measure and the expected height measure.

10. The method of claim 1 , wherein determining the destination matching error includes:

identifying a disparity between (a) the platform model or the height measure and (b) the expected platform model or the expected height measure, respectively; and

further comprising:

determining a placement area error representing an object of previously placed objects at a task location associated with the placement area is missing at the task location.

11. The method of claim 10 , further comprising:

in response to determining the destination matching error, determining a transport or manipulation error representing the object of the previously placed objects was dropped while transporting the object to the task location.

12. The method of claim 1 , wherein determining the destination matching error includes:

identifying a disparity between (a) the platform model or the height measure and (b) the expected platform model or the expected height measure, respectively; and

further comprising:

determining a collision error representing a collision has occurred between (i) a robotic unit of the robotic system, (ii) the target object, (iii) previously placed objects at a task location associated with the placement area, (iv) a container at the task location, and/or (v) an object not included in the packing plan but present at the task location.

13. The method of claim 1 , further comprising:

in response to determining the source matching error or the destination matching error

deriving a candidate position based, at least in part, on overlapping the discretized object model over the platform model at a corresponding location;

validating the candidate position according to one or more placement constraints associated with the height measure;

calculating a placement score for the candidate position, wherein the placement score is calculated according to one or more placement preferences; and

dynamically deriving the placement location based on selecting the candidate position according to the placement score, wherein the placement location is for placing the target object at a task location associated with the placement area.

14. The method of claim 13 , wherein validating the candidate position includes deriving an approach path for placing the target object at the candidate position.

15. The method of claim 13 , wherein:

adjusting the placement location for the target object includes generating information for placing the target object at the placement location over the placement area; and

the method further comprises communicating the generated information for placing the target object at the placement location over the placement area.

16. The method of claim 15 , further comprising verifying placement accuracy of the target object at the placement location.

17. The method of claim 13 , further comprising updating or replacing the packing plan based, at least in part, on the determined status error or the placement location for placing the target object.

18. A robotic system comprising: at least one processor; and

at least one memory device connected to the at least one processor and having stored thereon instructions executable by the processor to:

before placing a target object at a placement location for the target object specified in a packing plan, determine:

(i) a source matching error by identifying a disparity between

(a) an object model that represents the target object and that is discretized according to a unit cell, and (b) the packing plan or master data, or

(ii) a destination matching error by identifying a disparity between (a) a platform model that represents a placement area and that is discretized according to a further unit cell, or a height measure representing a maximum height within a portion of the placement area and (b) an expected platform model or an expected height measure, respectively; and

adjust the placement location for the target object or for another object based, at least in part, on the determined source matching error or the determined destination matching error.

19. A tangible, non-transient computer-readable medium having processor instructions stored thereon that, when executed by a robotic system via one or more processors thereof, cause the robotic system to perform a method, the instructions comprising:

instructions to, before placing a target object at a placement location for the target object specified in a packing plan, determine:

(i) a source matching error by identifying a disparity between (a) an object model representing the target object and (b) the packing plan or master data, or

(ii) a destination matching error by identifying a disparity between

(a) a platform model representing a placement area or a height measure representing a maximum height within a portion of the placement area and (b) an expected platform model or an expected height measure, respectively; and

instructions to adjust the placement location for the target object or for another object based, at least in part, on the source matching error or the destination matching error,

wherein the object model is discretized according to a unit dimension or a unit area, and

wherein the platform model is discretized according to a further unit dimension or a further unit area.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Apr 28, 2021
From: DIANKOV, ROSEN NIKOLAEV; KANUNIKOV, DENYS
To: MUJIN, INC.
Reel/Frame 056075/0010 →
Continuity (3)
Continuation 16802451 · Feb 26, 2020
Continuation 16428809 · May 31, 2019
Related Publication 20210237274A1 · Aug 5, 2021
Cited By (7)
US 12,214,975 US 12,258,227 US 12,337,481 US 12,390,928 US 12,485,544 US 12,486,125 US 12,715,716