Granted Patent B2

US 11,319,166 · App. 15/931,530 · Granted May 3, 2022

Inventors: Rosen Nikolaev Diankov (Tokyo, JP); Denys Kanunikov (Tokyo, JP)

Assignee: MUJIN, Inc.

B65G47/905B65B57/00

Abstract

Claims (78)

1. A method for operating a robotic system, the method comprising:

accessing discretized object models that represent available objects designated for placement at a task location;

accessing a discretized model representing the task location;

calculating a processing order for subgroupings that represent groupings of the available objects;

based at least in part on the subgroupings and the processing order, generating two-dimensional (2D) placement plans for placing the available objects along a horizontal plane;

based at least in part on the 2D placement plans, generating a three-dimensional (3D) stacking plan for stacking the available objects, wherein the 2D placement plans correspond to layers within the 3D stacking plan; and

implementing the 3D stacking plan for placing the available objects at the task location.

2. The method of claim 1 , wherein:

the method further comprises calculating, based at least in part on the 3D stacking plan, a packing sequence for identifying a placing order of the available objects; and

implementing the 3D stacking plan includes implementing the 3D stacking plan according to the packing sequence.

3. The method of claim 2 , wherein:

calculating the packing sequence includes:

adjusting the 2D placement plans by reassigning a placement location, representing placement of one or more of the available objects, from a lower-layer 2D placement plan to a higher-layer 2D placement plan, and calculating the packing sequence for each placement plan; and

implementing the 3D stacking plan further includes implementing the 3D stacking plan for placing the available objects layer-by-layer.

4. The method of claim 2 , wherein calculating the packing sequence includes analyzing the 3D stacking plan according to height measurements for the available objects, placement locations representing planned placement of the available objects relative to a unit location, or a combination thereof.

5. The method of claim 1 , further comprising determining the subgroupings of the available objects according to specified priorities, fragility ratings, object weight ranges, heights, object types, or a combination thereof.

6. The method of claim 1 , wherein calculating the processing order includes prioritizing the subgroupings according to a number of objects therein, a combined horizontal area, weight ranges, fragility ratings, or a combination thereof.

7. The method of claim 6 , wherein calculating the processing order includes calculating the processing order for an initial set representing the available objects before implementing the 3D stacking plan.

8. The method of claim 6 , wherein calculating the processing order includes calculating the processing order for a remaining set representing the available objects yet to be placed after or while implementing the 3D stacking plan.

9. The method of claim 1 , wherein:

generating the 2D placement plans includes:

identifying object types representing a grouping condition for the available objects; and

iteratively deriving placement locations representing planned placements for each of the available objects based at least in part on:

determining an initial placement location for one of the object types, and

deriving candidate placement locations representing planned placements for a next object based on one or more placement conditions; and

the 2D placement plans each represent a unique placement combination of the initial placement location and the candidate placement locations of one or more of the available objects.

10. The method of claim 9 , wherein the one or more placement conditions include object dimensions, collision probabilities, fragility ratings, object weight ranges, separation requirements, object quantity conditions, or a combination thereof.

11. The method of claim 9 , wherein iteratively deriving placement locations further includes:

tracking the 2D placement plans across iterations;

calculating placement scores for the 2D placement plans according to the one or more placement conditions; and

sequencing the 2D placement plans according to the placement scores.

12. The method of claim 11 , wherein tracking the 2D placement plans includes:

generating and updating a search tree including the 2D placement plans; and

stopping the iterations based on determining an empty source status or an unchanged score status.

13. The method of claim 9 , wherein the candidate placement locations are within a threshold distance from the initial placement location along a horizontal direction.

14. The method of claim 1 , wherein generating the 3D stacking plan includes generating the 3D stacking plan according to stacking rules regarding a measure of: an overlap between a top object and a support object, an overhang of the top object over the support object, a center-of-mass (CoM) location of the top object relative to one or more support objects, a separation distance between two or more support objects, a difference in heights of two or more support objects, or a combination thereof.

15. The method of claim 1 , wherein:

generating the 2D placement plans and generating the 3D stacking plan include deriving candidate stacking scenarios;

each candidate stacking scenario is recursively or iteratively derived based at least in part on:

generating a lower 2D plan for forming a lower layer,

identifying one or more planar sections of the first candidate 2D plan, wherein each planar section represents adjacent and/or continuous horizontal top surfaces,

generating an upper 2D plan based at least in part on testing one or more placement locations of remaining subgroupings relative to the one or more planar sections,

calculating a placement score for the candidate stacking scenario based at least in part on a number representing placed objects, and

stopping the iterations or the recursions based at least in part on determining an empty source status, a full layer status, or an unchanged score status; and

generating the 3D stacking plan includes generating the 3D stacking plan based on selecting one of the candidate stacking scenarios according to the placement score.

16. A robotic system comprising:

at least one processor; and

at least one memory device communicatively connected to the at least one processor and having stored thereon instructions that, when executed by the at least one processor, cause the robotic system to:

access discretized object models that represent available objects designated for placement at a task location;

access a discretized model representing the task location;

calculate a processing order for subgroupings of the available objects;

based at least in part on the subgroupings and the processing order, generate two or more two-dimensional (2D) placement plans for stacking the available objects along a horizontal plane; and

based at least in part on the two or more 2D placement plans, generate a three-dimensional (3D) stacking plan for stacking the available objects, wherein the 3D stacking plan includes the two or more 2D placement plans arranged along a vertical direction; and

implement the three-dimensional (3D) stacking plan for placing the available objects at the task location.

17. A tangible, non-transient computer-readable medium having processor instructions stored thereon, when executed by one or more processors of a robotic system, for causing the robotic system to implement a method, the method comprising:

accessing discretized object models that represent available objects designated for placement at a task location;

accessing a discretized model representing the task location;

calculating a processing order for subgroupings that represent groupings of the available objects;

based at least in part on the subgroupings and the processing order, generating a two-dimensional (2D) placement plan for placing the available objects along a horizontal plane;

based at least in part on the 2D placement plan, generating a three-dimensional (3D) stacking plan, wherein the 2D placement plan corresponds to at least one layer within the 3D stacking plan; and

implementing the 3D stacking plan for placing the available objects at the task location.

18. The tangible, non-transient computer-readable medium of claim 17 , wherein:

the method further comprises, calculating, based at least in part on the 3D stacking plan, a packing sequence for identifying a placing order of the available objects; and

implementing the 3D stacking plan includes implementing the 3D stacking plan according to the packing sequence.

19. The tangible, non-transient computer-readable medium of claim 18 , wherein:

the method further comprises determining the subgroupings that represent groupings of the available objects according to specified priorities, fragility ratings, object weight ranges, heights, object types, or a combination thereof;

calculating the processing order includes prioritizing the subgroupings according to a number of objects represented therein, a combined horizontal area, the object weight ranges, the fragility ratings, or a combination thereof; and

generating the 2D placement plan includes:

identifying object types representing a grouping condition for the available objects, and

iteratively deriving placement locations representing planned placement for objects of the available objects based at least in part on:

determining an initial placement location for one of the object types;

deriving candidate placement locations for a next item based on one or more placement conditions; and

sequencing the 2D placement plan according to the one or more placement conditions.

20. The tangible, non-transient computer-readable medium of claim 18 , wherein:

generating the 2D placement plan and generating the 3D stacking plan include:

converting the 2D placement plan into at least one 3D state based on height information for the objects added in the 2D placement plan, and

calculating at least one placement score for the 2D placement plan based on the at least one 3D state and at least one 3D placement preference or criteria; and

generating the 3D stacking plan further includes generating the 3D stacking plan based at least in part on selecting the 2D placement plan according to the at least one placement score.

Assignments (1)

Continuity (2)

Cited By (13)

US 12,202,145 US 12,214,975 US 12,227,374 US 12,258,223 US 12,258,227 US 12,304,750 US 12,311,551 US 12,337,481 US 12,390,928 US 12,485,544 US 12,486,125 US 12,623,843 US 12,715,716