Perception-based robotic manipulation system and method for automated truck unloader that unloads/unpacks product from trailers and containers

1. An automated truck unloader ( 10 ) for unloading/unpacking a plurality of product ( 46 ), the automated truck unloader ( 10 ) comprising:

an industrial robot ( 56 ) disposed configured to provide selective articulated movement of an end effector ( 134 ) between a powered transportation path ( 88 ) and a reachable space such that the industrial robot ( 56 ) is operable to handle the product ( 46 ) in the reachable space;

the powered transportation path ( 88 ) configured to transfer product ( 46 ) away from the industrial robot ( 56 );

a camera ( 322 );

a control subassembly ( 62 ) located in communication with the industrial robot ( 56 ) and the camera ( 322 ), the control subassembly ( 62 ) coordinating the selective articulated movement of the industrial robot ( 56 ); and

the control assembly ( 62 ) including a memory ( 360 ) accessible to a processor ( 350 ), the memory ( 360 ) including processor-executable instructions that, when executed cause the processor to:

construct a 3-D point cloud model ( 402 ) from a plurality of first data images ( 400 ) collected by the camera ( 322 ), the 3-D point cloud model ( 402 ) being a representation of a physical environment of the industrial robot ( 56 ), the physical environment including the plurality of product ( 46 ),

transform the 3-D point cloud model ( 402 ) into a 3-D voxel model ( 404 ),

specify a search operation within the 3-D voxel model ( 404 ) to identify a candidate product corner ( 406 ), the candidate product corner ( 406 ) belonging to a candidate product of the plurality of product ( 46 ),

provide an instruction for the camera ( 322 ) to capture at least one second data image of a local area of the physical environment corresponding to the candidate product corner ( 406 ),

specify a dimensioning operation ( 408 ), based on the at least one second data image, to dimension the candidate product about the candidate product corner ( 406 ),

specify a removal operation to unload the candidate product, and

calculate instructions for removing the candidate product.

2. The automated truck unloader ( 10 ) as recited in claim 1 , wherein the 3-D point cloud model ( 402 ) further comprises a partial 3-D point cloud model.

3. The automated truck unloader ( 10 ) as recited in claim 1 , wherein the 3-D voxel model ( 404 ) further comprises a full 3-D voxel model.

4. The automated truck unloader ( 10 ) as recited in claim 1 , wherein the instruction to specify a dimensioning operation to dimension the candidate product about the candidate product corner ( 406 ) further comprises determining one or more features of the at least one second data image based on matching the at least one second data image to a known cuboid model.

5. The automated truck unloader ( 10 ) as recited in claim 1 , wherein the instruction to specify a dimensioning operation to dimension the candidate product about the candidate product corner ( 406 ) further comprises defining the location of the candidate product based on discontinuities in the at least one second data image.

6. The automated truck unloader ( 10 ) as recited in claim 1 , wherein the end effector further comprises a suction cup-based gripper arm.

7. The automated truck unloader ( 10 ) as recited in claim 1 , wherein the end effector ( 134 ) further comprises a suction cup-based gripper arm adapted for manipulating a box with cooperating grapplers that grip the box in a gripping position selected from the group consisting of parallel to the box and perpendicular to the box.

8. The automated truck unloader ( 10 ) as recited in claim 1 , wherein the end effector ( 134 ) further comprises:

a main frame ( 280 ) having a support frame for attachment to the industrial robot;

a moving frame half ( 282 ) selectively pivotally coupled to the main frame ( 280 ) by a joint ( 284 ) to provide a range of motion between approximately a parallel presentation between the main frame ( 280 ) and the moving frame and approximately an orthogonal presentation between the main frame and the moving frame;

a first plurality of suction cups ( 302 , 304 ) associated with a face of the main frame ( 280 );

a second plurality of suction cups ( 302 , 304 ) associated with a face of the main frame ( 280 );

a vacuum manifold ( 314 , 316 ) housed within the main frame in pneumatic communication with the first and second pluralities of suction cups ( 302 , 304 );

a vacuum sensor ( 318 , 320 ) mounted in the main frame ( 280 ) proximate to the first plurality of suction cups ( 302 , 304 ), the vacuum sensor ( 318 , 320 ), in response to detecting an object, being configured to actuate the vacuum manifold ( 314 , 316 ) and generate a vacuum force to grip the object; and

the visual detection subsystem housed in the main frame ( 280 ), the visual detection subsystem being configured to capture an image of the product space for processing by the control subassembly ( 62 ).

9. An automated truck unloader ( 10 ) for unloading/unpacking a plurality of product, the automated truck unloader ( 10 ) comprising:

an industrial robot ( 56 ) disposed configured to provide selective articulated movement of an end effector ( 134 ) between a powered transportation path ( 88 ) and a reachable space such that the industrial robot ( 56 ) is operable to handle the product in the reachable space;

the powered transportation path ( 88 ) configured to transfer product away from the industrial robot ( 56 );

a camera ( 322 );

a control subassembly ( 62 ) located in communication with the industrial robot ( 56 ) and the camera ( 322 ), the control subassembly ( 62 ) coordinating the selective articulated movement of the industrial robot ( 56 ); and

the control assembly ( 62 ) including a memory ( 360 ) accessible to a processor ( 350 ), the memory ( 360 ) including processor-executable instructions that, when executed cause the processor to:

construct a 3-D point cloud model ( 402 ) from a plurality of first data images ( 400 ) collected by the camera ( 322 ), the 3-D point cloud model ( 402 ) being a representation of a physical environment of the industrial robot ( 56 ), the physical environment including the plurality of product ( 46 ),

transform the 3-D point cloud model ( 402 ) into a 3-D voxel model ( 404 ),

specify a search operation within the 3-D voxel model ( 404 ) to identify a candidate product corner ( 406 ), the candidate product corner ( 406 ) belonging to a candidate product of the plurality of product ( 46 ),

provide an instruction for the camera ( 322 ) to capture at least one second data image of a local area of the physical environment corresponding to the candidate product corner ( 406 ),

calculate, based on the at least one second data image, instructions for removing the candidate product, and

update the 3-D voxel model with the at least one second data image.

10. An automated truck unloader ( 10 ) for unloading/unpacking a plurality of product, the automated truck unloader ( 10 ) comprising:

an industrial robot ( 56 ) disposed configured to provide selective articulated movement of an end effector ( 134 ) between a powered transportation path ( 88 ) and a reachable space such that the industrial robot ( 56 ) is operable to handle the product in the reachable space;

the powered transportation path ( 88 ) configured to transfer product away from the industrial robot ( 56 );

a camera ( 322 );

a control subassembly ( 62 ) located in communication with the industrial robot ( 56 ) and the camera ( 322 ), the control subassembly ( 62 ) coordinating the selective articulated movement of the industrial robot ( 56 ); and

the control assembly ( 62 ) including a memory ( 360 ) accessible to a processor ( 350 ), the memory ( 360 ) including processor-executable instructions that, when executed cause the processor ( 350 ) to:

construct a 3-D point cloud model ( 402 ) from a plurality of first data images ( 400 ) collected by the camera ( 322 ), the 3-D point cloud model ( 402 ) being a representation of a physical environment of the industrial robot ( 56 ), the physical environment including the plurality of product ( 46 ),

transform the 3-D point cloud model ( 402 ) into a 3-D voxel model ( 404 ),

identify segmented planes within the 3-D voxel model ( 404 ) using connected component plane segmentation,

identify sufficiently sized segmented planes by fitting a minimum area bounding rectangle to each set of inliers for the previously identified segmented planes,

identify a candidate product by selecting one of the sufficiently sized segmented planes with closed edge point to forecasted box corner dimensions, and

calculate instructions for removing the candidate product.