Perception-based robotic manipulation system and method for automated truck unloader that unloads/unpacks product from trailers and containers
An automated truck unloader for unloading/unpacking product, such as boxes or cases, from trailers and containers is disclosed. In one embodiment, a mobile base structure provides a support framework for a drive subassembly, conveyance subassembly, an industrial robot, a distance measurement subassembly, and a control subassembly. Under the operation of the control subassembly, an industrial robot having a suction cup-based gripper arm selectively removes boxes from the trailer and places the boxes on a powered transportation path. The control subassembly coordinates the selective articulated movement of the industrial robot and the activation of the drive subassembly based upon a perception-based robotic manipulation system.
1. An automated truck unloader for unloading/unpacking a plurality of product, the automated truck unloader comprising:
an industrial robot conformed to handle the plurality of product;
a camera;
a control subassembly located in communication with the industrial robot and the camera, the control subassembly coordinating selective articulated movement of the industrial robot; and
the control assembly including a memory accessible to a processor, the memory including processor-executable instructions that, when executed cause the processor to:
construct a 3-D point cloud model from a plurality of first data images collected by the camera, the 3-D point cloud model being a representation of a physical environment of the industrial robot, the physical environment including the plurality of product,
transform the 3-D point cloud model into a 3-D voxel model,
specify a search operation within the 3-D voxel model to identify a candidate product corner, the candidate product corner belonging to a candidate product of the plurality of product,
provide an instruction for the camera to capture at least one second data image of a local area of the physical environment corresponding to the candidate product corner,
specify a dimensioning operation, based on the at least one second data image, to dimension the candidate product about the candidate product corner, and
specify a removal operation to unload the candidate product.
2. The automated truck unloader as recited in claim 1 , wherein the processor-executable instructions further comprise processor-executable instructions that, when executed cause the processor to calculate instructions for removing the candidate product.
3. The automated truck unloader as recited in claim 1 , wherein the 3-D voxel model further comprises a full 3-D voxel model.
4. The automated truck unloader as recited in claim 1 , wherein the instruction to specify a dimensioning operation to dimension the candidate product about the candidate product corner further comprises determining one or more features of the at least one second data image based on matching the at least one second data image to a known cuboid model.
5. The automated truck unloader as recited in claim 1 , wherein the instruction to specify a dimensioning operation to dimension the candidate product about the candidate product corner further comprises defining the location of the candidate product based on discontinuities in the at least one second data image.
6. The automated truck unloader as recited in claim 1 , wherein the industrial robot further comprises an end effector.
7. The automated truck unloader as recited in claim 6 , wherein the end effector further comprises a suction cup-based gripper arm.
8. The automated truck unloader as recited in claim 6 , wherein the end effector further comprises a suction cup-based gripper arm adapted for manipulating a box with cooperating grapplers that grip the box in a gripping position selected from the group consisting of parallel to the box and perpendicular to the box.
9. The automated truck unloader as recited in claim 8 , wherein the end effector further comprises:
a main frame having a support frame for attachment to the industrial robot; and
a moving frame half selectively pivotally coupled to the main frame by a joint to provide a range of motion between approximately a parallel presentation between the main frame and the moving frame and approximately an orthogonal presentation between the main frame and the moving frame.
10. The automated truck unloader as recited in claim 9 , further comprising:
a first plurality of suction cups associated with a face of the main frame;
a second plurality of suction cups associated with a face of the main frame;
a vacuum manifold housed within the main frame in pneumatic communication with the first and second pluralities of suction cups; and
a vacuum sensor mounted in the main frame proximate to the first plurality of suction cups, the vacuum sensor, in response to detecting an object, being configured to actuate the vacuum manifold and generate a vacuum force to grip the object.
11. The automated truck unloader as recited in claim 9 , wherein the visual detection subsystem is housed in the main frame, the visual detection subsystem being configured to capture an image of the product space for processing by the control subassembly.
12. An automated truck unloader for unloading/unpacking a plurality of product, the automated truck unloader comprising:
an industrial robot conformed to handle the plurality of product;
a camera;
a control subassembly located in communication with the industrial robot and the camera, the control subassembly coordinating selective articulated movement of the industrial robot; and
the control assembly including a memory accessible to a processor, the memory including processor-executable instructions that, when executed cause the processor to:
construct a 3-D point cloud model from a plurality of first data images collected by the camera, the 3-D point cloud model being a representation of a physical environment of the industrial robot, the physical environment including the plurality of product,
transform the 3-D point cloud model into a 3-D voxel model,
specify a search operation within the 3-D voxel model to identify a candidate product corner, the candidate product corner belonging to a candidate product of the plurality of product,
capture at least one second data image of a local area of the physical environment corresponding to the candidate product corner, and
specify a dimensioning operation, based on the at least one second data image, to dimension the candidate product about the candidate product corner.
13. The automated truck unloader as recited in claim 12 , wherein the 3-D voxel model further comprises a full 3-D voxel model.
14. The automated truck unloader as recited in claim 12 , wherein the instruction to specify a dimensioning operation to dimension the candidate product about the candidate product corner further comprises determining one or more features of the at least one second data image based on matching the at least one second data image to a known cuboid model.
15. The automated truck unloader as recited in claim 12 , wherein the instruction to specify a dimensioning operation to dimension the candidate product about the candidate product corner further comprises defining the location of the candidate product based on discontinuities in the at least one second data image.
16. The automated truck unloader as recited in claim 12 , wherein the industrial robot further comprises an end effector.
17. An automated truck unloader for unloading/unpacking a plurality of product, the automated truck unloader comprising:
an industrial robot conformed to handle the plurality of product;
a control subassembly located in communication with the industrial robot, the control subassembly coordinating selective articulated movement of the industrial robot; and
the control assembly including a memory accessible to a processor, the memory including processor-executable instructions that, when executed cause the processor to:
construct a 3-D point cloud model from a plurality of collected first data images, the 3-D point cloud model being a representation of a physical environment of the industrial robot, the physical environment including the plurality of product,
transform the 3-D point cloud model into a 3-D voxel model,
specify a search operation within the 3-D voxel model to identify a candidate product corner, the candidate product corner belonging to a candidate product of the plurality of product,
capture at least one second data image of a local area of the physical environment corresponding to the candidate product corner, and
specify a dimensioning operation, based on the at least one second data image, to dimension the candidate product about the candidate product corner.
18. The automated truck unloader as recited in claim 17 , wherein the 3-D voxel model further comprises a full 3-D voxel model.
19. The automated truck unloader as recited in claim 17 , wherein the instruction to specify a dimensioning operation to dimension the candidate product about the candidate product corner further comprises determining one or more features of the at least one second data image based on matching the at least one second data image to a known cuboid model.
20. The automated truck unloader as recited in claim 17 , wherein the instruction to specify a dimensioning operation to dimension the candidate product about the candidate product corner further comprises defining the location of the candidate product based on discontinuities in the at least one second data image.