Perception-based robotic manipulation system and method for automated truck unloader that unloads/unpacks product from trailers and containers
An automated truck unloader for unloading/unpacking product, such as boxes or cases, from trailers and containers is disclosed. In one embodiment, a mobile base structure provides a support framework for a drive subassembly, conveyance subassembly, an industrial robot, a distance measurement subassembly, and a control subassembly. Under the operation of the control subassembly, an industrial robot having a suction cup-based gripper arm selectively removes boxes from the trailer and places the boxes on a powered transportation path. The control subassembly coordinates the selective articulated movement of the industrial robot and the activation of the drive subassembly based upon a perception-based robotic manipulation system.
1. A method for unloading/unpacking a plurality of product with an automated truck unloader, the method comprising:
positioning the automated truck unloader to capture a plurality of first data images of a physical environment including the plurality of product, the automated truck unloader having an industrial robot conformed to handle the plurality of product, the automated truck unloader having a camera;
capturing the plurality of first data images of the physical environment with the camera;
constructing a 3-D point cloud model from the plurality of first data images collected by the camera, the 3-D point cloud model being a representation of the physical environment;
transforming the 3-D point cloud model into a 3-D voxel model;
specifying a search operation within the 3-D voxel model to identify a candidate product corner, the candidate product corner belonging to a candidate product of the plurality of product;
capturing, with the camera, at least one second data image of a local area of the physical environment corresponding to the candidate product corner;
specifying a dimensioning operation, based on the at least one second data image, to dimension the candidate product about the candidate product corner; and
specifying a removal operation to unload the candidate product.
2. The method as recited in claim 1 , further comprising calculating instructions for removing the candidate product.
3. The method as recited in claim 1 , further comprising executing the removal operation to unload the candidate product.
4. The method as recited in claim 1 , wherein capturing the plurality of first data images of the physical environment with the camera further comprises performing a set number of detection poses by the industrial robot, each of the detection poses providing a different view of the physical environment.
5. The method as recited in claim 1 , wherein transforming the 3-D point cloud model into a 3-D voxel model further comprises transforming the 3-D point cloud model into a full 3-D voxel model.
6. The method as recited in claim 1 , wherein specifying the dimensioning operation to dimension the candidate product about the candidate product corner further comprises determining one or more features of the at least one second data image based on matching the at least one second data image to a known cuboid model.
7. The method as recited in claim 1 , wherein the specifying a dimensioning operation to dimension the candidate product about the candidate product corner further comprises defining the location of the candidate product based on discontinuities in the at least one second data image.
8. The method as recited in claim 1 , wherein specifying the removal operation to unload the candidate product further comprises specifying the removal operation to unload the candidate product with an end effector.
9. The method as recited in claim 1 , wherein specifying the removal operation to unload the candidate product further comprises specifying the removal operation to unload the candidate product with a suction cup-based gripper arm.
10. The method as recited in claim 1 , wherein specifying the removal operation to unload the candidate product further comprises specifying the removal operation to unload the candidate product with a suction cup-based gripper arm adapted for manipulating a box with cooperating grapplers that grip the box in a gripping position selected from the group consisting of parallel to the box and perpendicular to the box.
11. The method as recited in claim 1 , wherein capturing the plurality of first data images of the physical environment with the camera further comprises capturing the plurality of first data images with a visual detection subsystem configured to capture an image of the product space for processing.
12. A method for unloading/unpacking a plurality of product with an automated truck unloader, the method comprising:
positioning the automated truck unloader to capture a plurality of first data images of a physical environment including the plurality of product, the automated truck unloader having an industrial robot conformed to handle the plurality of product, the automated truck unloader having a camera;
capturing the plurality of first data images of the physical environment with the camera;
constructing a 3-D point cloud model from the plurality of first data images collected by the camera, the 3-D point cloud model being a representation of the physical environment;
transforming the 3-D point cloud model into a 3-D voxel model;
specifying a search operation within the 3-D voxel model to identify a candidate product corner, the candidate product corner belonging to a candidate product of the plurality of product;
capturing, with the camera, at least one second data image of a local area of the physical environment corresponding to the candidate product corner; and
specifying a dimensioning operation, based on the at least one second data image, to dimension the candidate product about the candidate product corner.
13. The method as recited in claim 12 , wherein capturing the plurality of first data images of the physical environment with the camera further comprises performing a set number of detection poses by the industrial robot, each of the detection poses providing a different view of the physical environment.
14. The method as recited in claim 12 , wherein specifying the dimensioning operation to dimension the candidate product about the candidate product corner further comprises determining one or more features of the at least one second data image based on matching the at least one second data image to a known cuboid model.
15. The method as recited in claim 12 , wherein transforming the 3-D point cloud model into a 3-D voxel model further comprises transforming the 3-D point cloud model into a full 3-D voxel model.
16. The method as recited in claim 12 , wherein the specifying a dimensioning operation to dimension the candidate product about the candidate product corner further comprises defining the location of the candidate product based on discontinuities in the at least one second data image.
17. A method for unloading/unpacking a plurality of product with an automated truck unloader, the method comprising:
positioning the automated truck unloader to capture a plurality of first data images of a physical environment including the plurality of product, the automated truck unloader having a camera;
capturing the plurality of first data images of the physical environment with the camera;
constructing a 3-D point cloud model from the plurality of first data images collected by the camera, the 3-D point cloud model being a representation of the physical environment;
transforming the 3-D point cloud model into a 3-D voxel model;
specifying a search operation within the 3-D voxel model to identify a candidate product corner, the candidate product corner belonging to a candidate product of the plurality of product;
capturing, with the camera, at least one second data image of a local area of the physical environment corresponding to the candidate product corner; and
specifying a dimensioning operation, based on the at least one second data image, to dimension the candidate product about the candidate product corner.
18. The method as recited in claim 17 , wherein specifying the dimensioning operation to dimension the candidate product about the candidate product corner further comprises determining one or more features of the at least one second data image based on matching the at least one second data image to a known cuboid model.
19. The method as recited in claim 17 , wherein transforming the 3-D point cloud model into a 3-D voxel model further comprises transforming the 3-D point cloud model into a full 3-D voxel model.
20. The method as recited in claim 17 , wherein the specifying a dimensioning operation to dimension the candidate product about the candidate product corner further comprises defining the location of the candidate product based on discontinuities in the at least one second data image.