Perception-based robotic manipulation system and method for automated truck unloader that unloads/unpacks product from trailers and containers
An automated truck unloader for unloading/unpacking product, such as boxes or cases, from trailers and containers is disclosed. In one embodiment, a mobile base structure provides a support framework for a drive subassembly, conveyance subassembly, an industrial robot, a distance measurement subassembly, and a control subassembly. Under the operation of the control subassembly, an industrial robot having a suction cup-based gripper arm selectively removes boxes from the trailer and places the boxes on a powered transportation path. The control subassembly coordinates the selective articulated movement of the industrial robot and the activation of the drive subassembly based upon a perception-based robotic manipulation system.
1. A method for unloading/unpacking a plurality of product from trailers and containers, the method comprising:
capturing a plurality of first data images of a physical environment with a camera, the physical environment including the plurality of product;
constructing a 3-D model from the plurality of first data images collected by the camera, the 3-D model being a representation of the physical environment;
transforming the 3-D model into a 3-D voxel model;
specifying a search operation within the 3-D voxel model to identify a candidate product corner, the candidate product corner belonging to a candidate product of the plurality of product;
capturing, with the camera, at least one second data image of a local area of the physical environment corresponding to the candidate product corner;
specifying a dimensioning operation, based on the at least one second data image, to dimension the candidate product about the candidate product corner; and
specifying a removal operation to unload the candidate product.
2. The method as recited in claim 1 , further comprising calculating instructions for removing the candidate product.
3. The method as recited in claim 1 , further comprising executing the removal operation to unload the candidate product.
4. The method as recited in claim 1 , wherein constructing a 3-D model from the plurality of first data images collected by the camera further comprises constructing a 3-D point cloud model.
5. The method as recited in claim 1 , wherein constructing a 3-D model from the plurality of first data images collected by the camera further comprises constructing a partial 3-D point cloud model.
6. The method as recited in claim 1 , wherein transforming the 3-D model into a 3-D voxel model further comprises transforming the 3-D model into a full 3-D voxel model.
7. The method as recited in claim 1 , wherein transforming the 3-D model into a 3-D voxel model further comprises transforming the 3-D model into a partial 3-D voxel model.
8. The method as recited in claim 1 , wherein specifying the dimensioning operation to dimension the candidate product about the candidate product corner further comprises determining one or more features of the at least one second data image based on matching the at least one second data image to a known cuboid model.
9. The method as recited in claim 1 , wherein specifying a dimensioning operation to dimension the candidate product about the candidate product corner further comprises defining the location of the candidate product based on discontinuities in the at least one second data image.
10. The method as recited in claim 1 , wherein specifying the removal operation to unload the candidate product further comprises specifying the removal operation to unload the candidate product with an end effector.
11. The method as recited in claim 1 , wherein specifying the removal operation to unload the candidate product further comprises specifying the removal operation to unload the candidate product with a suction cup-based gripper arm.
12. The method as recited in claim 1 , wherein specifying the removal operation to unload the candidate product further comprises specifying the removal operation to unload the candidate product with a suction cup-based gripper arm adapted for manipulating a box with cooperating grapplers that grip the box in a gripping position selected from the group consisting of parallel to the box and perpendicular to the box.
13. A method for unloading/unpacking a plurality of product from trailers and containers, the method comprising:
capturing a plurality of first data images of a physical environment with a camera, the physical environment including the plurality of product;
constructing a 3-D model from the plurality of first data images collected by the camera, the 3-D model being a representation of the physical environment;
transforming the 3-D model into a 3-D voxel model;
specifying a search operation within the 3-D voxel model to identify a candidate product corner, the candidate product corner belonging to a candidate product of the plurality of product;
capturing, with the camera, at least one second data image of a local area of the physical environment corresponding to the candidate product corner; and
specifying a dimensioning operation, based on the at least one second data image, to dimension the candidate product about the candidate product corner.
14. The method as recited in claim 13 , wherein specifying the dimensioning operation to dimension the candidate product about the candidate product corner further comprises determining one or more features of the at least one second data image based on matching the at least one second data image to a known cuboid model.
15. The method as recited in claim 13 , wherein transforming the 3-D model into a 3-D voxel model further comprises transforming the 3-D model into a full 3-D voxel model.
16. The method as recited in claim 13 , wherein transforming the 3-D model into a 3-D voxel model further comprises transforming the 3-D model into a partial 3-D voxel model.
17. The method as recited in claim 13 , wherein constructing a 3-D model from the plurality of first data images collected by the camera further comprises constructing a 3-D point cloud model.
18. The method as recited in claim 13 , wherein constructing a 3-D model from the plurality of first data images collected by the camera further comprises constructing a partial 3-D point cloud model.
19. A method for unloading/unpacking a plurality of product from trailers and containers, the method comprising:
capturing a plurality of first data images of a physical environment with a camera, the physical environment including the plurality of product;
constructing a 3-D model from the plurality of first data images collected by the camera, the 3-D model being a representation of the physical environment;
transforming the 3-D model into a 3-D voxel model;
identifying a candidate product corner within the 3-D voxel model, the candidate product corner belonging to a candidate product of the plurality of product;
capturing, with the camera, at least one second data image of a local area of the physical environment corresponding to the candidate product corner; and
specifying a dimensioning operation, based on the at least one second data image, to dimension the candidate product about the candidate product corner.
20. The method as recited in claim 19 , wherein constructing a 3-D model from the plurality of first data images collected by the camera further comprises constructing a model selected from the group consisting of 3-D point cloud models and partial 3-D point cloud models.