Method and system for point cloud based grasp planning framework
A fully automated and reliable picking of a diverse range of unseen objects in clutter is a challenging problem. The present disclosure provides an optimum grasp pose selection to pick an object from a bin. Initially, the system receives an input image pertaining to a surface. Further, a plurality of sampled grasp poses are generated in a random configuration. Further, a depth difference value is computed for each of a plurality of pixels corresponding to each of the plurality of sampled grasp poses. Further, a binary map is generated for each of the plurality of sampled grasp poses and a plurality of subregions are obtained. Further, a plurality of feasible grasp poses are selected based on the plurality of subregions and a plurality of conditions. Further, the plurality of feasible grasp poses are refined and an optimum grasp pose is obtained based on a Grasp Quality Score (GQS).
1 . A processor implemented method comprising:
receiving, by one or more hardware processors, an input image pertaining to a surface in a robotic bin picking environment, wherein the surface comprises a plurality of heterogenous unseen objects;
generating, by the one or more hardware processors, a plurality of sampled grasp poses in a random configuration based on the input image, using a baseline grasp planning technique, wherein each of the plurality of sampled grasp poses is represented as rectangles;
computing, by the one or more hardware processors, a depth difference value for each of a plurality of pixels corresponding to each of the plurality of sampled grasp poses, based on a comparison between each of the plurality of pixels corresponding to each of the plurality of sampled grasp poses and a corresponding center pixel;
generating, by the one or more hardware processors, a binary map for each of the plurality of sampled grasp poses based on the depth difference value by assigning a binary value one to a plurality of pixels with a depth difference value greater than a predefined depth threshold and zero otherwise;
obtaining, by the one or more hardware processors, a plurality of subregions corresponding to each of the plurality of sampled grasp poses based on the binary map, wherein the plurality of subregions comprises a contact region, a free region and a collision region, by:
identifying a left starting point and a left ending point of a left free region of each of the plurality of sampled grasp poses based on the binary map, wherein the left free region is a region with the binary value one;
identifying a right starting point and a right ending point of a right free region of each of the plurality of sampled grasp poses based on the binary map; and
computing the plurality of subregions based on the left starting point, the left ending point, the right starting point and the right ending point using a subregion computation technique;
selecting, by the one or more hardware processors, a plurality of feasible grasp poses from the plurality of sampled grasp poses based on the plurality of subregions and a plurality of conditions;
refining, by the one or more hardware processors, each of the plurality of feasible grasp poses by (i) shifting a center corresponding to each of the plurality of feasible grasp poses along width of the corresponding grasp pose such that the contact region is divided into two equal halves, and (ii) adjusting the width corresponding to each of the plurality of feasible grasp poses such that the collision region is excluded; and
obtaining, by the one or more hardware processors, an optimum grasp poses for a robotic arm based on a refined plurality of feasible grasp poses using a Grasp Quality Score (GQS).
2 . The processor implemented method of claim 1 , wherein the plurality of conditions for selecting the plurality of feasible grasp poses from the plurality of sampled grasp poses comprises (i) if a width associated with the contact region corresponding to each of the plurality of sampled grasp poses is less than a maximum griper opening, and (ii) if the width associated with the left free region and the right free region corresponding to each of the plurality of sampled grasp poses are greater than a gripper finger width.
3 . The processor implemented method of claim 1 , wherein obtaining the optimum grasp pose based on a refined plurality of feasible grasp poses using the GQS comprises:
obtaining a Free Region Length (FRL) corresponding to each of the refined plurality of feasible grasp poses by (i) computing a plurality of free region widths on either side of each of the refined plurality of feasible grasp poses based on the free region using a pixel traversal technique, and (ii) obtaining the FRL by selecting a minimum free region width from the plurality of free region widths corresponding to the refined plurality of grasp poses;
computing a FRL score by normalizing the FRL corresponding to each of the refined plurality of feasible grasp poses using a normalization technique;
computing a Contact Region Size (CRS) corresponding to each of the refined plurality of feasible grasp poses, wherein the contact region size is a number of pixels that constitute the contact region within a fixed rectangular region around a center point;
computing a CRS score by normalizing the CRS corresponding to each of the refined plurality of feasible grasp poses using the normalization technique;
computing the GQS corresponding to each of the refined plurality of feasible grasp poses by adding the FRL score and the CRS score; and
selecting the optimum grasp pose from the refined plurality of feasible grasp poses based on the GQS, wherein the refined grasp pose with a maximum GQS from among the plurality of feasible grasp poses is selected as the optimum grasp pose.
4 . The processor implemented method of claim 1 , further comprising performing a disperse action if at least one feasible grasp pose is not obtained, wherein the disperse action is performed iteratively using a linear push policy until at least one feasible grasp pose is obtained.
5 . A system comprising:
at least one memory storing programmed instructions; one or more Input/Output (I/O) interfaces; and one or more hardware processors operatively coupled to the at least one memory, wherein the one or more hardware processors are configured by the programmed instructions to:
receive an input image pertaining to a surface in a robotic bin picking environment, wherein the surface comprises a plurality of heterogenous unseen objects;
generate a plurality of sampled grasp poses in a random configuration based on the input image, using a baseline grasp planning technique, wherein each of the plurality of sampled grasp poses is represented as rectangles;
compute a depth difference value for each of a plurality of pixels corresponding to each of the plurality of sampled grasp poses, based on a comparison between each of the plurality of pixels corresponding to each of the plurality of sampled grasp poses and a corresponding center pixel;
generate a binary map for each of the plurality of sampled grasp poses based on the depth difference value by assigning a binary value one to a plurality of pixels with a depth difference value greater than a predefined depth threshold and zero otherwise;
obtain a plurality of subregions corresponding to each of the plurality of sampled grasp poses based on the binary map, wherein the plurality of subregions comprises a contact region, a free region and a collision region, by:
identifying a left starting point and a left ending point of a left free region of each of the plurality of sampled grasp poses based on the binary map, wherein the left free region is a region with the binary value one;
identifying a right starting point and a right ending point of a right free region of each of the plurality of sampled grasp poses based on the binary map; and
computing the plurality of subregions based on the left starting point, the left ending point, the right starting point and the right ending point using a subregion computation technique;
select a plurality of feasible grasp poses from the plurality of sampled grasp poses based on the plurality of subregions and a plurality of conditions;
refine each of the plurality of feasible grasp poses by (i) shifting a center corresponding to each of the plurality of feasible grasp poses along width of the corresponding grasp pose such that the contact region is divided into two equal halves, and (ii) adjusting the width corresponding to each of the plurality of feasible grasp poses such that the collision region is excluded; and
obtain an optimum grasp poses for a robotic arm based on a refined plurality of feasible grasp poses using a Grasp Quality Score (GQS).
6 . The system of claim 5 , wherein the plurality of conditions for selecting the plurality of feasible grasp poses from the plurality of sampled grasp poses comprises (i) if a width associated with the contact region corresponding to each of the plurality of sampled grasp poses is less than a maximum griper opening, and (ii) if the width associated with the left free region and the right free region corresponding to each of the plurality of sampled grasp poses are greater than a gripper finger width.
7 . The system of claim 5 , wherein obtaining the optimum grasp pose based on a refined plurality of feasible grasp poses using the GQS comprises:
obtaining a Free Region Length (FRL) corresponding to each of the refined plurality of feasible grasp poses by (i) computing a plurality of free region widths on either side of each of the refined plurality of feasible grasp poses based on the free region using a pixel traversal technique, and (ii) obtaining the FRL by selecting a minimum free region width from the plurality of free region widths corresponding to the refined plurality of grasp poses;
computing a FRL score by normalizing the FRL corresponding to each of the refined plurality of feasible grasp poses using a normalization technique;
computing a Contact Region Size (CRS) corresponding to each of the refined plurality of feasible grasp poses, wherein the contact region size is a number of pixels that constitute the contact region within a fixed rectangular region around a center point;
computing a CRS score by normalizing the CRS corresponding to each of the refined plurality of feasible grasp poses using the normalization technique;
computing the GQS corresponding to each of the refined plurality of feasible grasp poses by adding the FRL score and the CRS score; and
selecting the optimum grasp pose from the refined plurality of feasible grasp poses based on the GQS, wherein the refined grasp pose with a maximum GQS from among the plurality of feasible grasp poses is selected as the optimum grasp pose.
8 . The system of claim 5 , further comprising performing a disperse action if at least one feasible grasp pose is not obtained, wherein the disperse action is performed iteratively using a linear push policy until at least one feasible grasp pose is obtained.
9 . One or more non-transitory machine readable information storage mediums comprising one or more instructions which when executed by one or more hardware processors causes:
receiving an input image pertaining to a surface in a robotic bin picking environment, wherein the surface comprises a plurality of heterogenous unseen objects;
generating a plurality of sampled grasp poses in a random configuration based on the input image, using a baseline grasp planning technique, wherein each of the plurality of sampled grasp poses is represented as rectangles;
computing a depth difference value for each of a plurality of pixels corresponding to each of the plurality of sampled grasp poses, based on a comparison between each of the plurality of pixels corresponding to each of the plurality of sampled grasp poses and a corresponding center pixel;
generating a binary map for each of the plurality of sampled grasp poses based on the depth difference value by assigning a binary value one to a plurality of pixels with a depth difference value greater than a predefined depth threshold and zero otherwise;
obtaining a plurality of subregions corresponding to each of the plurality of sampled grasp poses based on the binary map, wherein the plurality of subregions comprises a contact region, a free region and a collision region, by:
identifying a left starting point and a left ending point of a left free region of each of the plurality of sampled grasp poses based on the binary map, wherein the left free region is a region with the binary value one;
identifying a right starting point and a right ending point of a right free region of each of the plurality of sampled grasp poses based on the binary map; and
computing the plurality of subregions based on the left starting point, the left ending point, the right starting point and the right ending point using a subregion computation technique;
selecting a plurality of feasible grasp poses from the plurality of sampled grasp poses based on the plurality of subregions and a plurality of conditions;
refining each of the plurality of feasible grasp poses by (i) shifting a center corresponding to each of the plurality of feasible grasp poses along width of the corresponding grasp pose such that the contact region is divided into two equal halves, and (ii) adjusting the width corresponding to each of the plurality of feasible grasp poses such that the collision region is excluded; and
obtaining an optimum grasp poses for a robotic arm based on a refined plurality of feasible grasp poses using a Grasp Quality Score (GQS).
10 . The one or more non-transitory machine readable information storage mediums of claim 9 , wherein the plurality of conditions for selecting the plurality of feasible grasp poses from the plurality of sampled grasp poses comprises (i) if a width associated with the contact region corresponding to each of the plurality of sampled grasp poses is less than a maximum griper opening, and (ii) if the width associated with the left free region and the right free region corresponding to each of the plurality of sampled grasp poses are greater than a gripper finger width.
11 . The one or more non-transitory machine readable information storage mediums of claim 9 , wherein obtaining the optimum grasp pose based on a refined plurality of feasible grasp poses using the GQS comprises:
obtaining a Free Region Length (FRL) corresponding to each of the refined plurality of feasible grasp poses by (i) computing a plurality of free region widths on either side of each of the refined plurality of feasible grasp poses based on the free region using a pixel traversal technique, and (ii) obtaining the FRL by selecting a minimum free region width from the plurality of free region widths corresponding to the refined plurality of grasp poses;
computing a FRL score by normalizing the FRL corresponding to each of the refined plurality of feasible grasp poses using a normalization technique;
computing a Contact Region Size (CRS) corresponding to each of the refined plurality of feasible grasp poses, wherein the contact region size is a number of pixels that constitute the contact region within a fixed rectangular region around a center point;
computing a CRS score by normalizing the CRS corresponding to each of the refined plurality of feasible grasp poses using the normalization technique;
computing the GQS corresponding to each of the refined plurality of feasible grasp poses by adding the FRL score and the CRS score; and
selecting the optimum grasp pose from the refined plurality of feasible grasp poses based on the GQS, wherein the refined grasp pose with a maximum GQS from among the plurality of feasible grasp poses is selected as the optimum grasp pose.
12 . The one or more non-transitory machine readable information storage mediums of claim 9 , further comprising performing a disperse action if at least one feasible grasp pose is not obtained, wherein the disperse action is performed iteratively using a linear push policy until at least one feasible grasp pose is obtained.