Training artificial networks for robotic picking
Various embodiments of the present technology generally relate to robotic devices and artificial intelligence. More specifically, some embodiments relate to an artificial neural network training method that does not require extensive training data or time expenditure. The few-shot training model disclosed herein includes attempting to pick up items and, in response to a failed pick up attempt, transferring and generalizing information to similar regions to improve probability of success in future attempts. In some implementations, the training method is used to robotic device for picking items from a bin and perturbing items in a bin. When no picking strategies with high probability of success exist, the robotic device may perturb the contents of the bin to create new available pick-up points. In some implementations, the device may include one or more computer-vision systems.
1 . A method of training an artificial neural network, the method comprising:
performing, with a robotic picking device, an attempt to pick up an item at a first region on the item, wherein the first region corresponds to a first group of pixels;
in response to determining that the attempt was unsuccessful, generalizing information about the first region, wherein the generalized information about the first region includes characteristics of the first region and an associated probability of producing the unsuccessful attempt based on the characteristics;
transferring the generalized information about the first region to one or more other regions on the item, wherein transferring the generalized information comprises indicating a probability of failure associated with at least one different region corresponding to a different group of pixels, the at least one different region having one or more same characteristics as the first region;
identifying a perturbation strategy for repositioning the item based on the characteristics and performing the perturbation strategy with a perturbation element of the robotic picking device;
identifying a second region on the item at which to attempt to pick up the item, wherein the second region is distinct from the first region and corresponds to a second group of pixels lacking at least one characteristic of the first group of pixels;
performing, with the robotic picking device, a second attempt to pick up the item at the second region on the item; and
upon determining that the second attempt to pick up the item was unsuccessful, performing a third attempt to pick up the item at either the first region or the second region on the item.
2 . The method of claim 1 , wherein the characteristics of the first region comprise locational features of the item at the first region with respect to the robotic picking device, material properties of the item, and a shape of the item.
3 . The method of claim 1 , wherein transferring the generalized information about the first region to the one or more other regions on the item includes generating a probability map of regions on the item, wherein the probability map includes likelihoods of failure for picking attempts at the regions.
4 . The method of claim 1 further comprising transferring the generalized information about the first region on the item to one or more new regions on a different item having at least one characteristic matching a characteristic of the characteristics of the item.
5 . The method of claim 1 , wherein the robotic picking device comprises a picking element configured to pick up items.
6 . The method of claim 1 , wherein the item is in a first position and the method further comprises:
in response to determining that the item is in a new position, identifying a relative motion of the item when moving between the first position and the new position to determine a translation and a rotation of the item; and
identifying the first region on the item in the new position.
7 . The method of claim 1 further comprising in response to determining that the second attempt to pick up the item was unsuccessful, adjusting a probability of failure associated with the second region.
8 . The method of claim 1 , wherein the first region or the second region is selectively chosen based on probabilities associated with the first region and the second region in a position following performance of the perturbation strategy.
9 . The method of claim 1 , wherein the robotic picking device comprises:
a picking element configured to pick up items; and
the perturbation element comprising a compressed air valve configured to blow air towards the item to reposition the item.
10 . A system comprising:
a robotic picking device, the robotic picking device comprising:
a picking element; and
a perturbation element;
one or more computer-readable storage media;
a processing system operatively coupled to the one or more computer-readable storage media; and
program instructions, stored on the one or more computer-readable storage media, for training a neural network, wherein the program instructions, when read and executed by the processing system, direct the processing system to:
perform, with the robotic picking device, an attempt to pick up an item at a first region on the item, wherein the first region corresponds to a first group of pixels;
in response to a determination that the attempt was unsuccessful, generalize information about the first region, wherein the generalized information about the first region includes characteristics of the first region and an associated probability of producing the unsuccessful attempt based on the characteristics;
transfer the generalized information about the first region to one or more other regions on the item, wherein to transfer the generalized information, the program instructions direct the processing system to indicate a probability of failure associated with at least one different region corresponding to a different group of pixels, the at least one different region having one or more same characteristics as the first region;
identify a perturbation strategy for repositioning the item based on the characteristics and perform the perturbation strategy with a perturbation element of the robotic picking device;
identify a second region on the item at which to attempt to pick up the item, wherein the second region is distinct from the first region and corresponds to a second group of pixels lacking at least one characteristic of the first group of pixels;
perform, with the robotic picking device, a second attempt to pick up the item at the second region on the item; and
upon determining that the second attempt to pick up the item was unsuccessful, perform a third attempt to pick up the item at either the first region or the second region on the item.
11 . The system of claim 10 , wherein the characteristics of the first region comprise locational features of the item at the first region with respect to the robotic picking device, material properties of the item, and a shape of the item.
12 . The system of claim 10 , wherein to transfer the generalized information about the first region to the one or more other regions on the item, the program instructions direct the processing system to generate a probability map of regions on the item, wherein the probability map includes likelihoods of failure for picking attempts at the regions.
13 . The system of claim 10 , wherein the program instructions, when read and executed by the processing system, further direct the processing system to transfer the generalized information about the first region on the item to one or more new regions on a different item having at least one characteristic matching a characteristic of the characteristics of the item.
14 . The system of claim 10 , wherein the robotic picking device comprises a picking element configured to pick up items.
15 . The system of claim 10 , wherein the item is in a first position and the program instructions, when read and executed by the processing system, further direct the processing system to:
in response to determining that the item is in a new position, identify a relative motion of the item when moving between the first position and the new position to determine a translation and a rotation of the item; and
identify the first region on the item in the new position.
16 . The system of claim 10 , wherein the program instructions, when read and executed by the processing system, further direct the processing system to, in response to determining that the second attempt to pick up the item was unsuccessful, adjusting a probability of failure associated with the second region.
17 . The system of claim 10 , wherein the first region or the second region is selectively chosen based on probabilities associated with the first region and the second region in a position following performance of the perturbation strategy.
18 . The system of claim 10 , wherein the perturbation element comprises a compressed air valve configured to blow air towards the item to reposition the item.