Robotic system for performing pattern recognition-based inspection of pharmaceutical containers
A robotic inspection platform comprises a robotic arm, an imager, and a controller. The controller causes the robotic arm to retrieve, using its end effector, a container, and to manipulate the container such that the container is sequentially placed in a plurality of orientations while in view of the imager. The controller also causes the imager to capture images, with each of the images being captured while the container is in a respective one of the orientations. The controller also determines one or more attributes of the container, and/or a sample within the container, by analyzing the images using a pattern recognition model and, based on the attribute(s), determines whether the container and/or sample satisfies one or more criteria. If the container and/or sample fails to satisfy the criteria, the controller causes the robotic arm to place the container in an area (e.g., bin) reserved for rejected containers and/or samples.
1. A robotic inspection platform, comprising:
a robotic arm comprising an end effector and a plurality of articulable segments, the robotic arm being configured to manipulate containers with greater than four degrees of freedom of movement;
an imaging system comprising a single imager; and
a controller configured to:
cause the robotic arm to retrieve, using the end effector, a container,
cause the robotic arm to manipulate the container using the end effector, such that the container is sequentially placed in a plurality of orientations while the container is in view of the single imager,
cause the single imager to capture a plurality of images, with each of the plurality of images being captured while the container is in a respective one of the plurality of orientations,
determine one or more attributes of the container, or a sample within the container, by analyzing the plurality of images using a pattern recognition model, and
based on the one or more attributes of the container or the sample, either (i) cause the robotic arm to place the container in an area reserved for rejected containers and/or samples, or (ii) cause the robotic arm to place the container in an area reserved for containers and/or samples that have not been rejected.
2. The robotic inspection platform of claim 1 , wherein the controller is configured to determine the one or more attributes of the container or the sample by analyzing the plurality of images using a machine learning model, the machine learning model being trained to classify container and/or sample attributes.
3. The robotic inspection platform of claim 2 , wherein the controller is configured to facilitate training of the machine learning model at least by:
causing the robotic arm to sequentially retrieve, using the end effector, a plurality of training containers; and
for each of the plurality of training containers that is retrieved, (i) causing the robotic arm to manipulate the training container using the end effector such that the training container is sequentially placed in the plurality of orientations while the training container is in view of the single imager, (ii) causing the single imager to capture a respective plurality of training images, with each of the respective plurality of training images being captured while the training container is in a respective one of the plurality of orientations, and (iii) causing the respective plurality of images to be presented to a user via a display to facilitate manual labeling of attributes for the training container and/or a sample within the training container.
4. The robotic inspection platform of claim 1 , wherein the controller is configured to distinguish between different types of objects by analyzing the plurality of images using the pattern recognition model.
5. The robotic inspection platform of claim 4 , wherein the different types of objects include bubbles and particles.
6. The robotic inspection platform of claim 4 , wherein the different types of objects include one or both of (i) specific types of bubbles and (ii) specific types of particles.
7. The robotic inspection platform of claim 4 , wherein the controller is further configured to distinguish between the different types of objects based on one or more criteria that include at least one criterion dependent upon object type.
8. The robotic inspection platform of claim 1 , wherein the plurality of orientations includes a plurality of rotations about a longitudinal axis of the container.
9. The robotic inspection platform of claim 1 , wherein the area reserved for rejected containers and/or samples is a bin.
10. The robotic inspection platform of claim 1 , wherein the controller is configured to cause the robotic arm to retrieve the container from a plate, and wherein the area reserved for containers and/or samples that have not been rejected is an area within the plate.
11. The robotic inspection platform of claim 1 , wherein the robotic arm is configured to manipulate the containers with six degrees of freedom of movement, and
wherein the container has a first roll angle and a first pitch angle in a first orientation of the plurality of orientations, and the container has (i) a second roll angle that is different from the first roll angle or (ii) a second pitch angle that is different from the first pitch angle in a second orientation of the plurality of orientations.
12. A method for performing inspection of containers using pattern recognition, the method comprising:
causing a robotic arm to retrieve, using an end effector of the robotic arm, a container, the robotic arm being configured to manipulate containers with greater than four degrees of freedom of movement;
causing the robotic arm to manipulate the container using the end effector, such that the container is sequentially placed in a plurality of orientations while the container is in view of a single imager;
causing the single imager to capture a plurality of images, with each of the plurality of images being captured while the container is in a respective one of the plurality of orientations;
determining one or more attributes of the container, or a sample within the container, by analyzing the plurality of images using a pattern recognition model; and
based on the one or more attributes of the container or the sample, either (i) causing the robotic arm to place the container in an area reserved for rejected containers and/or samples, or (ii) causing the robotic arm to place the container in an area reserved for containers and/or samples that have not been rejected.
13. The method of claim 12 , wherein determining the one or more attributes of the container or the sample by analyzing the plurality of images using a pattern recognition model includes:
determining the one or more attributes of the container or the sample by analyzing the plurality of images using a machine learning model, the machine learning model being trained to classify container and/or sample attributes.
14. The method of claim 13 , further comprising facilitating training of the machine learning model at least by:
causing the robotic arm to sequentially retrieve, using the end effector, a plurality of training containers; and
for each of the plurality of training containers that is retrieved, (i) causing the robotic arm to manipulate the training container using the end effector such that the training container is sequentially placed in the plurality of orientations while the training container is in view of the single imager, (ii) causing the single imager to capture a respective plurality of training images, with each of the respective plurality of training images being captured while the training container is in a respective one of the plurality of orientations, and (iii) causing the respective plurality of images to be presented to a user via a display to facilitate manual labeling of attributes for the training container and/or a sample within the training container.
15. The method of claim 12 , wherein determining the one or more attributes of the container or the sample by analyzing the plurality of images using the pattern recognition model includes distinguishing between different types of objects by (i) analyzing the plurality of images using the pattern recognition model, and (ii) based on one or more criteria that include at least one criterion dependent upon object type.
16. The method of claim 12 , wherein the plurality of orientations includes a plurality of rotations about a longitudinal axis of the container.
17. The method of claim 12 , wherein the robotic arm is configured to manipulate the containers with six degrees of freedom of movement, and
wherein the container has a first roll angle and a first pitch angle in a first orientation of the plurality of orientations, and the container has (i) a second roll angle that is different from the first roll angle or (ii) a second pitch angle that is different from the first pitch angle in a second orientation of the plurality of orientations.
18. A method for performing inspection of containers using pattern recognition, the method comprising:
causing a robotic arm to retrieve, using an end effector of the robotic arm, a container;
causing a single imager to be sequentially placed in a plurality of orientations while the container is in view of the single imager;
causing the single imager to capture a plurality of images of the container, with each of the plurality of images being captured while the single imager is in a respective one of the plurality of orientations;
determining one or more attributes of the container or a sample within the container by analyzing the plurality of images using a pattern recognition model; and
based on the one or more attributes of the container or the sample, either (i) causing the robotic arm to place the container in an area reserved for rejected containers and/or samples, or (ii) causing the robotic arm to place the container in an area reserved for containers and/or samples that have not been rejected.
19. The method of claim 18 , wherein determining the one or more attributes of the container or the sample by analyzing the plurality of images using a pattern recognition model includes:
determining the one or more attributes of the container or the sample by analyzing the plurality of images using a machine learning model, the machine learning model being trained to classify container and/or sample attributes.
20. The method of claim 19 , further comprising facilitating training of the machine learning model at least by:
causing the robotic arm to sequentially retrieve, using the end effector, a plurality of training containers; and
for each of the plurality of training containers that is retrieved, (i) causing the single imager to be sequentially placed in the plurality of orientations while the training container is in view of the single imager, (ii) causing the single imager to capture a respective plurality of training images, with each of the respective plurality of training images being captured while the single imager is in a respective one of the plurality of orientations, and (iii) causing the respective plurality of images to be presented to a user via a display to facilitate manual labeling of attributes for the training container and/or a sample within the training container.