Drawer vision system
Methods and systems for detecting properties of sample tubes in a laboratory environment include a drawer vision system that can be trained and calibrated. Images of a tube tray captured by at least one camera are analyzed to extract image patches that allow a processor to automatically determine if a tube slot is occupied, if the tube has a cap, and if the tube has a tube top cup. The processor can be trained using a random forest technique and a plurality of training image patches. Cameras can be calibrated using a three-dimensional calibration target that can be inserted into the drawer.
1. A method for detecting properties of sample tubes, comprising steps of:
a) capturing a series of images of a sample tray using at least one overhead camera;
b) receiving at a processor the series of images of the tray from the at least one overhead camera;
c) extracting, by the processor, a plurality of image patches from each image, each image patch corresponding to a portion of each image based on features in each image;
d) automatically determining, using the processor, from a first subset of the plurality image patches, each patch corresponding to one of a plurality of slots in the tray, whether each of a plurality of slots contains a sample tube;
e) for those plurality of slots that contain a sample tube, automatically determining, using the processor, from a second subset of the plurality image patches, each patch corresponding to the top of the sample tube, whether each sample tube has a cap; and
f) for those tubes that do not have a cap, automatically determining, using the processor, from the second subset of the plurality image patches whether each sample tube has a tube-top cup or is a plain tube.
2. The method of claim 1 , wherein the series of images comprises images of the tray at predetermined positions in a tray drawer.
3. The method of claim 1 , wherein the processor uses a set of fiducial markers on the tray surface to determine the location of each patch corresponding to one of a plurality of slots in the tray.
4. The method of claim 1 , wherein the step of automatically determining whether each of a plurality of slots contains a sample tube comprises:
a. for each slot, identifying a patch in at least one image in the series of images that corresponds to that slot based on optical marks on the tray surface; and
b. determining, for each identified patch, a probability that the slot is occupied by a sample tube.
5. The method of claim 1 , wherein the step of automatically determining whether each sample tube has a cap comprises:
a. for each sample tube, identifying a patch in at least one image in the series of images that corresponds to the top of the sample tube based on the detection of a circle in the at least one image; and
b. determining, for each identified patch, a probability that the sample tube has a cap.
6. The method of claim 1 , wherein the step of automatically determining whether each sample tube has a tube-top cup comprises:
a. for each sample tube, identifying a patch in at least one image in the series of images that corresponds to the top of the sample tube; and
b. determining, for each identified patch, a probability that the sample tube has a tube-top cup.
7. The method of claim 1 , further comprising the step of automatically determining, using the processor, for each sample tube, at least one of: tube type; tube height, tube diameter; tube offset; cap color; and fluid type.
8. The method of claim 1 , further comprising the step of automatically identifying, using the processor, a tray type from the series of images.
9. The method of claim 1 , further comprising the step of calibrating at least one camera, which is configured to capture the plurality of images, using a 3D target having a plurality of unique digital markers.
10. The method of claim 1 , further comprising the step of training the processor to perform the determining steps using a random forest technique and a plurality of training images.
11. A vision system for use in an in vitro diagnostics environments comprising:
a drawer configured to receive a tray, wherein the tray comprises a plurality of slots, each configured to receive a sample tube;
at least one overhead camera configured to capture a series of images of the tray as the drawer is moved;
a processor configured to perform the following steps:
a. receiving the series of images of the tray from the at least one camera;
b. extracting a plurality of image patches from each image, each image patch corresponding to a portion of each image based on features in each image;
c. automatically determining, from a first subset of the plurality image patches, each patch corresponding to one of a plurality of slots in the tray, whether each of a plurality of slots contains a sample tube;
d. for those plurality of slots that contain a sample tube, automatically determining, from a second subset of the plurality image patches, each patch corresponding to the top of the sample tube, whether each sample tube has a cap; and
e. for those tubes that do not have a cap, automatically determining, from the second subset of the plurality image patches whether each sample tube has a tube-top cup.
12. The system of claim 11 , wherein the series of images comprises images of the tray at predetermined positions in the drawer.
13. The system of claim 11 , wherein a set of fiducial markers on the surface of the tray to determine each patch corresponding to one of a plurality of slots in the tray.
14. The system of claim 11 , wherein the step of automatically determining whether each of a plurality of slots contains a sample tube comprises:
a. for each slot, identifying a patch in at least one image in the series of images, which corresponds to that slot based on optical marks on the tray surface; and
b. determining, for each identified patch, a probability that the slot is occupied by a sample tube.
15. The system of claim 11 , wherein the step of automatically determining whether each sample tube has a cap comprises:
a. for each sample tube, identifying a patch in at least one image in the series of images that corresponds to the top of the sample tube based on the detection of a circle in the at least one image; and
b. determining, for each identified patch, a probability that the sample tube has a cap.
16. The system of claim 11 , wherein the step of automatically determining whether each sample tube has a tube-top cup comprises:
a. for each sample tube, identifying a patch in at least one image in the series of images that corresponds to the top of the sample tube; and
b. determining, for each identified patch, a probability that the sample tube has a tube-top cup.
17. The system of claim 11 , further comprising the step of automatically determining, using the processor, for each sample tube, at least one of: tube type; tube height; tube diameter; tube offset; cap color; and fluid type.
18. The system of claim 11 , wherein the processor is configured to perform the step of automatically identifying a tray type from the series of images.
19. The system of claim 11 , wherein the processor is configured to perform the step of calibrating the at least one camera using a 3D target having a plurality of unique digital markers.
20. The system of claim 11 , wherein the processor is configured to perform the step of training the processor to perform the determining steps using a random forest technique and a plurality of training images.