IP Library Granted Patent US 9,619,692
Granted Patent B2
US 9,619,692 · App. 14/429,752 · Granted Apr 11, 2017

Image capture-based predictive control device

Inventor: Alexander Gelbman (Lake Worth, FL)
Assignee: Siemens Healthcare Diagnostics Inc.
G06K9/00214G06F3/0304G06K9/00771G06T7/20H04N5/23229G06T2207/20081
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 9,619,692
App. No.
14/429,752
Granted
Apr 11, 2017
Kind
B2
Abstract

A system and method for identifying objects being carried by an operator who is approaching an instrument. The system includes image-, motion-, and depth-capturing sensors that are in communication with the instrument. The captured image, motion, and depth data are compared to data stored in a database and the objects are identified. Once the objects have been identified, an action that corresponds to the identified objects is initiated in the instrument.

Claims (90)

1. A method of initiating a response corresponding to at least one object being transported by an operator, wherein each of the at least one object is to be introduced into an instrument, the method comprising:

capturing first image data of a first object of the at least one object being transported by the operator;

transmitting the captured first image data to a processing device;

determining whether the operator is approaching the instrument;

determining whether the operator is transporting at least one object;

in response to determining that the operator is approaching the instrument and that the operator is carrying at least one object, comparing, by the processing device, said captured first image data to previously stored data;

identifying, by the processing device, the first object based at least in part on said comparison;

in response to identifying the first object as being an object to be received by the instrument, retrieving a predetermined first action corresponding to the identified first object;

monitoring a status of consumable materials used by the instrument;

determining a sequence of the predetermined first action and at least one additional action to be implemented on the instrument based on the status of the consumable materials; and

automatically implementing, by the processing device, the predetermined first action and the at least one additional action on the instrument in accordance with the sequence.

2. The method of claim 1 , wherein capturing first image data comprises at least one of:

capturing motion data about the first object, and

capturing depth data about the first object.

3. The method as recited in claim 2 , wherein comparing said captured first image data to previously stored data comprises:

comparing at least one of the captured motion data and captured depth data to the previously stored data.

4. The method as recited in claim 1 , wherein the operator is transporting a second object, in addition to the first object, the method further comprising:

capturing second image data of the second object;

transmitting the captured second image data to the processing device;

comparing, by the processing device, said captured second image data to previously stored data;

identifying, by the processing device, the second object based on said comparison; and

in response to identifying the second object as being another object to be received by the instrument, retrieving a predetermined second action corresponding to the identified second object,

wherein determining the sequence of the predetermined first action and the at least one additional action includes determining a sequence of the predetermined first and second actions based on pre-established respective priorities of the predetermined first and second actions and based on the status of the consumable materials.

5. The method as recited in claim 4 , wherein determining the sequence comprises prioritizing the first and second predetermined actions as a function of relative urgencies of a respective replacement status of the first and second objects.

6. The method as recited in claim 1 , wherein retrieving a predetermined first action comprises accessing stored information.

7. The method as recited in claim 1 , wherein identifying the first object based on said comparison comprises:

setting a comparison value for each comparison of the captured first image data to the previously stored data;

comparing the comparison value to a match threshold value; and

identifying the first object if the comparison value meets or exceeds the match threshold value.

8. The method as recited in claim 1 , wherein the first object is chosen from:

a tray, a puck or a rack containing one or more specimen tubes or containment vessels; and

a reagent container.

9. The method as recited in claim 1 , wherein the predetermined first action comprises one of:

a) opening one of: a drawer, a door, or a cover; or

b) providing access to a loading area.

10. The method as recited in claim 1 , wherein the instrument is a clinical diagnostic laboratory instrument.

11. The method as recited in claim 1 , wherein, when identifying the first object, the processing device is further configured to, in response to not identifying the first object based on the comparison of the captured first image data to previously stored data, identify the first object based at least in part on at least one of: how the first object is being carried by the operator, how the first object moves while being carried, and any identified items that are being carried with the first object.

12. The method as recited in claim 1 , wherein, when identifying the first object, the processing device is further configured to identify the first object based at least in part on at least one of: an amount of material contained in the first object and a color of the material contained in the first object.

13. A system for initiating a response corresponding to at least one object being transported by an operator, wherein each of the at least one object is to be introduced into an instrument, the system comprising:

an image-capturing device that captures first image data of a first object of the at least one object being transported by the operator;

a database comprising stored pre-identified object data; and

a processing device, coupled to the image-capturing device and the database, configured to:

determine whether the operator is approaching the instrument;

determine whether the operator is transporting at least one object;

in response to determining that the operator is approaching the instrument and that the operator is carrying at least one object, compare the first image data to the stored pre-identified object data;

identify the first object as a first pre-identified object based at least in part on the comparison;

in response to identifying the first object as being an object to be received by the instrument, retrieve a predetermined first action corresponding to the first pre-identified object;

monitoring a status of consumable materials used by the instrument;

determining a sequence of the predetermined first action and at least one additional action to be implemented on the instrument based on the status of the consumable materials; and

automatically implement the predetermined first action and the at least one additional action on the instrument in accordance with the sequence.

14. The system as recited in claim 13 , wherein the image-capturing device comprises at least one of:

a motion-capturing device configured to capture motion data; and

a depth-capturing device configured to capture depth data.

15. The system as recited in claim 14 , wherein the processing device is further configured to, when comparing said first image data to the stored pre-identified object data, compare at least one of the captured motion data and captured depth data to the stored pre-identified object data.

16. The system as recited in claim 13 , wherein the processing device is further configured to:

receive second image data of a second object of the at least one object being transported by the operator;

compare said second image data to the stored pre-identified object data;

identify the second object based at least in part on said comparison;

in response to identifying the second object as being another object to be received by the instrument, retrieve a predetermined second action corresponding to the identified second object,

wherein determining the sequence of the predetermined first action and the at least one additional action includes determine a sequence of the predetermined first and second actions based on pre-established respective priorities of the predetermined first and second actions and based on the status of the consumable materials.

17. The system as recited in claim 16 , wherein determining the sequence comprises:

prioritizing the first and second predetermined actions as a function of relative urgencies of a respective replacement status of the first and second objects.

18. The system as recited in claim 13 , wherein the processing device is further configured to:

set a comparison value for each comparison of the first image data to the stored pre-identified object data;

compare the comparison value to a match threshold value; and

identify the first object if the comparison value meets or exceeds the match threshold value.

19. The system as recited in claim 13 , wherein the first object is chosen from:

a tray, a puck or a rack containing one or more specimen tubes or containment vessels; and

a reagent container.

20. The system as recited in claim 13 , wherein the predetermined first action comprises one of:

a) opening one of: a drawer, a door or a cover; or

b) providing access to a loading area.

21. The system as recited in claim 13 , wherein the instrument is a clinical diagnostic laboratory instrument.

22. A system for initiating a response corresponding to at least one object being transported by an operator, the system comprising:

an image-capturing device that captures first image data of a first object of the at least one object being transported by the operator;

a database comprising stored pre-identified object data; and

a processing device, coupled to the image-capturing device and the database, configured to:

determine whether the operator is approaching an instrument;

determine whether the operator is transporting at least one object;

in response to determining that the operator is approaching the instrument and that the operator is carrying at least one object, compare the first image data to the stored pre-identified object data;

identify the first object as a first pre-identified object based at least in part on the comparison;

retrieve a predetermined first action corresponding to the first pre-identified object;

receive second image data of a second object of the at least one object being transported by the operator;

compare said second image data to the stored pre-identified object data;

identify the second object based at least in part on said comparison;

retrieve a predetermined second action corresponding to the identified second object;

determine a sequence of the predetermined first and second actions based on respective priorities; and

implement the predetermined first and second actions in the determined sequence on the instrument,

wherein the processing device is further configured to monitor a status of consumable materials used by the instrument; and

wherein, in response to the first object being a replacement for the consumable materials, determining the sequence of the predetermined first and second actions is further based on the status of the consumable materials used by the instrument.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 10, 2015
From: GELBMAN, ALEXANDER
To: SIEMENS HEALTHCARE DIAGNOSTICS INC.
Reel/Frame 035819/0189 →
Continuity (2)
Provisional Application 61704584 · Sep 24, 2012
Related Publication 20150269415A1 · Sep 24, 2015