IP Library Granted Patent US 11,379,979
Granted Patent B2
US 11,379,979 · App. 16/998,358 · Granted Jul 5, 2022

Computer imaging pre-processing for automated medication dispensing analysis

Inventors: Christopher R. Markson (Hawthorne, NJ); Pritesh J. Shah (Paramus, NJ); Christopher G. Lehmuth (St. Louis, MO)
Assignee: Express Scripts Strategic Development, Inc.
G06T7/0012G06T7/12G06T7/13G06T7/70G06T11/001G16H20/13G16H30/40G06T2207/20076G06T2207/20081G06T2207/30242
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,379,979
App. No.
16/998,358
Granted
Jul 5, 2022
Kind
B2
Abstract

A computer system includes an input configured to receive a first image of medication located in a receptacle, memory, and a processor configured to execute instructions including creating a second image based on the first image, dividing pixels of the second image into first and second subsets, and scanning the second image along a first axis to count, for each point along the first axis, a number of pixels in the first subset along a line perpendicular to the first axis that intersects the first axis at the point. The instructions also include estimating positions of first and second edges of the receptacle along the first axis based on the counts of the pixels, defining an opening of the receptacle based on the estimated positions of the first and second edges, and outputting a processed image that indicates areas of the image that are outside of the defined opening.

Claims (67)

1. A computer system comprising:

an input configured to receive a first image of medication located in a receptacle captured by a camera;

memory configured to store computer-executable instructions; and

at least one processor configured to execute the computer-executable instructions, wherein the instructions include:

creating a second image in which each pixel of the second image is based on a corresponding pixel of the first image;

dividing pixels of the second image into a first subset and a second subset such that pixels of the first subset include candidate pixels corresponding to the receptacle;

scanning the second image along a first axis to count, for each point along the first axis, a number of pixels in the first subset along a line perpendicular to the first axis that intersects the first axis at the point;

defining an ellipse, by:

estimating a position of a first edge of the receptacle along the first axis based on the counts of the pixels at each point along the first axis; and

estimating a position of a second edge of the receptacle along the first axis based on the counts of the pixels at each point along the first axis;

defining an opening of the receptacle based on the ellipse, including the estimated positions of the first edge of the receptacle and the second edge of the receptacle;

outputting a processed image that indicates areas that are outside of the defined opening; and

identifying a likelihood of mis-dispensed medication based on the processed image.

2. The computer system of claim 1 , wherein the opening is defined as an ellipse.

3. The computer system of claim 1 , wherein:

the instructions include analyzing the medication based on the processed image; and

the areas of the processed image that are outside of the defined opening are excluded from the analysis.

4. The computer system of claim 1 , wherein the instructions include identifying the likelihood of mis-dispensed medication according to at least one of a color of the medication in the processed image, a shape of the medication in the processed image, a size of the medication in the processed image, and markings of the medication in the processed image.

5. The computer system of claim 4 , wherein the instructions include, in response to the identified likelihood of mis-dispensed medication being above a threshold:

performing a remedial action including at least one of notifying an operator,

requesting a manual inspection of the receptacle, and discarding the receptacle; and

updating quality statistics based on at least one of the identified likelihood of mis-dispensed medication and feedback from the manual inspection.

6. The computer system of claim 4 , wherein the instructions include, in response to the identified likelihood of mis-dispensed medication being below a threshold, capping and packaging the receptacle for distribution.

7. The computer system of claim 1 wherein creating the second image includes, for each pixel of the second image, generating the pixel by adjusting at least one value of the corresponding pixel of the first image.

8. The computer system of claim 7 , wherein creating the second image includes, for each pixel of the second image, generating the pixel by calculating a grayscale value of the corresponding pixel of the first image.

9. The computer system of claim 7 , wherein creating the second image includes, for each pixel of the second image, generating the pixel by:

setting a green value of the pixel to a sum of (i) a green value of the corresponding pixel of the first image and (ii) a blue value of the corresponding pixel of the first image minus (iii) a red value of the corresponding pixel of the first image;

setting a green value of the pixel to zero; and

setting a blue value of the pixel to zero.

10. The computer system of claim 1 , wherein:

estimating the position of the first edge of the receptacle includes determining a first local maximum of the counts of the pixels at each point along the first axis; and

estimating the position of the second edge of the receptacle includes determining a second local maximum of the counts of the pixels at each point along the first axis.

11. The computer system of claim 10 , wherein the instructions include, in response to the second local maximum being less than the first local maximum by more than a predetermined amount, estimating the position of the second edge of the receptacle to be an edge of the first image.

12. The computer system of claim 1 , wherein the instructions include:

scanning the second image along a second axis to count, for each point along the second axis, a number of pixels in the first subset along a line perpendicular to the second axis that intersects the second axis at the point;

estimating a position of a third edge of the receptacle along the second axis based on the counts of the pixels at each point along the second axis; and

estimating a position of a fourth edge of the receptacle along the second axis based on the counts of the pixels at each point along the second axis.

13. The computer system of claim 12 , wherein:

the first axis is parallel to a first edge of the first image; and

the second axis is perpendicular to the first axis.

14. A vision system comprising:

the computer system of claim 1 ; and

the camera.

15. A method of image pre-processing for automated medication dispensing, the method comprising:

receiving a first image of medication located in a receptacle captured by a camera;

creating a second image in which each pixel of the second image is based on a corresponding pixel of the first image;

dividing pixels of the second image into a first subset and a second subset such that pixels of the first subset include candidate pixels corresponding to the receptacle;

scanning the second image along a first axis to count, for each point along the first axis, a number of pixels in the first subset along a line perpendicular to the first axis that intersects the first axis at the point;

defining an ellipse, by:

estimating a position of a first edge of the receptacle along the first axis based on the counts of the pixels at each point along the first axis; and

estimating a position of a second edge of the receptacle along the first axis based on the counts of the pixels at each point along the first axis;

defining an opening of the receptacle based on the ellipse, including the estimated positions of the first edge of the receptacle and the second edge of the receptacle;

outputting a processed image that indicates areas that are outside of the defined opening; and

identifying a likelihood of mis-dispensed medication based on the processed image.

16. The method of claim 15 , further comprising identifying the likelihood of mis-dispensed medication according to at least one of a color of the medication in the processed image, a shape of the medication in the processed image, a size of the medication in the processed image and markings of the medication in the processed image.

17. The method of claim 16 , further comprising, in response to the identified likelihood of mis-dispensed medication being above a threshold:

performing a remedial action including at least one of notifying an operator,

requesting a manual inspection of the receptacle, and discarding the receptacle; and

updating quality statistics based on at least one of the identified likelihood of mis-dispensed medication and feedback from the manual inspection.

18. The method of claim 16 , further comprising, in response to the identified likelihood of mis-dispensed medication being below a threshold, capping and packaging the receptacle for distribution.

19. The method of claim 15 , wherein:

estimating the position of the first edge of the receptacle includes determining a first local maximum of the counts of the pixels at each point along the first axis; and

estimating the position of the second edge of the receptacle includes determining a second local maximum of the counts of the pixels at each point along the first axis.

20. The method of claim 15 , further comprising:

scanning the second image along a second axis to count, for each point along the second axis, a number of pixels in the first subset along a line perpendicular to the second axis that intersects the second axis at the point;

estimating a position of a third edge of the receptacle along the second axis based on the counts of the pixels at each point along the second axis; and

estimating a position of a fourth edge of the receptacle along the second axis based on the counts of the pixels at each point along the second axis.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Aug 20, 2020
From: MARKSON, CHRISTOPHER R.; SHAH, PRITESH J.; LEHMUTH, CHRISTOPHER G.
To: EXPRESS SCRIPTS STRATEGIC DEVELOPMENT, INC.
Reel/Frame 053552/0561 →
Continuity (3)
Continuation 16190548 · Nov 14, 2018
Provisional Application 62590255 · Nov 22, 2017
Related Publication 20210004958A1 · Jan 7, 2021
Cited By (1)
US 12,728,074