IP Library Granted Patent US 10,699,162
Granted Patent B2
US 10,699,162 · App. 16/048,446 · Granted Jun 30, 2020

Method and system for sorting and identifying medication via its label and/or package

Inventors: Sheng-Luen Chung (Taipei, TW); Chih-Fang Chen (Taipei, TW); Jing-Syuan Wang (Taipei, TW)
Assignees: MACKAY MEMORIAL HOSPITAL; NATIONAL TAIWAN UNIVERSITY OF SCIENCE AND TECHNOLOGY
G06K9/6255G06K9/3208G06K9/3216A61J1/035G06T5/002G06T7/13G06T7/74
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,699,162
App. No.
16/048,446
Granted
Jun 30, 2020
Kind
B2
Abstract

Disclosed herein is an improved pharmaceutical management system and methods implemented by the system for sorting and identifying a medicine via its label and/or package. The method comprises steps of: (1) receiving a plurality of raw images of a package of a medication; (b) juxtaposing two of the plurality of raw images to produce a combined image, in which the two raw images are different from each other; (c) processing the combined image to produce a reference image; and (d) establishing the medication library with the aid of the reference image. The system comprises an image capturing device, an image processor, and a machine learning processor. The image processor is programmed with instructions to execute the method for producing a combined image.

Claims (44)

1. A computer implemented method for building a medication library, comprising,

(a) receiving a plurality of raw images of a blister package of a medication;

(b) juxtaposing two of the plurality of raw images to produce a juxtaposed image, in which the two raw images are different from each other;

(c) processing the juxtaposed image to produce a reference image; and

(d) establishing the medication library with the aid of the reference image,

wherein the step (b) comprises,

(b-1) respectively processing the plurality of raw images to produce a plurality of first processed images respectively having defined contours,

(b-2) identifying the corners of each defined contours of the first processed images to determine the coordinates thereof;

(b-3) rotating each of the first processed images of the step (b-1) based on the determined coordinates of the step (b-2) to produce a plurality of second processed images; and

(b-4) juxtaposing any two of the second processed images to produce the juxtaposed image of the step (b); and

wherein the juxtaposed image produced in the step (b) fits a fixed template.

2. The computer implemented method of claim 1 , further comprising capturing the plurality of raw images of the blister package of the medication simultaneously prior to the step (a).

3. The computer implemented method of claim 1 , wherein each of the plurality of raw images of step (b-1) are subjected to treatments of, (i) a grayscale-converting treatment, (ii) a noise-reduction treatment, (iii) an edge-identification treatment, (iv) a convex hull treatment, and (v) a contouring treatment.

4. The computer implemented method of claim 1 , wherein the step (b-2) is carried out by a line transforming algorithm or a centroid algorithm.

5. The computer implemented method of claim 1 , wherein the juxtaposed image comprises the image of both sides of the blister package of the medication.

6. The computer implemented method of claim 1 , wherein the step (c) is carried out by a machine learning algorithm.

7. A computer implemented method for identifying a medication via its blister package, comprising,

(a) obtaining the front and back images of the blister package of the medication simultaneously;

(b) juxtaposing the front and back images of the step (a) to produce a candidate image that fits the fixed template;

(c) comparing the candidate image with a reference image of a medication library established by the method of claim 1 ; and

(d) outputting the result of the step (c).

8. The computer implemented method of claim 7 , wherein the step comprises,

(b-1) respectively processing the front and back images of the step (a) to produce two first processed images respectively having defined contours;

(b-2) identifying the corners of each defined contours of the two first processed images to determine the coordinates thereof;

(b-3) rotating each of the two first processed images of the step (b-1) based on the determined coordinates of the step (b-2) to produce two second processed images; and

(b-4) combining the two second processed images of the step (b-3) to produce the candidate image of the step (b).

9. The computer implemented method of claim 8 , wherein the front and back images of the step (b-1) are respectively subjected to treatments of, (i) a grayscale-converting treatment, (ii) a noise-reduction treatment, (iii) an edge-identification treatment, (iv) a convex hull treatment, and (v) a contouring treatment.

10. The computer implemented method of claim 8 , wherein the step (b-2) is carried out by a line transforming algorithm or a centroid algorithm.

11. The computer implemented method of claim 7 , wherein the step (c) is carried out by a machine learning algorithm.

12. The computer implemented method of claim 7 , further comprising transferring the candidate image into the medication library prior to the step (c).

13. A pharmaceutical management system, comprising,

an image capturing device configured to capture a plurality of images of a package of a medication;

an image processor programmed with instructions to execute a method for producing a candidate image, wherein the method comprises,

(1) respectively processing the plurality of images of the package of the medication to produce a plurality of first processed images respectively having defined contours;

(2) identifying the corners of each defined contours of the first processed images to determine the coordinates thereof;

(3) rotating each of the first processed images based on the identified coordinates of the step (2) to produce a plurality of second processed images; and

(4) juxtaposing two of the second processed images of the step (3) to produce the candidate image, in which the two second processed images are different from each other; and

a machine learning processor programmed with instructions to execute a method for comparing the candidate image with a reference image of a medication library established by the method of claim 1 .

14. The system of claim 13 , wherein the image capturing device comprises,

a transparent board on which the medication is placed; and

two image-capturing units individually disposed above each side of the transparent board.

15. The system of claim 13 , wherein each of the plurality of images of step (1) are subjected to treatments of, (i) a grayscale-converting treatment, (ii) a noise-reduction treatment, (iii) an edge-identification treatment, (iv) a convex hull treatment, and (v) a contouring treatment.

16. The system of claim 13 , wherein the step (2) is carried out by a line transforming algorithm or a centroid algorithm.

17. The system of claim 13 , wherein the method for comparing the candidate image with a reference image of a medication library is carried out by a machine learning algorithm.

Assignments (2)
NUNC PRO TUNC ASSIGNMENT Recorded Apr 12, 2023
From: MACKAY MEMORIAL HOSPITAL
To: NATIONAL TAIWAN UNIVERSITY OF SCIENCE AND TECHNOLOGY
Reel/Frame 063294/0404 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Aug 10, 2018
From: CHUNG, SHENG-LUEN; CHEN, CHIH-FANG; WANG, JING-SYUAN
To: MACKAY MEMORIAL HOSPITAL; NATIONAL TAIWAN UNIVERSITY OF SCIENCE AND TECHNOLOGY
Reel/Frame 046609/0570 →
Continuity (1)
Related Publication 20200034658A1 · Jan 30, 2020