IP Library Granted Patent US 11,817,207
Granted Patent B1
US 11,817,207 · App. 17/528,797 · Granted Nov 14, 2023

Medication inventory system including image based boundary determination for generating a medication tray stocking list and related methods

Inventors: James W. McCracken, Jr. (Lewisville, NC); Gregory J. Brendel (West Mifflin, PA); Patrick S. Connelly (Carnegie, PA); Marko Milojevic (Jamestown, NC); Jared O. Santibanez (Forney, TX); Juleo Amosah (Winston-Salem, NC); Justin A. Krull (Weirton, WV)
Assignee: INMAR RX SOLUTIONS, INC.
G16H40/20A61J7/0069G06T7/13G16H30/00G16H40/67G06T2207/10024
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,817,207
App. No.
17/528,797
Granted
Nov 14, 2023
Kind
B1
Abstract

A medication tray may include a boundary wall defining a boundary outline. Markers may be carried by the boundary wall. A mobile device may obtain a first image of the medication tray, and determine a number of the markers from the first image. When the number equals or exceeds a threshold, an edge detection algorithm may be applied to the first image to identify the boundary outline based upon the markers, and generate a stocking list of the tray based upon the outline. When the determined number does not exceed the threshold, the device may obtain a second image of the tray, and determine an updated number of the markers from the first and second images. When the updated number equals or exceeds the threshold, the algorithm may be applied to the images to identify the outline based upon the markers, and generate the stocking list based upon the outline.

Claims (84)

1. A medication inventory system comprising:

a medication tray comprising a plurality of compartments for storing respective medications, the medication tray comprising a boundary wall defining a boundary outline of the medication tray, and the medication tray having a tray identifier associated therewith for identifying a desired medication stocking list of desired medications within the medication tray;

a plurality of boundary markers carried by the boundary wall of the medication tray; and

a mobile wireless communications device configured to

obtain a first image of the medication tray,

determine a number of the plurality of boundary markers from the first image, and when the determined number equals or exceeds a threshold number

apply an edge detection algorithm to the first image to identify the boundary outline of the medication tray based upon the plurality of boundary markers, and

generate a current medication stocking list of the medication tray based upon the identified boundary outline and the tray identifier from the first image, and

when the determined number does not exceed the threshold number,

obtain at least one second image of the medication tray,

determine an updated number of the plurality of boundary markers from the first and the at least one second images by combining the first and at least one second images based upon determining image pixel locations in each of the first and at least one second images, mapping the pixel locations of the at least one second image to a pixel coordinate space of the first image, and mapping the pixel locations of each of the first and at least one second images to a pixel coordinate space representative of the medication tray, and when the updated determined number equals or exceeds the threshold number

apply the edge detection algorithm to the combined first and at least one second images to identify the boundary outline of the medication tray based upon the plurality of boundary markers, and

generate the current medication stocking list of the medication tray based upon the identified boundary outline and the tray identifier from the first and at least one second images.

2. The medication inventory system of claim 1 wherein the mobile wireless communication device is configured to, when the determined number does not exceed the threshold number, determine whether the at least one second image is a same image of the first image based upon pixel distances, and when so, discard the at least one second image.

3. The medication inventory system of claim 2 wherein the first and at least one second images each comprise respective first and at least one second pixel sets; and

wherein the mobile wireless communications device is configured to determine whether the at least one second image is the same as the first image based upon alignment of the first and at least one second pixel sets.

4. The medication inventory system of claim 1 wherein each boundary marker comprises a body having a slot therein to be slidably positioned on the boundary wall.

5. The medication inventory system of claim 1 wherein each boundary marker has machine readable indicia on an upper surface thereof.

6. The medication inventory system of claim 1 wherein each boundary marker has a color different than the boundary wall.

7. The medication inventory system of claim 1 wherein the mobile wireless communications device is configured to:

extract red, green, blue (RGB) pixel data from the at least one image;

convert the RGB pixel data to hue saturation value (HSV) space;

segment the at least one image into colored segments based upon the HSV space; and

apply the edge detection algorithm to the colored segments to identify the boundary outline of the medication tray.

8. The medication inventory system of claim 1 wherein the mobile wireless communications device is configured to apply contour tracing to identify the boundary outline of the medication tray.

9. The medication inventory system of claim 1 wherein the mobile wireless communications device is configured to identify corners of the boundary outline of the medication tray based upon the edge detection algorithm.

10. The medication inventory system of claim 1 wherein the mobile wireless communications device is configured to determine the desired medication stocking list for the medication tray based upon the tray identifier.

11. The medication inventory system of claim 10 wherein the mobile wireless communications device is configured to determine at least one missing medication based upon the current medication stocking list and the desired medication stocking list.

12. A mobile wireless communications device for a medication inventory system comprising a medication tray comprising a plurality of compartments for storing respective medications, the medication tray comprising a boundary wall defining a boundary outline of the medication tray, and the medication tray having a tray identifier associated therewith for identifying a desired medication stocking list of desired medications within the medication tray, the mobile wireless communications device comprising:

a controller and an associated memory configured to

obtain a first image of the medication tray,

determine a number of a plurality of boundary markers carried by the boundary wall of the medication tray from the first image, and when the determined number equals or exceeds a threshold number

apply an edge detection algorithm to the first image to identify the boundary outline of the medication tray based upon a plurality of boundary markers, and

generate a current medication stocking list of the medication tray based upon the identified boundary outline and the tray identifier from the first image, and

when the determined number does not exceed the threshold number,

obtain at least one second image of the medication tray,

determine an updated number of the plurality of boundary markers from the first and at least one second images by combining the first and at least one second images based upon determining image pixel locations in each of the first and at least one second images, mapping the pixel locations of the at least one second image to a pixel coordinate space of the first image, and mapping the pixel locations of each of the first and at least one second images to a pixel coordinate space representative of the medication tray, and when the updated determined number equals or exceeds the threshold number

apply the edge detection algorithm to the combined first and at least one second images to identify the boundary outline of the medication tray based upon the plurality of boundary markers, and

generate the current medication stocking list of the medication tray based upon the identified boundary outline and the tray identifier from the first and at least one second images.

13. The mobile wireless communications device of claim 12 wherein the controller is configured to, when the determined number does not exceed the threshold number, determine whether the at least one second image is a same image of the first image based upon pixel distances, and when so, discard the at least one second image.

14. The mobile wireless communications device of claim 13 wherein the first and at least one second images each comprise respective first and at least one second pixel sets; and wherein the mobile wireless communications device is configured to determine whether the at least one second image is the same as the first image based upon alignment of the first and at least one second pixel sets.

15. The mobile wireless communications device of claim 12 wherein the controller is configured to:

extract red, green, blue (RGB) pixel data from the at least one image;

convert the RGB pixel data to hue saturation value (HSV) space;

segment the at least one image into colored segments based upon the HSV space; and

apply the edge detection algorithm to the colored segments to identify the boundary outline of the medication tray.

16. The mobile wireless communications device of claim 12 wherein the controller is configured to apply contour tracing to identify the boundary outline of the medication tray.

17. The mobile wireless communications device of claim 12 wherein the controller is configured to identify corners of the boundary outline of the medication tray based upon the edge detection algorithm.

18. A method of processing medication inventory in a medication inventory system comprising a medication tray comprising a plurality of compartments for storing respective medications, the medication tray comprising a boundary wall defining a boundary outline of the medication tray, and the medication tray having a tray identifier associated therewith for identifying a desired medication stocking list of desired medications within the medication tray, the method comprising:

using a mobile wireless communications device to

obtain a first image of the medication tray,

determine a number of a plurality of boundary markers carried by the boundary wall of the medication tray from the first image, and when the determined number equals or exceeds a threshold number

apply an edge detection algorithm to the first image to identify the boundary outline of the medication tray based upon a plurality of boundary markers, and

generate a current medication stocking list of the medication tray based upon the identified boundary outline and the tray identifier from the first image, and

when the determined number does not exceed the threshold number,

obtain at least one second image of the medication tray,

determine an updated number of the plurality of boundary markers from the first and at least one second images by combining the first and at least one second images based upon determining image pixel locations in each of the first and at least one second images, mapping the pixel locations of the at least one second image to a pixel coordinate space of the first image, and mapping the pixel locations of each of the first and at least one second images to a pixel coordinate space representative of the medication tray, and when the updated determined number equals or exceeds the threshold number

apply the edge detection algorithm to the combined first and at least one second images to identify the boundary outline of the medication tray based upon the plurality of boundary markers, and

generate the current medication stocking list of the medication tray based upon the identified boundary outline and the tray identifier from the first and at least one second images.

19. The method of claim 18 wherein using the mobile wireless communications device comprises using the mobile wireless communication device to, when the determined number does not exceed the threshold number, determine whether the at least one second image is a same image of the first image based upon pixel distances, and when so, discard the at least one second image.

20. The method of claim 19 wherein the first and at least one second images each comprise respective first and at least one second pixel sets; and wherein using the mobile wireless communications device comprises using the mobile wireless communication device to determine whether the at least one second image is the same as the first image based upon alignment of the first and at least one second pixel sets.

21. The method of claim 18 wherein using the mobile wireless communications device comprises using the mobile wireless communications device to:

extract red, green, blue (RGB) pixel data from the at least one image;

convert the RGB pixel data to hue saturation value (HSV) space;

segment the at least one image into colored segments based upon the HSV space; and

apply the edge detection algorithm to the colored segments to identify the boundary outline of the medication tray.

22. The method of claim 18 wherein using the mobile wireless communications device comprises using the mobile wireless communications device to apply contour tracing to identify the boundary outline of the medication tray.

23. A non-transitory computer readable medium for a medication inventory system comprising a medication tray comprising a plurality of compartments for storing respective medications, the medication tray comprising a boundary wall defining a boundary outline of the medication tray, and the medication tray having a tray identifier associated therewith for identifying a desired medication stocking list of desired medications within the medication tray, the non-transitory computer readable medium comprising computer executable instructions that when executed by a controller of a mobile wireless communications device cause the controller to perform operations comprising:

obtaining a first image of the medication tray; and

determining a number of a plurality of boundary markers carried by the boundary wall of the medication tray from the first image, and when the determined number equals or exceeds a threshold number

applying an edge detection algorithm to the first image to identify the boundary outline of the medication tray based upon a plurality of boundary markers, and

generating a current medication stocking list of the medication tray based upon the identified boundary outline and the tray identifier from the first image; and

when the determined number does not exceed the threshold number,

obtaining at least one second image of the medication tray,

determining an updated number of the plurality of boundary markers from the first and at least one second image by combining the first and at least one second images based upon determining image pixel locations in each of the first and at least one second images, mapping the pixel locations of the at least one second image to a pixel coordinate space of the first image, and mapping the pixel locations of each of the first and at least one second images to a pixel coordinate space representative of the medication tray, and when the updated determined number equals or exceeds the threshold number

applying the edge detection algorithm to the combined first and at least one second images to identify the boundary outline of the medication tray based upon the plurality of boundary markers, and

generating the current medication stocking list of the medication tray based upon the identified boundary outline and the tray identifier from the first and at least one second images.

24. The non-transitory computer readable medium of claim 23 wherein the operations comprise, when the determined number does not exceed the threshold number, determining whether the at least one second image is a same image of the first image based upon pixel distances, and when so, discard the at least one second image.

25. The non-transitory computer readable medium of claim 24 wherein the first and at least one second images each comprise respective first and at least one second pixel sets; and wherein the operations comprise determining whether the at least one second image is the same as the first image based upon alignment of the first and at least one second pixel sets.

26. The non-transitory computer readable medium of claim 23 wherein the operations comprise:

extracting red, green, blue (RGB) pixel data from the at least one image;

converting the RGB pixel data to hue saturation value (HSV) space;

segmenting the at least one image into colored segments based upon the HSV space; and

applying the edge detection algorithm to the colored segments to identify the boundary outline of the medication tray.

Assignments (2)
SECURITY INTEREST Recorded Jun 28, 2023
From: INMAR, INC.; INMAR SUPPLY CHAIN SOLUTIONS, LLC; AKI TECHNOLOGIES, INC.; INMAR ANALYTICS, INC.; INMAR BRAND SOLUTIONS, INC.; INMAR CLEARING, INC.; INMAR RX SOLUTIONS, INC.; INMAR - YOUTECH, LLC; QUALANEX, LLC; CAROLINA COUPON CLEARING, INC. (N/K/A INMAR CLEARING, INC.); COLLECTIVE BIAS, INC. (N/K/A INMAR BRAND SOLUTIONS, INC.); MED-TURN, INC. (N/K/A INMAR RX SOLUTIONS, INC.)
To: JEFFERIES FINANCE LLC
Reel/Frame 064148/0903 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Nov 18, 2021
From: MCCRACKEN, JR., JAMES W.; MILOJEVIC, MARKO; CONNELLY, PATRICK S.; AMOSAH, JULEO; SANTIBANEZ, JARED O.; BRENDEL, GREG J.; KRULL, JUSTIN A.
To: INMAR RX SOLUTIONS, INC.
Reel/Frame 058153/0296 →