Medication inventory system including bounding box image capture and related methods
A medication inventory system may include a medication tray that includes compartments for storing respective medications. The medication tray may include a boundary wall defining a boundary outline of the medication tray, and the medication tray may have a tray identifier associated therewith. The medication inventory system may also include a user device that includes a housing, a camera carried by the housing, a display carried by the housing, and a controller coupled to the camera and the display. The controller may generate on the display a bounding box overlay image on a live image from the camera of the medication tray, capture an image from the camera when the bounding box overlay image is aligned relative to the boundary outline of the medication tray. The controller may also generate a current medication stocking list of the medication tray based upon the captured image and the tray identifier.
1. A medication inventory system comprising:
a medication tray comprising a plurality of compartments for storing respective medications, the medication tray comprising a boundary wall defining a boundary outline of the medication tray, and the medication tray having a tray identifier associated therewith; and
a user device comprising a housing, a camera carried by the housing, a display carried by the housing, an orientation sensor carried by the housing, and a controller coupled to the camera and the display, the controller configured to
determine an orientation of the user device relative to the medication tray based upon the orientation sensor,
determine an amount of pixels of at least one of the medications and the tray identifier from a live image from the camera of the medication tray,
apply a transform to the live image of the medication tray based upon the orientation of the user device and the amount of pixels of the at least one of the medications and the tray identifier so that the live image displayed as if the camera were overhead the medication tray,
display the transformed live image on the display,
generate on the display a bounding box overlay image on the transformed live image from the camera of the medication tray based upon the orientation of the user device,
capture an image from the camera when the bounding box overlay image is aligned relative to the boundary outline of the medication tray, and
generate a current medication stocking list of the medication tray based upon the captured image and the tray identifier.
2. The medication inventory system of claim 1 wherein the controller is configured to change a shape of the bounding box overlay image based upon the orientation of the housing.
3. The medication inventory system of claim 1 wherein the orientation sensor comprises one of a gyroscope and an accelerometer.
4. The medication inventory system of claim 1 wherein the controller is configured to identify the boundary outline of the medication tray based upon the tray identifier and the captured image of the medication tray.
5. The medication inventory system of claim 4 comprising a plurality of boundary markers carried by the boundary wall; and wherein the controller is configured to apply an edge detection algorithm to the captured image to identify the boundary outline of the medication tray based upon the plurality of boundary markers.
6. The medication inventory system of claim 1 wherein the controller is configured to determine a desired medication stocking list for the medication tray based upon the tray identifier.
7. The medication inventory system of claim 6 wherein the controller is configured to determine at least one missing medication based upon the current medication stocking list and the desired medication stocking list.
8. A user device comprising:
a housing;
a camera carried by the housing;
a display carried by the housing;
an orientation sensor carried by the housing; and
a controller coupled to the camera, the display, and the orientation sensor, the controller configured to
determine an orientation of the user device relative to the medication tray based upon the orientation sensor,
determine an amount of pixels of at least one of the medications and the tray identifier from a live image from the camera of the medication tray,
apply a transform to the live image of the medication tray based upon the orientation of the user device and the amount of pixels of the at least one of the medications and the tray identifier so that the live image is displayed as if the camera were overhead the medication tray,
display the transformed live image on the display,
generate on the display a bounding box overlay image on the transformed live image from the camera of a medication tray based upon the orientation of the user device, the medication tray comprising a plurality of compartments for storing respective medications, and a boundary wall defining a boundary outline of the medication tray, and the medication tray having a tray identifier associated therewith,
capture an image from the camera when the bounding box overlay image is aligned relative to the boundary outline of the medication tray, and
generate a current medication stocking list of the medication tray based upon the captured image and the tray identifier.
9. The user device of claim 8 , wherein the controller is configured to change a shape of the bounding box overlay image based upon the orientation of the housing.
10. The user device of claim 8 wherein the orientation sensor comprises one of a gyroscope and an accelerometer.
11. The user device of claim 8 wherein the controller is configured to identify the boundary outline of the medication tray based upon the tray identifier and the captured image of the medication tray.
12. A method of processing medication inventory in a medication inventory system comprising a medication tray comprising a plurality of compartments for storing respective medications, the medication tray comprising a boundary wall defining a boundary outline of the medication tray, and the medication tray having a tray identifier associated therewith, the method comprising:
using a user device comprising a housing, a camera carried by the housing, a display carried by the housing, and an orientation sensor carried by the housing to
determine an orientation of the user device relative to the medication tray based upon the orientation sensor,
determine an amount of pixels of at least one of the medications and the tray identifier from a live image from the camera of the medication tray,
apply a transform to the live image of the medication tray based upon the orientation of the user device and the amount of pixels of the at least one of the medications and the tray identifier so that the live image is displayed as if the camera were overhead the medication tray,
display the transformed live image on the display,
generate on the display a bounding box overlay image on the transformed live image from the camera of the medication tray based upon the orientation of the user device,
capture an image from the camera when the bounding box overlay image is aligned relative to the boundary outline of the medication tray, and
generate a current medication stocking list of the medication tray based upon the captured image and the tray identifier.
13. The method of claim 12 wherein using the user device comprises using the user device to change a shape of the bounding box image based upon the orientation of the housing.
14. The method of claim 12 wherein using the user device comprises using the user device to identify the boundary outline of the medication tray based upon the tray identifier and the captured image of the medication tray.
15. A non-transitory computer readable medium for processing medication inventory, the non-transitory computer readable medium comprising computer executable instructions that when executed by a controller of a user device cause the controller to perform operations comprising:
determining an orientation of the user device relative to a medication tray having a tray identifier associated therewith based upon an orientation sensor of the user device;
determining an amount of pixels of at least one medication stored in the medication tray and the tray identifier from a live image from a camera of the user device of the medication tray;
applying a transform to the live image of the medication tray based upon the orientation of the user device and the amount of pixels of the at least one medication stored within the medication tray and the tray identifier so that the live image is displayed as if the camera were overhead the medication tray;
displaying the transformed live image on a display of the user device;
generating, on the display of the user device, a bounding box overlay image on the transformed live image of a medication tray based upon the orientation of the user device, the medication tray comprising a plurality of compartments for storing respective ones of the medications, and a boundary wall defining a boundary outline of the medication tray;
capturing an image from the camera of the user device when the bounding box overlay image is aligned relative to the boundary outline of the medication tray; and
generating a current medication stocking list of the medication tray based upon the captured image and the tray identifier.
16. The non-transitory computer readable medium of claim 15 wherein the operations comprise cooperating with an orientation sensor of the user device to determine an orientation of the user device and generate the bounding box overlay image based upon the orientation of the user device.
17. The non-transitory computer readable medium of claim 16 wherein the operations comprise changing a shape of the bounding box overlay image based upon the orientation of the housing.
18. The non-transitory computer readable medium of claim 15 wherein the operations comprise identifying boundary outline of the medication tray based upon the tray identifier and the captured image of the medication tray.