Method and apparatus for pattern tracking
A method and apparatus for pattern tracking. The method includes the steps of performing a foreground detection process to determine a hand-pill-hand region, performing image segmentation to separate the determined hand portion of the hand-pill-hand region from the pill portion thereof, building three reference models, one for each hand region and one for the pill region, initializing a dynamic model for tracking the hand-pill-hand region, determining N possible next positions for the hand-pill-hand region, for each such determined position, determining various features, building a new model for that region in accordance with the determined position, for each position, comparing the new model and a reference model, determining a position whose new model generates a highest similarity score, determining whether that similarity score is greater than a predetermined threshold, and wherein if it is determined that the similarity score is greater than the predetermined threshold, the object is tracked.
1. A method for confirming ingestion of a medication pill by a patient, comprising:
displaying to a patient on a display one or more instructions encouraging proper administration of a medication pill;
receiving one or more images by a camera associated with a computing apparatus of a patient ingesting a medication pill that is visible to the camera in response to the one or more instructions;
performing a foreground detection process on the one or more images to determine a hand-pill-hand region by the computing apparatus;
performing image segmentation on the hand-pill-hand region to separate the determined hand portion of the hand-pill-hand region from the pill portion thereof by the computing apparatus;
displaying to the patient, in response to a difficulty in performing image segmentation, and in response to the output of a proximity detector associated with the computing device, one or more instructions to bring the medication pill closer to the camera;
confirming the identity of the pill portion by comparing one or more characteristics of the pill portion to one or more corresponding characteristics of one or more reference images stored to a recording medium associated with the computing apparatus, thereby confirming an expected pill in the hand of the patient;
performing a further foreground detection process on the one or more images to determine a pill positioned in a patient's mouth by recognizing the pill portion surrounded by an unbroken ring of color corresponding to the inner portion of the mouth of the patient by the computing apparatus;
wherein upon confirmation of the expected pill in the hand of the patient and the expected pill in the mouth of the patient, medication administration is confirmed; and
displaying on a display associated with the computing apparatus an indication of the confirmation of the medication administration.
2. The method of claim 1 , wherein the determination of the pill in the mouth of the patient is performed after determination of the pill in the hand of the patient.
3. The method of claim 1 , wherein the one or more characteristics may be selected from the group of: shape, color, texture, gray-scale intensity and histogram of colors.
4. A method for confirming ingestion of a medication pill by a patient, comprising:
displaying to a patient on a display one or more instructions encouraging proper administration of a medication pill;
receiving one or more images by a camera associated with a computing apparatus of a patient ingesting a medication pill that is visible to the camera in response to the one or more instructions;
performing a foreground detection process of an image or series of images of the one or more received images to determine a mouth-pill-mouth region by the computing apparatus;
performing image segmentation of the image or series of images to separate the determined mouth portion of the mouth-pill-mouth region from the pill portion thereof by the computing apparatus,
wherein performing image segmentation comprises performing a further foreground detection process of the image or series of images to determine a pill positioned in a patient's mouth by recognizing the rill portion surrounded by an unbroken ring of color corresponding to an inner portion of the mouth of the patient by the computing apparatus;
displaying to the patient, in response to a difficulty in performing image segmentation, and in response to the output of a proximity detector associated with the computing device, one or more instructions to bring the medication pill closer to the camera;
confirming the identity of the pill portion in the mouth of the patient by comparing one or more characteristics of the pill portion to one or more corresponding characteristics of one or more reference images, thereby confirming ingestion of the pill portion by the patient by the computing apparatus; and
displaying on a display associated with the computing apparatus an indication of the confirmation of the medication administration.
5. The method of claim 4 , further comprising:
building one or more reference models including at least the mouth region and the pill region;
determining a feature vector of the mouth-pill-mouth region;
comparing the determined feature vector to one or more reference feature vectors;
determining a distance between the determined feature vector and each of the one or more reference feature vectors;
determining an image or set of images corresponding to the determined feature vector to be similar to an image or set of images whose corresponding feature vector when the determined distance is less than a predetermined threshold;
fitting each determined distance into a Gaussian distribution to determine a confidence probability of a match between the two corresponding images or sets of images, thereby confirming the identity of the pill in the mouth of the patient.
6. The method of claim 4 , further comprising storing a color signature of the mouth-pill-mouth portion of the one or more of the acquired images as the indication of the one or more characteristics.
7. The method of claim 6 , further comprising determining the location of the mouth-pill-mouth portion in a next one or more of the acquired images in accordance with the stored color signature.
8. The method of claim 6 , further comprising distinguishing the pill even though it is colored similarly to a background in accordance with the stored color signature.
9. The method of claim 8 , wherein the background is a portion of the mouth of the user.
10. The method of claim 8 , wherein the background is an environmental surface.
11. The method of claim 4 , further comprising:
building one or more reference models including at least one of the hand region and the pill region.
12. The method of claim 11 , further comprising determining a feature vector difference between at least one of the one or more reference models and a mouth-pill-mouth portion of one or more of the one or more acquired images.
13. The method of claim 12 , wherein if the vector difference is less than a predetermined threshold, there is determined to be a match.
14. The method of claim 11 , wherein the foreground detection process takes into account one or more environmental factors.
15. The method of claim 14 , wherein the one or more environmental factors comprises ambient light.
16. The method of claim 4 , wherein the one or more characteristics comprises shape.
17. The method of claim 4 , wherein the one or more characteristics comprises color.
18. The method of claim 4 , wherein the series of images comprises a plurality of consecutive acquired images.
19. The method of claim 1 , wherein the proximity detector is the camera.
20. The method of claim 4 , wherein the proximity detector is the camera.