Method and apparatus for recognition of patient activity

1. A method for training a medication administration monitoring apparatus, the method comprising:

defining one or more predetermined medications;

acquiring one or more images of at least one user administering medication;

training a first deep neural network utilizing one or more learning processes to recognize a face portion of the at least one user from the one or more images;

training a second deep neural network utilizing one or more learning processes to recognize a mouth portion of the at least one user from the one or more images based upon at least an output generated from the training of the first deep neural network;

training a third deep neural network utilizing one or more learning processes to recognize an open mouth of the at least one user from the one or more images based upon at least an output generated from the training of the second deep neural network; and

training a fourth deep neural network utilizing one or more learning processes to detect a raised tongue of the at least one user from the one or more images based upon at least upon an output generated from the training of the third deep neural network,

wherein use of output from the first, second, third, and fourth deep neural networks allows for a confirmation of proper medication administration.

2. The method of claim 1 , wherein the training of each network is performed in accordance with a supervised learning process.

3. The method of claim 1 , wherein the training of each network is performed in accordance with an unsupervised learning process.

4. The method of claim 1 , wherein the training of a plurality of the networks are performed in sequence, employing an image processing system including the plurality of the networks a single time.

5. The method of claim 1 , comprising, upon use of the networks to process images, utilizing one or more of the processed images to further train one or more of the networks.

6. The method of claim 1 , further comprising the steps of:

receiving audio data associated with the medication administration; and

employing the received audio data to at least in part train one or more of the networks.

7. The method of claim 1 , further comprising the steps of:

receiving manually input data associated with the medication administration; and

employing the manually input data associated with the medication administration.

8. The method of claim 7 , wherein the manually input data is received from the at least one user.

9. The method of claim 7 , wherein the manually input data is system generated data.

10. The method of claim 1 , further comprising the step of confirming that the one or more predetermined medications cover a desired spectrum of possible medications.

11. The method of claim 1 , comprising:

training a fifth deep neural network utilizing one or more learning processes to recognize a medication pill on or under the tongue of the at least one user from the one or more images based upon at least an output generated from the training of the fourth deep neural network,

wherein use of output from the fifth deep neural network allows for the confirmation of proper medication administration.

12. One or more non-transitory computer readable storage media storing instructions that, when executed by one or more computers, cause the one or more computers to perform operations comprising:

defining one or more predetermined medications;

acquiring one or more images of at least one user administering medication;

training a first deep neural network utilizing one or more learning processes to recognize a face portion of the at least one user from the one or more images;

training a second deep neural network utilizing one or more learning processes to recognize a mouth portion of the at least one user from the one or more images based upon at least an output generated from the training of the first deep neural network;

training a third deep neural network utilizing one or more learning processes to recognize an open mouth of the at least one user from the one or more images based upon at least an output generated from the training of the second deep neural network; and

training a fourth deep neural network utilizing one or more learning processes to detect a raised tongue of the at least one user from the one or more images based upon at least upon an output generated from the training of the third deep neural network,

wherein use of output from the first, second, third, and fourth deep neural networks allows for a confirmation of proper medication administration.

13. The one or more non-transitory computer readable storage media of claim 12 , wherein the training of each network is performed in accordance with a supervised learning process or an unsupervised learning process.

14. The one or more non-transitory computer readable storage media of claim 12 , wherein the training of a plurality of the networks are performed in sequence, employing an image processing system including the plurality of the networks a single time.

15. The one or more non-transitory computer readable storage media of claim 12 , wherein the operations comprise, upon use of the networks to process images, utilizing one or more of the processed images to further train one or more of the networks.

16. The one or more non-transitory computer readable storage media of claim 12 , wherein the operations further comprise:

receiving audio data associated with the medication administration; and

employing the received audio data to at least in part train one or more of the networks.

17. The one or more non-transitory computer readable storage media of claim 12 , wherein the operations further comprise:

receiving manually input data associated with the medication administration; and

employing the manually input data associated with the medication administration.

18. The one or more non-transitory computer readable storage media of claim 12 , wherein the operations further comprise confirming that the one or more predetermined medications cover a desired spectrum of possible medications.

19. The one or more non-transitory computer readable storage media of claim 12 , wherein the operations further comprise:

training a fifth deep neural network utilizing one or more learning processes to recognize a medication pill on or under the tongue of the at least one user from the one or more images based upon at least an output generated from the training of the fourth deep neural network; and

training a sixth deep neural network utilizing one or more learning processes to recognize an identity of the medication pill from the one or more images based upon at least an output generated from the training of the fifth deep neural network;

wherein use of output from one or more of the fifth and sixth deep neural networks allows for the confirmation of proper medication administration.

20. A system comprising:

one or more computers; and

one or more non-transitory computer readable storage media storing instructions that, when executed by the one or more computers, cause the one or more computers to perform operations comprising:

defining one or more predetermined medications;

acquiring one or more images of at least one user administering medication;

training a first deep neural network utilizing one or more learning processes to recognize a face portion of the at least one user from the one or more images;

training a second deep neural network utilizing one or more learning processes to recognize a mouth portion of the at least one user from the one or more images based upon at least an output generated from the training of the first deep neural network;

training a third deep neural network utilizing one or more learning processes to recognize an open mouth of the at least one user from the one or more images based upon at least an output generated from the training of the second deep neural network; and

training a fourth deep neural network utilizing one or more learning processes to detect a raised tongue of the at least one user from the one or more images based upon at least upon an output generated from the training of the third deep neural network,

wherein use of output from the first, second, third, and fourth deep neural networks allows for a confirmation of proper medication administration.