In-home patient-focused rehabilitation system
Described is a system for patient-specific rehabilitation that can be performed outside a clinic. The system monitors a patient in real-time to generate a quantitative assessment of a physical state and a motivational state of the patient using sensor data obtained from sensors. Predictions related to the patient are generated utilizing patient-specific biomechanical and neurocognitive models implemented with predictive simulations. Video feed of the patient is registered with sensor data and a set of simulation data. Rehabilitation guidance instructions are conveyed to the patient through dialog-based interactions.
1. A system for patient user-specific rehabilitation, the system comprising:
one or more processors and a non-transitory computer-readable medium having executable instructions encoded thereon such that when executed, the one or more processors perform operations of:
monitoring, using a Mobile Patient Monitor (MPM), sensor data obtained from a plurality of sensors in real-time to generate a quantitative assessment of a physical state and a motivational state of a user;
generating, using a Virtual Patient (VP) unit, predictions related to the user utilizing user-specific biomechanical and neurocognitive models implemented with predictive simulations;
generating rehabilitation guidance using a Virtual Coach (VC) unit through real-time communication with the VP unit;
registering, using an Augmented Reality (AR) unit, video feed of the user with sensor data from the MPM and a set of simulation data from the VP unit; and
conveying rehabilitation guidance instructions to the user through dialog-based user-VC interactions; and
a lightweight exoskeleton worn by the user configured to convey real-time rehabilitation guidance to the user.
2. The system as set forth in claim 1 , wherein the one or more processors further performs an operation of generating at least one biomechanical state prediction with the VP unit using a set of motion and force measurements obtained with the MPM.
3. The system as set forth in claim 1 , wherein the one or more processors further performs operations of:
processing the sensor data and a set of user profile data for use as simulation inputs to the VP unit;
generating model-based quantitative assessments of the user's physical state and motivational state using the simulation inputs;
storing the model-based quantitative assessments in a user data archive; and
performing simulations to reveal biomechanical and neurocognitive information related to the user.
4. The system as set forth in claim 1 , wherein the one or more processors further perform operations of:
abstracting a set of multi-dimensional time series vectors from the sensor data;
processing, by a Virtual Patient Cognitive Model (VPCM) of the VP unit, the set of multi-dimensional time series vectors;
generating a set of cognitive state estimates from the set of multi-dimensional time series vectors;
determining how the set of cognitive state estimates affect rehabilitation treatment outcomes through processing of the set of cognitive state estimates by the VC unit in order to choose a strategy for desired user results; and
storing a dynamic user profile, comprising the determined effect of the set of cognitive state estimates, in a rehabilitation knowledge base.
5. The system as set forth in claim 1 , wherein the one or more processors further performs operations of:
guiding the user, with the VC unit, through rehabilitation treatment by applying a set of rehabilitation knowledge to the user's specific needs through interaction with the VC unit;
generating candidate therapeutic actions using the set of rehabilitation knowledge; and
evaluating, with the user-specific biomechanical and neurocognitive models, likely outcomes if the candidate therapeutic actions are taken.
6. The system as set forth in claim 1 , wherein the one or more processors further perform operations of:
generating, with the VP unit, a plurality of simulation feeds related to the user,
wherein a first simulation feed represents a user reference avatar, mirroring the user's motion, and
wherein a second simulation feed represents a user goal avatar, representing motion that the VC unit is guiding the user toward.
7. A computer-implemented method for user-specific rehabilitation, comprising:
an act of causing one or more processors to execute instructions stored on a non-transitory memory such that upon execution, the one or more processors perform operations of:
monitoring, using a Mobile Patient Monitor (MPM), sensor data obtained from a plurality of sensors in real-time to generate a quantitative assessment of a physical state and a motivational state of a user;
generating, using a Virtual Patient (VP) unit, predictions related to the user utilizing user-specific biomechanical and neurocognitive models implemented with predictive simulations;
generating rehabilitation guidance using a Virtual Coach (VC) unit through real-time communication with the VP unit;
registering, using an Augmented Reality (AR) unit, video feed of the user with sensor data from the MPM and a set of simulation data from the VP unit;
conveying rehabilitation guidance instructions to the user through dialog-based user-VC interactions; and
conveying real-time rehabilitation guidance to the user via a lightweight exoskeleton worn by the user.
8. The method as set forth in claim 7 , wherein the one or more processors further performs an operation of generating at least one biomechanical state prediction with the VP unit using a set of motion and force measurements obtained with the MPM.
9. The method as set forth in claim 7 , wherein the one or more processors further performs operations of:
processing the sensor data and a set of user profile data for use as simulation inputs to the VP unit;
generating model-based quantitative assessments of the user's physical state and motivational state using the simulation inputs;
storing the model-based quantitative assessments in a user data archive; and
performing simulations to reveal biomechanical and neurocognitive information related to the user.
10. The method as set forth in claim 7 , wherein the one or more processors further perform operations of:
abstracting a set of multi-dimensional time series vectors from the sensor data;
processing, by a Virtual Patient Cognitive Model (VPCM) of the VP unit, the set of multi-dimensional time series vectors;
generating a set of cognitive state estimates from the set of multi-dimensional time series vectors;
determining how the set of cognitive state estimates affect rehabilitation treatment outcomes through processing of the set of cognitive state estimates by the VC unit in order to choose a strategy for desired user results; and
storing a dynamic user profile, comprising the determined effect of the set of cognitive state estimates, in a rehabilitation knowledge base.
11. The method as set forth in claim 7 , wherein the one or more processors further perform operations of:
guiding the user, with the VC unit, through rehabilitation treatment by applying a set of rehabilitation knowledge to the user's specific needs through interaction with the VC unit;
generating candidate therapeutic actions using the set of rehabilitation knowledge; and
evaluating, with the user-specific biomechanical and neurocognitive models, likely outcomes if the candidate therapeutic actions are taken.
12. The method as set forth in claim 7 , wherein the one or more processors further perform operations of:
generating, with the VP unit, a plurality of simulation feeds related to the user,
wherein a first simulation feed represents a user reference avatar, mirroring the user's motion, and
wherein a second simulation feed represents a patient user goal avatar, representing motion that the VC unit is guiding the user toward.
13. A computer program product for user-specific rehabilitation, the computer program product comprising:
computer-readable instructions stored on a non-transitory computer-readable medium that are executable by a computer having one or more processors for causing the processor to perform operations of:
monitoring, using a Mobile Patient Monitor (MPM), sensor data obtained from a plurality of sensors in real-time to generate a quantitative assessment of a physical state and a motivational state of a user;
generating, using a Virtual Patient (VP) unit, predictions related to the user utilizing user-specific biomechanical and neurocognitive models implemented with predictive simulations;
generating rehabilitation guidance using a Virtual Coach (VC) unit through real-time communication with the VP unit;
registering, using an Augmented Reality (AR) unit, video feed of the user with sensor data from the MPM and a set of simulation data from the VP unit; and
conveying rehabilitation guidance instructions to the user through dialog-based user-VC interactions; and
conveying real-time rehabilitation guidance to the user via a lightweight exoskeleton worn by the user.
14. The computer program product as set forth in claim 13 , further comprising instructions for causing the one or more processors to perform an operation of generating at least one biomechanical state prediction with the VP unit using a set of motion and force measurements obtained with the MPM.
15. The computer program product as set forth in claim 13 , further comprising instructions for causing the one or more processors to perform operations of:
processing the sensor data and a set of user profile data for use as simulation inputs to the VP unit;
generating model-based quantitative assessments of the user's physical state and motivational state using the simulation inputs;
storing the model-based quantitative assessments in a user data archive; and
performing simulations to reveal biomechanical and neurocognitive information related to the user.
16. The computer program product as set forth in claim 13 , further comprising instructions for causing the one or more processors to perform operations of:
abstracting a set of multi-dimensional time series vectors from the sensor data;
processing, by a Virtual Patient Cognitive Model (VPCM) of the VP unit, the set of multi-dimensional time series vectors;
generating a set of cognitive state estimates from the set of multi-dimensional time series vectors;
determining how the set of cognitive state estimates affect rehabilitation treatment outcomes through processing of the set of cognitive state estimates by the VC unit in order to choose a strategy for desired user results; and
storing a dynamic user profile, comprising the determined effect of the set of cognitive state estimates, in a rehabilitation knowledge base.
17. The computer program product as set forth in claim 13 , further comprising instructions for causing the one or more processors to perform operations of:
guiding the user, with the VC unit, through rehabilitation treatment by applying a set of rehabilitation knowledge to the user's specific needs through interaction with the VC unit;
generating candidate therapeutic actions using the set of rehabilitation knowledge; and
evaluating, with the user-specific biomechanical and neurocognitive models, likely outcomes if the candidate therapeutic actions are taken.