Virtual trainer optimizer method and system
View Patent ↗This disclosure provides an augmented Virtual Trainer (VT) method and system. According to an exemplary system, a video based physiological metric system is integrated with a VT system to provide health and/or safety related data associated with a user of the VT system. According to an exemplary embodiment, the disclosed augmented VT system modifies an exercise routine based on the physiological metrics and/or provides audio signals to the user.
1. A computer implemented virtual training (VT) method comprising:
a VT system electronically communicating via one or both of video and audio instructions for performing an exercise routine to a user of the VT system, the exercise routine to be performed by the user;
a sensing system sensing exercise motion of the user using one or more sensing elements, the sensing elements including an image capturing device acquiring an image of the user the image capturing system including a tandem depth sensing range camera and a RGB color camera configured to generate a RGBD (Red-Green-Blue-Depth) image of the user;
an exertion level inspection system processing the image of the user captured by the image capturing device to determine, by an application of animation techniques, an exertion level of the user during performance of the exercise routine, and based on the determined exertion level of the user, the exertion level inspection system performing one of the following optimizations:
a) visually and/or audibly communicating to the user one or more instructions which increase the user's exertion level;
b) visually and/or audibly communicating to the user one or more instructions which decrease the user's exertion level; or
c) visually and/or audibly communicating to the user one or more instructions which maintain the user's exertion levels,
wherein the method optimizes the user's exertion level by dynamically modifying, the instructions for performing the exercise routine to achieve a determined optimum exertion level of the user.
2. The computer implemented VT method according to claim 1 , comprising:
comparing the sensed exercise motion of the user performing the exercise to a motion associated with the instructions for performing the exercise.
3. The computer implemented VT method according to claim 1 , wherein the exertion level of the user is determined using a video-based method to detect one or more physiological metrics associated with the user.
4. The computer implemented VT method according to claim 3 , wherein the physiological metrics include one or more cardio-respiratory metrics, respiration rate, heart rate, body temperature and blood pressure.
5. The computer implemented VT method according to claim 1 , wherein the method performs two or more of the following:
performing video-based cardio-respiratory monitoring of the user;
performing video-based fall and lack of action monitoring of the user; and
performing video-based appearance anomaly monitoring of the user.
6. The computer implemented VT method according to claim 1 ,
wherein the instructions for performing the exercise routine are dynamically modified based on the determined exertion level of the user.
7. The computer implemented VT method according to claim 1 , wherein the VT method assesses health data related to the user and uses the health data to determine an appropriate exertion level, and/or the VT method uses an anticipated view of the user, obtained from the prescribed exercise routines, to facilitate measurement of one or more cardio-respiratory metrics associated with the user.
8. The computer implemented VT method according to claim 7 , wherein the anticipated view of the user includes any of:
an expected absolute or relative location, pose, and/or velocity of the face and/or other body parts;
and the facilitation includes any of:
improving tracking or localization of the regions of interest of the face and/or other body parts; compensating for the pose changes or velocity of the face and/or other body parts; determining the timing for measuring one or more cardio-respiratory metrics associated with the user; and determining the confidence for the measurement of one or more cardio-respiratory metrics associated with the user.
9. The computer implemented VT method according to claim 1 , wherein the image capturing device captures RGB (Red-Green-Blue) data and/or D (Depth) data.
10. The computer implemented VT method according to claim 1 , wherein the method comprises performing video-based fall and lack of action monitoring of the user.
11. A virtual training (VT) system comprising:
a controller configured to execute instructions to perform a virtual training method, and a sensing system including one or more sensing elements operatively associated with the controller, the virtual training method comprising:
the controller communicating via one or both of video and audio instructions for performing an exercise routine to a user of a VT system, the exercise routine to be performed by the user;
the sensing system sensing exercise motion of the user using the one or more sensing elements, the sensing elements including an image capturing device acquiring an image of the user the image capturing system including a tandem depth sensing range camera and a RGB color camera configured to generate a RGBD (Red-Green-Blue-Depth) image of the user;
the controller processing the RGBD image of the user to determine an exertion level of the user during performance of the exercise routine, and based on the determined exertion level of the user, the controller performing one of the following:
a) visually and/or audibly communicating to the user one or more instructions which increase the exertion level;
b) visually and/or audibly communicating to the user one or more instructions which decrease the execution level; or
c) visually and/or audibly communicating to the user one or more instructions which maintain exertion level,
wherein the method determines an optimum exertion level for the user and dynamically modifies, by the application of animation techniques, the instructions for performing the exercise routine to achieve the optimum exertion level.
12. The VT system according to claim 11 , the method comprising:
comparing the sensed exercise motion of the user performing the exercise to a motion associated with the instructions for performing the exercise.
13. The VT system according to claim 11 , wherein the exertion level of the user is determined using a video-based method to detect one or more physiological metrics associated with the user.
14. The VT system according to claim 13 , wherein the physiological metrics include one or more cardio-respiratory metrics, respiration rate, heart rate, body temperature and blood pressure.
15. The VT system according to claim 11 , wherein the method performs two or more of the following:
performing video-based cardio-respiratory monitoring of the user;
performing video-based fall and lack of action monitoring of the user; and
performing video-based appearance anomaly monitoring of the user.
16. The VT system according to claim 11 ,
wherein the instructions for performing the exercise routine are dynamically modified based on the determined exertion level of the user.
17. The VT system according to claim 11 , wherein the VT method assesses health data related to the user and uses the health data to determine an appropriate exertion level, and/or the VT method uses an anticipated view of the user, obtained from the prescribed exercise routines, to facilitate measurement of one or more cardio-respiratory metrics associated with the user.
18. The computer implemented VT method according to claim 17 , wherein the anticipated view of the user includes any of:
an expected absolute or relative location, pose, and/or velocity of the face and/or other body parts;
and the facilitation includes any of:
improving tracking or localization of the regions of interest of the face and/or other body parts; compensating for the pose changes or velocity of the face and/or other body parts; determining the timing for measuring one or more cardio-respiratory metrics associated with the user; and determining the confidence for the measurement of one or more cardio-respiratory metrics associated with the user.
19. The VT system according to claim 11 , wherein the image capturing device captures RGB (Red-Green-Blue) data and/or D (Depth) data.
20. The VT system according to claim 11 , wherein the method comprises performing video-based fall and lack of action monitoring of the user.