System, method and apparatus for detecting facial expression in a virtual reality system
A system, method and apparatus for detecting facial expressions according to EMG signals for a VR environment.
1. A system for determining a facial expression on a face of a subject in a VR (virtual reality) environment, comprising
an apparatus comprising a plurality of EMG (electromyography) electrodes arranged so as to be in contact with at least a portion of the face of a subject when the apparatus is worn by the subject;
a computational device configured to receive a plurality of EMG signals from said EMG electrodes and configured with:
a first set of instructions operating thereon to cause the computational device to: process said EMG signals to form processed EMG signals; and
classifying a facial expression according to said processed EMG using a classifier; and
a second set of instructions operating thereon comprising a VR application for receiving said classified facial expression; and
a display for displaying a VR environment according to the VR application.
2. The system of claim 1 , wherein classifying comprises determining whether the facial expression corresponds to a neutral expression or a non-neutral expression.
3. The system of claim 1 , wherein said VR application is configured to receive user input for one or more commands for operating and/or controlling said computational device and/or computer instructions operating thereon.
4. A method for providing feedback to a user in a VR (virtual reality) environment, comprising:
providing a wearable apparatus for determining a facial expression on a face of a subject for a VR (virtual reality) environment, the apparatus comprising:
a facemask including a plurality of EMG (electromyography) electrodes arranged thereon so as to be in contact with at least a portion of the face of the subject upon the subject wearing the facemask;
a computational device configured to receive a plurality of EMG signals from said EMG electrodes and configured with:
a first set of instructions operating thereon to cause the computational device to:
process said EMG signals to form processed EMG signals; and
classifying a facial expression according to said processed EMG using a classifier; and
a second set of instructions operating thereon comprising a VR application for receiving said classified facial expression;
a display for displaying a VR environment to the user; and
a mount, wherein at least one of said facemask, said computational device and said display are physically connected to said mount or to one-another;
receiving a plurality of EMG signals from said EMG electrodes;
determining the facial expression according to said processed EMG signals;
providing feedback to the user regarding the facial expression;
the method further comprising:
displaying the VR environment to the user, wherein the VR environment includes a system avatar;
displaying a facial expression by the system avatar;
analyzing a corresponding facial expression by the user; and
providing feedback to the user regarding the correspondence of the facial expression.
5. The method of claim 4 , further comprising:
displaying another avatar for representing a facial expression of the user; and
displaying said corresponding facial expression by the user by said other avatar.
6. The method of claim 4 , further comprising:
displaying the VR environment to the user, wherein the VR environment includes a system avatar;
communicating between the user and said system avatar;
analyzing a communication style of the user, including a facial expression; and
providing feedback on said communication style.