VIRTUAL, AUGMENTED OR MIXED REALITY INSTRUMENT TEACHING SYSTEM AND METHOD
Aspects of embodiments pertain to systems and methods for providing a user with information relating to the playing of a musical instrument. The systems and methods comprise presenting the user with instrument playing instructions to be executed by at least one user; identifying, of the at least one user, at least one object expected to operably engage with the musical instrument for executing the instrument playing instructions; tracking movement of the identified at least one object; and displaying to the user at least one assistive symbol at least partly in correspondence with the presented instrument playing instructions.
1 . A system for determining an emotional state of one or more users of an instrument teaching and/or singing teaching app, the system comprising:
a processor; and
a memory storing instructions executable by the processor to result in the execution of the following:
presenting to the at least one app user with information about a musical piece to be played and/or sung;
receiving data relating to one of the following of the at least one app user: user physiological parameter, behavioral parameter, or both;
processing the received data to determine one or more emotional states of the at least one app user; and
based on the determined one or more emotional states, determining whether the at least one app user is to be presented with updated information or not.
2 . The system of claim 1 , wherein the updated information pertains to one or more the following:
lesson assignment; changing a level of difficulty of a same musical exercise; adjusting tolerance thresholds; adapting UX feature presentations; adapting social sharing recommendations; or any combination of the above.
3 . The system of claim 1 , configured to determine, based on the one or more emotional states, the probability of continued app utilization.
4 . The system of claim 3 , further configured to adapt, based on the determined probability of continued app utilization, information presented to the user.
5 . The system of claim 4 , wherein the adapting of the information includes adapting one or more of the following: lesson assignment; changing a level of difficulty of a same musical exercise; adjusting tolerance thresholds; adapting UX feature presentations; adapting social sharing recommendations; or any combination of the above.
6 . The system of claim 1 , wherein monitoring the at least one behavioral parameter includes monitoring: facial expressions, gestures, posture, motion and/or movement of the at least one app user.
7 . The system of claim 1 , further configured to monitor at least one physiological and/or behavioral parameter of one or more audience members experiencing the performance of the at least one app user playing an instrument and/or singing a musical piece; and/or verbal and/or non-verbal interactions between the at least one app users and the at least one audience member.
8 . A system configured to provide a user with information relating to the playing of a musical instrument, the system comprising:
a processor; and
a memory storing instructions executable by the processor to result in the execution of the following:
identifying at least one actionable engagement executed by the user with the musical instrument;
receiving data descriptive about at least one next actionable engagement of the user with the instrument; and
determining, based on the current actionable engagement and the next actionable engagement, transition information to be presented to the user for making a transition from the current actionable engagement to the next actionable engagement.
9 . The system of claim 8 , further comprising displaying to the user the transition information.
10 . The system of claim 8 , further comprising evaluating the transition performed by the user.
11 . The system of claim 8 , wherein the transition information is displayed as guidance symbols or hints.
12 . The system of claim 8 , wherein the transition information is displayed in overlay to body parts of the user and/or in overlay to the instrument played by the user.
13 . The system of claim 8 , wherein the transition information includes the displaying of moving symbols for tracing by the user.
14 . The system of claim 8 , wherein the transition information includes one of the following: a virtual body part, an arrow, a geometric symbol, or any combination of the aforesaid
15 . A system configured to provide a user with information relating to the playing of a musical instrument, the system comprising:
a processor; and
a memory storing instructions executable by the processor to result in the execution of the following:
presenting the user with instrument playing instructions to be executed by at least one user;
identifying, of the at least one user, at least one object expected to operably engage with the musical instrument for executing the instrument playing instructions;
tracking movement of the identified at least one object; and
displaying to the user at least one assistive symbol at least partly in correspondence with the presented instrument playing instructions.
16 . The system of claim 15 , wherein the at least one assistive symbol is displayed in a virtual environment, mixed environment or in an augmented environment in overlay with the at least one identified object.
17 . The system of claim 15 , wherein the at least one object is a real-world body part viewable by the user via a see-through display, ora virtual representation of the real-world body part of the user.
18 . The system of claim 15 , wherein the assistive symbol is displayed in overlay with a corresponding at least one object to guide the user how to operably engage with the musical instrument.
19 . The system of claim 15 , comprising:
identifying operable engagement of the at least one object with the musical instrument;
determining a level of correspondence between the operable engagement of the at least one object and the received playing instructions; and
displaying the assistive symbol in accordance with the determined level of correspondence.
20 . The system of claim 19 , configured to identify a mismatch event where the level of correspondence does not meet a performance success criterion for playing the musical instrument; and
displaying an updated assistive symbol for providing the user with guidance on how to engage the object with the musical instrument to prevent recurrence of the mismatch event.