Interactive movement audio engine
A method for generating an audio output is described. Image inputs of interactive movements by a user captured by an image sensor are received. The interactive movements are mapped to a sequence of audio element identifiers. The sequence of audio element identifiers are processed to generate a musical sequence by performing music theory rule enforcement on the sequence of audio element identifiers. An audio output that represents the musical sequence is generated.
1 . A method for generating an audio output, the method comprising:
receiving image inputs of interactive movements by a user captured by an image sensor;
displaying to the user an output image that comprises a graphical user interface overlaid on the image inputs, wherein the interactive movements comprise a plurality of user elements of the user that overlap with a corresponding plurality of icons the graphical user interface;
mapping each of the plurality of user elements of the interactive movements so that the user element overlaps with the corresponding icon of the plurality of icons of the graphical user interface to a sequence of audio element identifiers;
processing the sequence of audio element identifiers to generate a musical sequence by performing music theory rule enforcement on the sequence of audio element identifiers, wherein processing the audio element identifiers comprises:
receiving one or more selectable music theory rules that enforce corresponding elements of music theory;
modifying at least one audio element identifier of the sequence of audio element identifiers that violates a music theory rule when the violated music theory rule is at least one of the one or more received selectable music theory rules; and
generating the musical sequence based on the modified audio element identifier; and
generating an audio output that represents the musical sequence.
2 . The method of claim 1 , wherein modifying the at least one audio element identifier comprises changing a pitch associated with the audio element identifier.
3 . The method of claim 2 , wherein changing the pitch comprises matching a chord progression that satisfies the music theory rule.
4 . The method of claim 1 , wherein modifying the at least one audio element identifier comprises omitting the at least one audio element identifier when generating the musical sequence.
5 . The method of claim 1 , wherein modifying the at least one audio element identifier comprises changing a duration of the at least one audio element identifier.
6 . The method of claim 1 , wherein mapping the interactive movements comprises:
selecting a set of predetermined musical instruments from a plurality of instrument sets;
mapping the interactive movements to instruments within the selected set of predetermined musical instruments.
7 . The method of claim 6 , the method further comprising generating the plurality of instrument sets using a neural network engine that identifies sets of predetermined musical instruments from music samples.
8 . The method of claim 1 , wherein the plurality of user elements of the user are fingers, hands, arms, feet, and/or legs of the user.
9 . The method of claim 1 , wherein the plurality of icons corresponding to a plurality of predetermined audio element identifiers.
10 . The method of claim 9 , wherein the plurality of predetermined audio element identifiers includes single-element identifiers and multi-element identifiers.
11 . The method of claim 1 , wherein the interactive movements are facial expression elements performed by the user.
12 . The method of claim 1 , wherein the interactive movements are gestures performed by the user.
13 . A system for generating an audio output, the system comprising:
one or more hardware processors configured by machine-readable instructions to:
receive image inputs of interactive movements by a user captured by an image sensor;
display to the user an output image that comprises a graphical user interface overlaid on the image inputs, wherein the interactive movements comprise a plurality of user elements of the user that overlap with a corresponding plurality of icons the graphical user interface;
map each of the plurality of user elements of the interactive movements so that the user element overlaps with the corresponding icon of the plurality of icons of the graphical user interface to a sequence of audio element identifiers;
process the sequence of audio element identifiers to generate a musical sequence by performing music theory rule enforcement on the sequence of audio element identifiers, wherein processing the audio element identifiers comprises:
receive one or more selectable music theory rules that enforce corresponding elements of music theory;
modify at least one audio element identifier of the sequence of audio element identifiers that violates a music theory rule when the violated music theory rule is at least one of the one or more received selectable music theory rules; and
generate the musical sequence based on the modified audio element identifier; and
generate an audio output that represents the musical sequence.
14 . The system of claim 13 , wherein the one or more hardware processors are further configured by machine-readable instructions to:
change a pitch associated with the audio element identifier.
15 . The system of claim 13 , wherein the plurality of icons corresponding to a plurality of predetermined audio element identifiers.
16 . The system of claim 13 , wherein modifying the at least one audio element identifier comprises changing a pitch associated with the audio element identifier.
17 . The system of claim 16 , wherein changing the pitch comprises matching a chord progression that satisfies the music theory rule.
18 . The system of claim 13 , wherein modifying the at least one audio element identifier comprises omitting the at least one audio element identifier when generating the musical sequence.
19 . A non-transient computer-readable storage medium comprising instructions being executable by one or more processors, that when executed by the one or more processors, cause the one or more processors to:
receive image inputs of interactive movements by a user captured by an image sensor;
display to the user an output image that comprises a graphical user interface overlaid on the image inputs, wherein the interactive movements comprise a plurality of user elements of the user that overlap with a corresponding plurality of icons the graphical user interface;
map each of the plurality of user elements of the interactive movements so that the user element overlaps with the corresponding icon of the plurality of icons of the graphical user interface to a sequence of audio element identifiers;
process the sequence of audio element identifiers to generate a musical sequence by performing music theory rule enforcement on the sequence of audio element identifiers, wherein processing the audio element identifiers comprises:
receive one or more selectable music theory rules that enforce corresponding elements of music theory;
modify at least one audio element identifier of the sequence of audio element identifiers that violates a music theory rule when the violated music theory rule is at least one of the one or more received selectable music theory rules; and
generate the musical sequence based on the modified audio element identifier; and
generate an audio output that represents the musical sequence.
20 . The computer-readable storage medium of claim 19 , wherein the instructions are executable by the one or more processors to cause the one or more processors to: change a pitch associated with the audio element identifier.