Context-aware input gestures for music creation applications
Music creation applications enable users to enter new objects into a musical project using gestures mediated by input maps that are adapted to the musical context of the object entry location. The musical context may be defined by an explicit aspect of the musical project or implicitly by notes or intervals already entered into the project. The adaptation of the input maps may include spatial scaling to provide greater space to objects favored by the context. The context-adapted input maps may be displayed and used to guide user input, such as with gestures on a touch screen. In some implementations, input-sensitive regions of input maps are scaled in accordance with the musical context without being displayed. The methods apply to musical object entry via pop-up input maps and virtual instrument displays.
1 . A method of enabling a user of a music creation application to enter data into a musical project being created using the music creation application, the method comprising:
using data from a portion of the musical project in a temporal vicinity of a location in the musical project at which a new object is to be placed to determine a current musical context;
generating and displaying a first input map of a plurality of musical objects within a graphical user interface of the music creation application, wherein at least one of a layout and object content of the first input map is based at least in part on the current musical context;
enabling a user of the music creation application to select from the first input map the new object to be entered at the location in the musical project; and
in response to a selection by the user of the new object from the first input map, entering the selected new object at the location within the musical project and displaying the new object within the graphical user interface of the music creation application.
2 . The method of claim 1 , wherein a spatial scaling of the first input map is based on the current musical context, such that objects favored by the current musical context are allocated more space in the first input map than objects that are not favored by the current musical context.
3 . The method of claim 2 , wherein the spatial scaling of the first input map is a spatial scaling of input-sensitive areas of the first input map.
4 . The method of claim 1 , wherein the first input map comprises a virtual keyboard instrument, and a spatial scaling scales an area sensitive to at least one of a touch gesture and a drag gesture such that keys of the virtual keyboard representing notes that are favored by the current musical context are allocated a larger sensitive area than keys representing notes that are not favored by the current musical context.
5 . The method of claim 1 , wherein the first input map comprises a virtual keyboard instrument, and keys of the virtual keyboard are displayed with a graphical feature indicative of the current musical context.
6 . The method of claim 1 , wherein the first input map is displayed with a default selection based on the current musical context.
7 . The method of claim 6 , wherein the new object is a note, and the default selection is a pitch of the note.
8 . The method of claim 1 , wherein the current musical context is explicitly associated with the musical project.
9 . The method of claim 1 , wherein the current musical context is determined by the music creation application by analyzing at least a portion of the musical project that has already been entered.
10 . The method of claim 1 , wherein the current musical context is a key of the musical project in the temporal vicinity of the location of the musical project at which the new object is to be placed.
11 . The method of claim 1 , wherein the current musical context is a chord in the temporal vicinity of the location of the musical project at which the new object is to be placed.
12 . The method of claim 1 , wherein the current musical context is based on diatonic intervals of notes that are already present within the musical project in the temporal vicinity of the location of the musical project at which the new object is to be placed.
13 . The method of claim 1 , wherein the current musical context is based on pitches of notes that are already present within the musical project in the temporal vicinity of the location of the musical project at which the new object is to be placed.
14 . The method of claim 1 , wherein the temporal vicinity of the location of the musical project at which the new object is to be placed is a span of less than one of 3, 4, 5, and 6 measures of the musical project temporally preceding the location in the musical project at which the new object is to be placed.
15 . The method of claim 1 , wherein the user selection of the new object to be entered comprises:
a selection of a first aspect of the new object from the first input map, wherein the first input map includes:
a plurality of options for the first aspect of the new object, wherein the at least one of the layout and object content of the first input map is based on a first aspect of the current musical context; and
a selection of a second aspect of the new object from a second input map displayed within the graphical user interface of the music creation application, wherein the second input map includes:
a plurality of options for the second aspect of the new object, and wherein at least one of a layout and object content of the second input map is based on a second aspect of the current musical context.
16 . The method of claim 15 , wherein the first aspect of the new object is a note duration, and the second aspect of the new object is a note pitch.
17 . The method of claim 1 , further comprising enabling the user to select from a second input map a modifier of the new musical object, wherein the second input map is based at least in part on the current musical context.
18 . The method of claim 1 , wherein the selection of the new object to be entered into the musical project involves a gesture by the user having a starting point on the first input map, a drag movement within the first input map, and a release point within the first input map, and wherein an effect of at least one of the starting point, drag movement, and release point is based in part on the current musical context.
19 . The method of claim 1 , wherein the selection of the new object to be entered into the musical project involves a gesture by the user that ends with a release point within the first input map, and wherein the music creation application performs error correction at the release point based in part on the current musical context.
20 . The method of claim 1 , wherein the user selection of the new object from the first input map includes a drag motion from an initial selection within the first input map to a final selection within the first input map, and wherein the music creation application generates haptic feedback during the drag motion that favors user selection of objects favored by the current musical context.
21 . The method of claim 1 , wherein the music creation application determines the current musical context based in part on a prior object entry behavior of the user.
22 . A computer program product comprising:
a non-transitory computer-readable medium with computer-readable instructions encoded thereon, wherein the computer-readable instructions, when processed by a processing device instruct the processing device to perform a method of enabling a user of a music creation application to enter data into a musical project, the method comprising:
using data from a portion of the musical project in a temporal vicinity of a location in the musical project at which a new object is to be placed to determine a current musical context;
generating and displaying an input map of a plurality of musical objects within a graphical user interface of the music creation application, wherein at least one of a layout and object content of the input map is based at least in part on the current musical context;
enabling a user of the music creation application to select from the input map the new object to be entered at the location in the musical project; and
in response to a selection by the user of the new object from the input map, entering the selected new object at the location within the musical project and displaying the new object within the graphical user interface of the music creation application.
23 . A system comprising:
a memory for storing computer-readable instructions; and
a processor connected to the memory, wherein the processor, when executing the computer-readable instructions, causes the system to perform a method of enabling a user of a music creation application to enter data into a musical project, the method comprising:
using data from a portion of the musical project in a temporal vicinity of a location in the musical project at which a new object is to be placed to determine a current musical context;
generating and displaying an input map of a plurality of musical objects within a graphical user interface of the music creation application, wherein at least one of a layout and object content of the input map is based at least in part on the current musical context;
enabling a user of the music creation application to select from the input map the new object to be entered at the location in the musical project; and
in response to a selection by the user of the new object from the input map, entering the selected new object at the location within the musical project and displaying the new object within the graphical user interface of the music creation application.