Spatial user interface for audio system
A system, apparatus, and method for generating a spatial user interface for an application, system or device. The user interface includes a means of representing user interface functions or commands as audio signals, with the audio signals being perceived by the user in a spatially different location depending on the function or command. A user input device is provided to enable a user to select a function or command, or to navigate through the spatial representations of the audio signals.
1. A system, comprising:
a spatial model generator including a spatial input decoder in communication with a spatial model manager and operative to receive input data associated with a user interface element, and to generate output data defining a spatial location at which a user will perceive an audio signal corresponding to the user interface element, the user interface element including a multi-dimensional spatial model that determines the spatial location;
a spatial audio effects processor in communication with the spatial model manager, the spatial audio effects processor operative to receive as an input an audio input and to generate the audio signal as an output comprised of multiple channels generated from a plurality of streams, a first portion of the plurality of streams comprising a voice stream, the spatial audio effects processor being configured to mix a second portion of the plurality of streams to form a second channel stream, to mix a third portion of the plurality of streams to form a third channel stream, to determine a strength of volume between the second channel stream and the third channel stream using the multi-dimensional spatial model,
each channel modified relative to other channels according to its distance from an origin of the multi-dimensional spatial model, its angle in the multi-dimensional spatial model or both, the audio signal representing the processed audio input as perceived by the user at the spatial location;
an application system in communication with the spatial audio effects processor and the spatial model manager, the application system being configured to provide user interface element information to the spatial model manager, the spatial audio effects processor being configured to communicate data corresponding to user interface elements to and from the application system; and
an input device electrically coupled with the spatial input decoder, the input device including a plurality of 3-dimensional accelerometers, the input device operative to select the audio signal or to alter a spatial arrangement of the audio signal that is perceived by the user, and the arrangement of the audio signal including a selected one or more of transforming, changing, rotating, restoring, navigating, or translating the multi-dimensional spatial model in a plurality of dimensions.
2. The system of claim 1 , wherein the spatial audio effects processor is further operative to process the audio input to include information that is suggestive to the user of the function or operation associated with the audio signal.
3. The system of claim 2 , wherein the information that is suggestive of the function or operation associate with the audio signal includes one or more of echo, volume, rising pitch, falling pitch, or reverberation.
4. The system of claim 1 , wherein the spatial model generator is operative to generate output data defining a spatial location that is suggestive to the user of the function or operation of the user interface element associated with the audio signal.
5. The system of claim 1 , wherein the input data associated with a user interface element is one or more of data related to a function of the user interface element, an operation of the user interface element, or a relative importance of the user interface element.
6. A system, comprising:
a plurality of audio signals, each signal being perceived by a user at a spatially distinct location, and each signal associated with a function or operation of the system, device, or application;
an input device including a plurality of 3-dimensional accelerometers, the input device operative to generate an input signal that is decoded by a spatial input decoder in communication with a spatial model manager, the spatial input decoder operative to select one of the plurality of audio signals to alter a spatial arrangement of the audio signals that is perceived by the user, the spatial model manager operative to alter a spatial arrangement of the audio signals using a multi-dimensional spatial model, and the arrangement of the audio signals including a selected one or more of transforming, changing, rotating, restoring, navigating, or translating the multi-dimensional spatial model in a plurality of dimensions;
a spatial audio effects processor operative to receive the input signal from the input device and in response to activate the functions or operation of the system, device or application, or to alter the spatial arrangement of the audio signals that is perceived by the user by generating an output comprised of multiple channels generated from a plurality of streams, a first portion of the plurality of streams comprising a voice stream, the spatial audio effects processor being configured to mix a second portion of the plurality of streams to form a second channel stream, to mix a third portion of the plurality of streams to form a third channel stream, to determine a strength of volume between the second channel stream and the third channel stream using the multi-dimensional spatial model, each channel modified relative to other channels according to its distance from an origin of the multi-dimensional spatial model, its angle in the multi-dimensional spatial model or both; and
an application system in communication with the spatial audio effects processor and the spatial model manager, the application system being configured to provide user interface element information to the spatial model manager, the spatial audio effects processor being configured to communicate data corresponding to user interface elements to and from the application system.
7. The user interface of claim 6 , wherein one or more of the plurality of audio signals includes information that is suggestive to the user of the functions or operation associate with the audio signal.
8. The user interface of claim 7 , wherein the information that is suggestive of the function of operation associated with the audio signal includes one or more of echo, volume, rising pitch, or reverberation.
9. The user interface of claim 6 , wherein one or more of the plurality of audio signals are perceived by the user at a spatially distinct locations that is suggestive to the user of the function of operation associated with the audio signal.
10. The user interface of claim 6 , wherein one or more of the plurality of audio signals are perceived by the user at a spatially distinct location that is suggestive to the user of the relative importance of the function or operation associated with the audio signal.
11. A method for generating a user interface, comprising:
accessing information regarding a user interface element;
generating, at a spatial model manager, a multi-dimensional spatial model for the user interface element based on the accessed information, the multi-dimensional spatial model representing a mapping of the user interface element to a position in a user's perceived audio space;
associating audio signals with the position in the user's perceived audio space, the audio signals modified according to a distance from an origin of the multi-dimensional spatial model, an angle in the multi-dimensional spatial model or both;
receiving, at a spatial input decoder, an input signal an input device having a plurality of 3-dimensional accelerometers;
processing, at a spatial audio effects processor, the received input signal using the spatial input decoder to determine whether the input represents a selection by the user interface element or a user request to alter the multi-dimensional spatial model, and the user request to alter including a selected one or more of transforming, changing, rotating, restoring, navigating, or translating the multi-dimensional spatial model in a plurality of dimensions, the audio signals included in a plurality of streams, a first portion of the plurality of streams comprising a voice stream, the processing being configured to mix a second portion of the plurality of streams to form a second channel stream, to mix a third portion of the plurality of streams to form a third channel stream, to determine a strength of volume between the second channel stream and the third channel stream using the multi-dimensional spatial model;
providing, by an application system, user interface element information to the spatial model manager; and
communicating, between the application system and the spatial audio effects processor, data corresponding to user interface elements.
12. The method of claim 11 , wherein the accessed information is information related to a function or operation of the user interface element.
13. The method of claim 11 , wherein the accessed information is information related to a relative importance of the user interface element.
14. The method of claim 11 , wherein the audio signals associated with the position in the user's perceived audio space is one or more of a voice prompt, music, or a sound effect.
15. The method of claim 14 , wherein the audio signals includes information that is suggestive to the user of the user interface element function or operation associated with the audio signals.
16. The method of claim 15 , wherein the information that is suggestive of the function or operation associated with the audio signals includes one or more of echo, volume, rising pitch, or reverberation.