Mobile devices and methods employing acoustic vector sensors
Acoustic vector data is sensed via acoustic vector sensor configurations in mobile devices and used to generate sound fields. From these sound fields, positioning and orientation of the mobile device is derived. The sound fields and features derived from them are classified to provide mobile device position and other information about the environment from which the sound fields are captured. Additionally, sound fields are analyzed to detect gestural input and trigger associated programmatic actions.
1. A method comprising:
obtaining acoustic reference data associated with an environment, the acoustic reference data describing at least one attribute of a sound source from which sound can propagate into the environment;
capturing with an acoustic vector sensor having plural axes, during a first time period, plural vector components within an acoustic field proximate to a device arranged within the environment;
processing the captured vector components of a first acoustic vector field by beamforming to generate first acoustic vector data therefrom, the first acoustic vector data describing at least one attribute of an acoustic feature of the environment at the device;
comparing the first acoustic vector data with the acoustic reference data; and
determining a spatial arrangement of the device based on the comparing.
2. The method of claim 1 wherein obtaining the acoustic reference data comprises:
obtaining location data describing a location of the device;
identifying, from sets of acoustic reference data corresponding to a plurality of locations, acoustic reference data corresponding to the location described by the obtained location data; and
retrieving the identified acoustic reference data.
3. The method of claim 2 wherein obtaining location data comprises detecting a characteristic of the sound propagating through the environment.
4. The method of claim 2 wherein the sets of acoustic reference data correspond to a plurality of locations within the environment in which sound can propagate.
5. The method of claim 1 wherein obtaining the acoustic reference data comprises:
capturing with the acoustic vector sensor plural vector components within the acoustic field proximate to the device during a second time period occurring before the first time period; and
processing the vector components captured during the second time period by beamforming to generate second acoustic vector data therefrom, the second acoustic vector data describing at least one attribute of an acoustic feature within the environment at the device during the second time period.
6. The method of claim 5 wherein processing the vector components captured during the second time period comprises performing a feature detection process.
7. The method of claim 5 wherein processing the vector components captured during the second time period comprises processing the second acoustic vector data to identify at least one substantially stationary acoustic feature.
8. The method of claim 5 wherein processing the vector components captured during the second time period comprises estimating a direction of arrival of sound emanating from the acoustic feature.
9. The method of claim 5 further comprising transmitting a sound signal from the device into the environment such that the sound signal is at least partially reflected within the environment during the second time period while the second acoustic vector data is captured.
10. The method of claim 9 wherein the sound signal comprises a chirp.
11. The method of claim 1 wherein processing the vector components captured during the first time period comprises performing a feature detection process.
12. The method of claim 1 wherein processing the vector components captured during the first time period comprises processing second acoustic vector data captured during a second time period to identify at least one substantially stationary acoustic feature.
13. The method of claim 1 wherein processing the vector components captured during the first time period comprises estimating a direction of arrival (DOA) for the acoustic feature.
14. The method of claim 1 wherein processing the vector components comprises converting plural acoustic vectors from a plural axes configuration of the acoustic vector sensor into a sound map.
15. The method of claim 14 comprising:
classifying sound sources within the sound map with a convolutional neural network to identify and locate the sound sources within the sound map.
16. A method comprising:
capturing acoustic vector sensor data from an acoustic vector sensor configuration having plural axes;
converting plural axes acoustic vector sensor data into a sound map; and
classifying sound sources within the sound map with a convolutional neural network to identify and locate the sound sources within the sound map, the convolutional neural network being trained to identify sound sources within the sound map by presentation of training sounds to the convolutional neural network, the training sounds being labeled with identifiers, enabling the convolutional neural network to classify sound sources within the sound map; wherein
the capturing, converting and classifying is performed for plural sets of acoustic vector sensor data, each captured at different times; and further comprising:
tracking location of sound sources in the sound map over the different times; and
from tracked location of the sound sources, determining a spatial arrangement of the sound sources in an environment around the acoustic vector sensor configuration; and
determining a location of a mobile device by matching the spatial relationship with a database of stored spatial relationships of sound sources.
17. The method of claim 16 wherein the acoustic vector sensor configuration is within a mobile device, the method further comprising: rendering the spatial arrangement of the sound sources on a display device of the mobile device.
18. A system, comprising:
a housing;
an acoustic port extending through the housing;
an acoustic vector sensor arranged within the housing and in communication with an environment surrounding the housing via the acoustic port, the at least one acoustic vector sensor configured to capture a vector component of an acoustic field proximate to an exterior of the housing at the acoustic port and convert the captured vector component into an electrical signal;
an output device; and
a processor configured to access the electrical signal and control an operation of the output device based, at least in part, on the accessed electrical signal.