Method and system for providing adjunct sensory information to a user
A method for providing information to a user, the method including: receiving an input signal from a sensing device associated with a sensory modality of the user; generating a preprocessed signal upon preprocessing the input signal with a set of preprocessing operations; extracting a set of features from the preprocessed signal; processing the set of features with a neural network system; mapping outputs of the neural network system to a device domain associated with a device including a distribution of haptic actuators in proximity to the user; and at the distribution of haptic actuators, cooperatively producing a haptic output representative of at least a portion of the input signal, thereby providing information to the user.
1. A system for providing information to a user, the system comprising:
a limb-worn device, the limb-worn device comprising an audio sensing device, wherein the audio sensing device receives an input signal associated with speech;
a processing system, wherein the processing system:
generates a preprocessed signal upon preprocessing the input signal with a set of preprocessing operations for a set of frames of the input signal;
processes the preprocessed signal to produce a set of outputs; and
maps the set of outputs to a device domain associated with the limb-worn device;
a plurality of haptic actuators coupled to the limb-worn device, wherein the plurality of haptic actuators, in near real time, cooperatively produces a haptic output representative of the input signal, thereby providing speech information to the user at the limb-worn device.
2. The system of claim 1 , wherein the processing system processes the preprocessed with a neural network system.
3. The system of claim 2 , wherein the neural network system comprises a convolutional layer and a recurrent architecture, and implements a statistical model for recognition of speech components from the input signal.
4. The system of claim 1 , wherein the limb-worn device comprises a wrist-borne device, and wherein the plurality of haptic actuators comprises at least four haptic actuators configured to be arranged around at least a partial circumference of the user's wrist.
5. The system of claim 1 , wherein the input signal is associated with speech and wherein the haptic output represents a phoneme component.
6. The system of claim 5 , wherein the phoneme component comprises a high frequency phoneme component comprising at least one of: a/f/phoneme, a/z/phenome, a/b/phenome, a/th/phoneme, a/dh/phenome, a/t/phenome, a/d/phenome, a/s/phoneme, and a/v/phoneme.
7. The system of claim 1 , wherein processing the preprocessed signal comprises extracting a plurality of frequency bins each comprising a set of frequencies from the preprocessed signal and determining a power parameter associated with each frequency bin.
8. The system of claim 7 , wherein each of the plurality of haptic actuators is associated with a subset of the plurality of frequency bins, wherein the set of outputs comprises a maximum power parameter associated with at least one subset of frequency bins.
9. The system of claim 8 , wherein the set of outputs comprises a maximum power parameter associated with each subset of the plurality of frequency bins.
10. A system for providing information to a user, the system comprising:
a wrist-borne device, the wrist-borne device comprising a microphone, wherein the microphone receives an input signal;
a processing system at least partially arranged onboard the wrist-borne device, wherein the processing system:
generates a preprocessed signal upon preprocessing the input signal with a set of preprocessing operations for a set of frames of the input signal;
processes the preprocessed signal to produce a set of outputs; and
maps the set of outputs to a device domain associated with the wrist-borne device;
a plurality of haptic actuators coupled to wrist-borne device and configured to be arranged around at least a partial circumference of the user's wrist, wherein the plurality of haptic actuators, in near real time, cooperatively produces a haptic output representative of the input signal, thereby providing audio information to the user at the wrist-borne device.
11. The system of claim 10 , wherein the processing system processes the preprocessed with a neural network system.
12. The system of claim 11 , wherein the neural network system comprises a convolutional layer and a recurrent architecture, and implements a statistical model for recognition of audio components from the input signal.
13. The system of claim 10 , wherein the processing system is at least partially arranged at the wrist-borne device.
14. The system of claim 13 , wherein the processing system is fully arranged at the wrist-borne device.
15. The system of claim 10 , wherein the input signal is associated with speech and wherein the haptic output represents a phoneme component.
16. The system of claim 15 , wherein the phoneme component comprises a high frequency phoneme component comprising at least one of: a/f/phoneme, a/z/phenome, a/b/phenome, a/th/phoneme, a/dh/phenome, a/t/phenome, a/d/phenome, a/s/phoneme, and a/v/phoneme.
17. The system of claim 10 , wherein processing the preprocessed signal comprises extracting a plurality of frequency bins each comprising a set of frequencies from the preprocessed signal and determining a power parameter associated with each frequency bin.
18. The system of claim 17 , wherein each of the plurality of haptic actuators is associated with a subset of the plurality of frequency bins, wherein the set of outputs comprises a maximum power parameter associated with at least one subset of frequency bins.
19. The system of claim 18 , wherein the set of outputs comprises a maximum power parameter associated with each subset of the plurality of frequency bins.
20. The system of claim 10 , wherein the input signal is associated with music, and wherein the set of outputs is associated with a set of loudest frequencies of the input signal.