Method and system for providing adjunct sensory information to a user

1. A method for providing information to a user, the method comprising:

receiving an input signal from an audio sensing device, the input signal associated with speech;

generating a preprocessed signal upon preprocessing the input signal with a set of preprocessing operations for a set of frames of the input signal;

extracting a set of features from the preprocessed signal;

processing the set of features with a neural network system;

mapping outputs of the neural network system to a device domain associated with a limb-worn device including a distribution of haptic actuators in proximity to the user; and

in near real time and at the distribution of haptic actuators, cooperatively producing a haptic output representative of speech components of the input signal, thereby providing speech information to the user at the limb-worn device, wherein the haptic output represents a high frequency phoneme components comprising at least one of: a /f/ phoneme, a /z/ phenome, a /b/ phenome, a /th/ phenome, a /dh/ phenome, a /t/ phenome, a /d/ phoneme, a /s/ phoneme, and a /v/ phoneme.

2. The method of claim 1 , wherein receiving the input signal is performed contemporaneously with producing the haptic output and delivering stimulation to the user with the haptic output, thereby allowing the user to perceive speech content captured through the input signal, through touch receptors, in near real time.

3. The method of claim 2 , wherein producing a haptic output representative of speech components comprises mapping outputs of the neural network system, associated with high frequency speech components, to the device domain and delivering tactile stimuli representative of the high frequency speech components to the user.

4. The method of claim 1 , wherein speech components represented in the haptic output comprise at least one of: phoneme-associated components and word-associated components.

5. The method of claim 1 , wherein mapping between the distribution of haptic actuators and speech components comprises a 1:1 mapping.

6. The method of claim 1 , wherein receiving the input signal comprises receiving input signals from a set of microphones of the audio sensing device, and implementing a noise removal operation that applies a beam forming operation to the input signals from the set of microphones.

7. The method of claim 1 , further comprising refining at least one of:

generating the preprocessed signal, extracting the set of features from the preprocessed signal, processing the set of features with the neural network system, mapping outputs of the neural network system to the device domain, and producing the haptic output with a training dataset, wherein refining comprises improving power consumption of the distribution of haptic actuators with a user tactile perception constraint.

8. The method of claim 1 , wherein the neural network system comprises a convolutional layer and a recurrent architecture, and implements a statistical model for recognition of speech components from the input signal.

9. A method for providing information to a user, the method comprising:

receiving an input signal from a sensing device associated with a sensory modality of the user;

generating a preprocessed signal upon preprocessing the input signal with a set of preprocessing operations;

extracting a set of features from the preprocessed signal;

processing the set of features with a neural network system, wherein the neural network system comprises a convolutional layer and a recurrent architecture, and implements a statistical model for recognition of speech components from the input signal;

mapping outputs of the neural network system to a device domain associated with a device including a distribution of haptic actuators in proximity to the user; and

at the distribution of haptic actuators, cooperatively producing a haptic output representative of at least a portion of the input signal, thereby providing information to the user.

10. The method of claim 9 , wherein the sensing device comprises at least two of: a microphone system, a motion sensing unit, and a location identifying unit, and wherein the sensing device is coupled to the device including the distribution of haptic actuators.

11. The method of claim 10 , wherein the device including the distribution of haptic actuators comprises at least one of a wrist-borne form factor and a torso-coupled form factor in retaining the distribution of haptic actuators proximal a body region of the user during use.

12. The method of claim 9 , wherein generating the preprocessed signal comprises segmenting the input signal into a set of frames, and wherein the method further comprises a) generating an accuracy metric for a predicted speech component captured in at least a subset of the set of frames, and b) modulating a stimulation parameter of the haptic output based upon the accuracy metric.

13. The method of claim 9 , wherein processing the set of features with the neural network system comprises compressing the neural network system with a network compression technique comprising a deep compression process including pruning, trained quantization, and Huffman coding, and executing the compressed neural network system on at least one of embedded hardware of the device and a mobile device of the user.

14. The method of claim 9 , wherein the set of features comprises spectral peak frequencies, spectral peak magnitudes, spectral rolloff-derived features, spectral centroid-derived features, frequency band energy ratios, gammatone transforms, measures of spectral deformation, zero crossing rates, lacunarity, and moments of crossing intervals.

15. The method of claim 9 , wherein the set of preprocessing operations comprises a filtering operation to remove environmental artifacts, a windowing operation, and a transformation operation implemented across a set of frames of the input signal.

16. The method of claim 9 , wherein the haptic output represents a high frequency phoneme components comprising at least one of: a /f/ phoneme, a /z/ phenome, a /b/ phenome, a /th/ phoneme, a /dh/ phenome, a /t/ phenome, a /d/ phenome, a /s/ phoneme, and a /v/ phoneme.

17. The method of claim 9 , wherein receiving the input signal comprises receiving input signals from a set of microphones of the sensing device a motion sensing unit of the sensing device; and wherein the method further includes implementing a noise removal operation that applies a beam forming operation to the input signals based on data from the set of microphones and the sensing device.

18. The method of claim 9 , wherein producing the haptic output comprises delivering a unique waveform using a subset of the distribution of haptic actuators, for each of a set of phonemes captured in the input signal.

19. The method of claim 9 , wherein receiving the input signal is performed contemporaneously with producing the haptic output and delivering stimulation to the user with the haptic output, thereby allowing the user to perceive content captured through the input signal, through touch receptors, in near real time.

20. The method of claim 9 , wherein the distribution of haptic actuators is arranged circumferentially about a band support structure operable to be positioned at a wrist region of the user during operation of the method.