Hearing apparatus with bone conduction sensor
The present disclosure relates to a hearing apparatus comprising: a bone conduction sensor configured to convert bone vibrations of voice sound information into a bone conduction signal; a signal processing unit configured to implement a synthetic speech generation process, the synthetic speech generation process implementing a speech model; wherein the synthetic speech generation process receives the bone conduction signal as a control input and outputs a synthetic speech signal.
1. A hearing apparatus comprising:
a bone conduction sensor configured to provide a bone conduction signal indicative of bone-conducted vibration conducted by a bone of a wearer of the hearing apparatus; and
a signal processing unit comprising a speech model, the speech model comprising a neural network model configured to obtain a representation of the bone conduction signal as a control input, wherein the signal processing unit is configured to provide a synthetic speech signal;
wherein the signal processing unit is configured to predict a current sample of a time series from one or more previous samples of the time series, the time series representing a speech waveform, wherein the signal processing unit is configured to predict the current sample of the time series based on the representation of the bone conduction signal;
wherein the neural network model comprises a layer implemented as a part of the neural network model of the signal processing unit that provides the synthetic speech signal;
wherein the neural network model is trained based on a plurality of training speech samples; and
wherein at least one of the training speech samples comprises a training bone conduction data representing a speech and a corresponding training microphone data representing airborne sound of the speech, the training microphone data and the training bone conduction data corresponding with each other temporally.
2. The hearing apparatus according to claim 1 , wherein the speech model defines an internal state that evolves over time.
3. The hearing apparatus according to claim 1 , wherein the neural network comprises a recurrent neural network.
4. The hearing apparatus according to claim 3 , wherein the recurrent neural network has a density estimation mode during operation.
5. The hearing apparatus according to claim 1 , wherein the neural network comprises a layered neural network comprising two or more layers, at least one of the two or more layers being a softmax layer.
6. The hearing apparatus according to claim 1 , wherein the speech model comprises an autoregressive speech model.
7. The hearing apparatus according to claim 1 , wherein the speech model is configured to compute a probability distribution over a plurality of output classes, at least one of the output classes representing a sample value of a sample of a sampled audio waveform.
8. The hearing apparatus according to claim 1 , further comprising a head-worn hearing device, the head-worn hearing device comprising the bone conduction sensor and a first communication interface.
9. The hearing apparatus according to claim 8 , wherein the head-worn hearing device further comprises the signal processing unit, and wherein the head-worn hearing device is configured to communicate the synthetic speech signal via the first communication interface to a handheld communication device.
10. The hearing apparatus according to claim 8 , further comprising a signal processing device, the signal processing device comprising the signal processing unit and a second communication interface;
wherein the first communication interface of the head-worn hearing device is configured to communicate the bone conduction signal to the second communication interface of the signal processing device.
11. The hearing apparatus according to claim 1 , further comprising an ambient microphone configured to detect air-borne speech spoken by the wearer of the hearing apparatus, and to provide an ambient microphone signal indicative of the detected air-borne speech.
12. The hearing apparatus according to claim 1 , further comprising a memory configured to store training data, the training data comprising one or more signal pairs, at least one of the signal pairs comprising the training bone conduction data and the training microphone data.
13. The hearing apparatus according to claim 1 , wherein the signal processing unit is configured to generate a synthetic filtered signal corresponding to a speech signal filtered by a first filter, after receiving the representation of the bone conduction signal as the control input.
14. The hearing apparatus according to claim 13 , wherein the synthetic filtered signal is the synthetic speech signal.
15. The hearing apparatus according to claim 1 , wherein the hearing apparatus is a hearing aid.
16. The hearing apparatus according to claim 1 , wherein the hearing apparatus is a BTE, RIE, ITE, ITC or CIC hearing instrument.
17. A hearing apparatus comprising:
a bone conduction sensor configured to provide a bone conduction signal indicative of bone-conducted vibration conducted by a bone of a wearer of the hearing apparatus; and
a signal processing unit comprising a speech model, the speech model comprising a neural network model configured to obtain a representation of the bone conduction signal as a control input, wherein the signal processing unit is configured to provide a synthetic speech signal;
wherein the signal processing unit is configured to generate a synthetic filtered signal corresponding to a speech signal filtered by a first filter; and
wherein the signal processing unit is configured to receive an ambient microphone signal associated with an ambient microphone, the ambient microphone signal and the bone conduction signal corresponding with each other temporally, and
wherein the signal processing unit is configured to create a filtered version of the received ambient microphone signal using a second filter, and to combine the generated synthetic filtered signal with the created filtered version of the received ambient microphone signal to create the synthetic speech signal.
18. The hearing apparatus according to claim 17 , wherein the speech model is a machine learning model, and wherein the machine learning model is trained based on a plurality of training speech samples.
19. The hearing apparatus according to claim 18 , wherein at least one of the training speech samples comprises a training bone conduction data and a corresponding training microphone data, the training microphone data and the training bone conduction data corresponding with each other temporally.
20. The hearing apparatus according to claim 17 , wherein the signal processing unit, when in a training mode, is configured to adapt one or more model parameters of the speech model.
21. The hearing apparatus according to claim 20 , wherein the adapted one or more model parameters are configured to allow the speech model to provide an improved match between a model output representing the synthetic speech and a corresponding training ambient microphone signal.
22. A processor-implemented method of obtaining a synthetic speech signal, comprising:
receiving, by a processing unit of an apparatus, a bone conduction signal from a bone conduction sensor, the bone conduction sensor configured to detect a bone-conducted vibration conducted by a bone of a person; and
using the signal processing unit to predict a current sample of a time series from one or more previous samples of the time series, the time series representing a speech waveform, wherein the signal processing unit is configured to predict the current sample of the time series based on the bone conduction signal, wherein the signal processing unit comprises a neural network model configured to receive the bone conduction signal as a control input;
wherein the neural network model comprises a layer implemented as a part of the neural network model of the signal processing unit;
wherein the neural network model is trained based on a plurality of training speech samples; and
wherein at least one of the training speech samples comprises a training bone conduction data representing a speech and a corresponding training microphone data representing airborne sound of the speech, the training microphone data and the training bone conduction data corresponding with each other temporally.