Harmonic feature processing for reducing noise
Devices, systems and methods are disclosed for reducing noise in input data by performing a hysteresis operation followed by a lateral excitation smoothing operation. For example, an audio signal may be represented as a sequence of feature vectors. A row of the sequence of feature vectors may, for example, be associated with the same harmonic of the audio signal at different points in time. To determine portions of the row that correspond to the harmonic being present, the system may compare an amplitude to a low threshold and a high threshold and select a series of data points that are above the low threshold and include at least one data point above the high threshold. The system may iteratively perform a spreading technique, spreading a center value of a center data point in a kernel to neighboring data points in the kernel, to further reduce noise.
1. A computer-implemented method for reducing noise in an audio signal, the method comprising:
obtaining a sequence of feature vectors from an audio signal, wherein each feature vector of the sequence is computed from a portion of the audio signal and represents the portion of the audio signal as a function of frequency;
obtaining a low threshold and a high threshold;
modifying the sequence of feature vectors by performing a hysteresis operation on a row of the sequence of feature vectors, wherein the row comprises element k of each feature vector of the sequence of feature vectors, and wherein the hysteresis operation comprises:
initializing all of the elements of the row to a first state,
determining that a first element of the row is part of a continuous sequence of elements of the row, wherein (a) at least one element of the sequence is greater than the high threshold and (b) all elements of the sequence are greater than the low threshold,
changing the state of the first element to a second state, and
for each element of the row having the first state, changing a value of the element;
further modifying the sequence of feature vectors by:
obtaining a scale factor between 0 and 1,
selecting element j of a first feature vector,
determining a spread value by multiplying the scale factor by a value of the element j of the first feature vector, and
adding the spread value to (i) element j+1 of the first feature vector, (ii) element j−1 of the first feature vector, (iii) element j of a feature vector subsequent to the first feature vector, and (iv) element j of a feature vector antecedent to the first feature vector; and
generating output data including the sequence of feature vectors.
2. The computer-implemented method of claim 1 , further comprising, after generating the output data:
performing one or more of (i) reconstructing an audio signal from the sequence of feature vectors, (ii) performing speech recognition using the sequence of feature vectors, or (iii) performing speaker verification or speaker identification using the sequence of feature vectors.
3. The computer-implemented method of claim 1 , wherein computing a sequence of feature vectors comprises computing a first feature vector from a first portion of the audio signal and computing the first feature vector comprises:
estimating a pitch of the first portion of the audio signal; and
determining a plurality of harmonic amplitudes of the first portion of the audio signal using the estimated pitch,
wherein the first feature vector comprises the plurality of harmonic amplitudes.
4. The computer-implemented method of claim 1 , wherein the element j of the first feature vector is selected randomly or pseudo-randomly.
5. The computer-implemented method of claim 1 , further comprising adding the spread value to (i) element j+1 of a feature vector subsequent to the first feature vector, or (ii) element j of a feature vector after the subsequent feature vector.
6. The computer-implemented method of claim 1 , wherein:
performing a spreading operation for an element of a feature vector comprises computing a second spread value by multiplying a value of the element by the scale factor, and adding the second spread value to at least two adjacent elements, wherein the adjacent element may be in the feature vector or an adjacent feature vector;
the method further comprises sequentially performing the spreading operation for a plurality of elements of a plurality of feature vectors.
7. The computer-implemented method of claim 1 , wherein obtaining the low threshold and the high threshold comprises:
obtaining a first value indicative of a noise level in the audio signal; and
determining the low threshold and the high threshold using the first value.
8. The computer-implemented method of claim 1 , wherein changing the value of the element comprises changing the value of the element to zero.
9. The computer-implemented method of claim 1 , wherein determining that the first element of the row is part of the continuous sequence of elements of the row, wherein (a) at least one element of the sequence is greater than the high threshold and (b) all elements of the sequence are greater than the low threshold comprises:
processing the elements of the row in a first direction; and
processing the elements of the row in a second direction.
10. A device comprising:
at least one processor;
a memory device including instructions operable to be executed by the at least one processor to configure the device to:
obtain a sequence of feature vectors from an audio signal, wherein each feature vector of the sequence is computed from a portion of the signal;
obtain a low threshold and a high threshold;
modify the sequence of feature vectors by performing a hysteresis operation on a row of the sequence of feature vectors, wherein the row comprises element k of each feature vector of the sequence of feature vectors, and wherein the hysteresis operation comprises:
initializing all of the elements of the row to a first state,
determining that a first element of the row is part of a continuous sequence of elements of the row, wherein (a) at least one element of the sequence is greater than the high threshold and (b) all elements of the sequence are greater than the low threshold,
changing the state of the first element to a second state, and
for each element of the row having the first state, changing a value of the element;
further modify the sequence of feature vectors by:
obtaining a scale factor between 0 and 1,
selecting element j of a first feature vector,
determining a spread value by multiplying the scale factor by a value of the element j of the first feature vector, and
adding the spread value to at least two adjacent elements, wherein each of the two adjacent elements is at least one of (i) in an element adjacent to element j in the first feature vector or (ii) in a second feature vector adjacent to the first feature vector; and
generate output data including the sequence of feature vectors.
11. The device of claim 10 , wherein the instructions further configure the device to:
perform one or more of (i) reconstructing an audio signal from the sequence of feature vectors, (ii) perform speech recognition using the sequence of feature vectors, or (iii) perform speaker verification or speaker identification using the sequence of feature vectors.
12. The device of claim 10 , wherein the instructions further configure the device to:
compute a first feature vector from a first portion of the signal, wherein computing the first feature vector comprises:
estimating a pitch of the first portion of the signal; and
determining a plurality of harmonic amplitudes of the first portion of the signal using the estimated pitch,
wherein the first feature vector comprises the plurality of harmonic amplitudes.
13. The device of claim 10 , wherein the element j of the first feature vector is selected randomly or pseudo-randomly.
14. The device of claim 10 , wherein the instructions further configure the device to:
add the spread value to (i) element j+1 of a feature vector subsequent to the first feature vector, or (ii) element j of a feature vector after the subsequent feature vector.
15. A non-transitory computer readable medium having stored thereon instructions to configure a computing device to:
obtain a sequence of feature vectors from an audio signal, wherein each feature vector of the sequence is computed from a portion of the signal;
obtain a low threshold and a high threshold;
modify the sequence of feature vectors by performing a hysteresis operation on a row of the sequence of feature vectors, wherein the row comprises element k of each feature vector of the sequence of feature vectors, and wherein the hysteresis operation comprises:
initializing all of the elements of the row to a first state,
determining that a first element of the row is part of a continuous sequence of elements of the row, wherein (a) at least one element of the sequence is greater than the high threshold and (b) all elements of the sequence are greater than the low threshold,
changing the state of the first element to a second state, and
for each element of the row having the first state, changing a value of the element;
further modify the sequence of feature vectors by:
obtaining a scale factor between 0 and 1,
selecting element j of a first feature vector,
determining a spread value by multiplying the scale factor by a value of the element j of the first feature vector, and
adding the spread value to at least two adjacent elements, wherein each of the two adjacent elements is at least one of (i) in an element adjacent to element j in the first feature vector or (ii) in a second feature vector adjacent to the first feature vector; and
generate output data including the sequence of feature vectors.
16. The non-transitory computer readable medium of claim 15 , wherein:
performing a spreading operation for an element of a feature vector comprises computing a second spread value by multiplying a value of the element by the scale factor, and adding the second spread value to at least two adjacent elements, wherein the adjacent element may be in the feature vector or an adjacent feature vector; and
the non-transitory computer readable medium further configures the computing device to:
sequentially perform the spreading operation for a plurality of elements of a plurality of feature vectors.
17. The non-transitory computer readable medium of claim 15 , further configuring the computing device to:
obtain a first value indicative of a noise level in the audio signal; and
determine the low threshold and the high threshold using the first value.
18. The non-transitory computer readable medium of claim 15 , wherein changing the value of the element comprises changing the value of the element to zero.
19. The non-transitory computer readable medium of claim 15 , wherein determining that the first element of the row is part of the continuous sequence of elements of the row, wherein (a) at least one element of the sequence is greater than the high threshold and (b) all elements of the sequence are greater than the low threshold comprises:
processing the elements of the row in a first direction; and
processing the elements of the row in a second direction.
20. The non-transitory computer readable medium of claim 15 , further configuring the computing device to:
perform a normalization step after adding the spread value to other elements.