Noise mitigation using machine learning
This disclosure relates to solutions for eliminating undesired audio artifacts, such as background noises, on an audio channel. A process for implementing the technology can include receiving a set of audio segments, analyzing the segments using a first ML model to identify a first probability of unwanted background noises in the segments, and if the first probability exceeds a threshold, analyzing the segments using a second ML model to determine a second probability that the one or more background features exist in the segments. In some aspects, the process can include attenuating audio artifacts in the segments, if the second probability exceeds a second threshold. In some implementations, dynamic time stretching and shrinking can be applied to the noise attenuation. Systems and machine-readable media are also provided.
1. A noise attenuation system comprising:
one or more processors; and
a memory coupled to the one or more processors, with instructions stored therein which, when executed by the one or more processors, cause the one or more processors to perform operations comprising:
receiving a set of audio segments from an audio capture device, the set of audio segments including a plurality of features,
analyzing the set of audio segments to yield a plurality of scores, comprising:
first applying a first machine learning model to determine a first probability that the set of audio segments includes one or more undesired features of the plurality of features;
second applying, in response to the first probability exceeds a first threshold, a second machine learning model to determine a second probability that the set of audio segments includes one or more undesired features of the plurality of features; and
attenuating, in response to the second probability exceeds a second threshold higher than the first threshold, at least one of the plurality of features.
2. The noise attenuation system of claim 1 , wherein the attenuating includes:
identifying an audio artifact from among the plurality of features,
attenuating a volume of the audio artifact, and
buffering the at least one of the plurality of features.
3. The noise attenuation system of claim 2 , wherein the operations include retrieving the at least one of the plurality of features from the buffer for insertion into an audio channel using time squeezing.
4. The noise attenuation system of claim 1 , wherein the operations include receiving another set of audio segments from the audio capture device when the second probability does not exceed the second threshold.
5. The noise attenuation system of claim 1 ,
wherein,
the first probability is determined using a logistic regression model, and
the second probability is determined using a deep neural network.
6. The noise attenuation system of claim 1 , wherein the operations include receiving another set of audio segments from the audio capture device when the first probability does not exceed the first threshold.
7. A computer-implemented method comprising:
receiving a set of audio segments from an audio capture device, the set of audio segments including a plurality of features;
analyzing the set of audio segments to yield a plurality of scores, comprising:
first applying a first machine learning model to determine a first probability that the set of audio segments includes one or more undesired features of the plurality of features;
second applying, in response to the first probability exceeding a first threshold, a second machine learning model to determine a second probability that the set of audio segments includes one or more undesired features of the plurality of features; and
attenuating, in response to the second probability exceeding a second threshold higher than the first threshold, the at least one of the plurality of features based on at least one of the plurality of scores and a threshold.
8. The computer-implemented method of claim 7 , wherein the attenuating includes:
identifying an audio artifact from among the plurality of features,
attenuating a volume of the audio artifact, and
buffering the at least one of the plurality of features.
9. The computer-implemented method of claim 8 , further comprising:
retrieving the at least one of the plurality of features from the buffer for insertion into an audio channel using time squeezing.
10. The computer-implemented method of claim 7 , further comprising:
receiving another set of audio segments from the audio capture device when the second probability does not exceed the second threshold.
11. The computer-implemented method of claim 7 ,
wherein,
the first probability is determined using a logistic regression model, and
the second probability is determined using a deep neural network.
12. The computer-implemented method of claim 7 , wherein further comprising receiving another set of audio segments from the audio capture device when the first probability does not exceed the first threshold.
13. A non-transitory computer-readable storage medium comprising instructions stored therein which, when executed by one or more processors, cause the one or more processors to perform operations comprising:
receiving a set of audio segments from an audio capture device, the set of audio segments including a plurality of features;
analyzing the set of audio segments to yield a plurality of scores, comprising:
first applying a first machine learning model to determine a first probability that the set of audio segments includes one or more undesired features of the plurality of features;
second applying, in response to the first probability exceeds a first threshold, a second machine learning model to determine a second probability that the set of audio segments includes one or more undesired features of the plurality of features; and
attenuating, in response to the second probability exceeds a second threshold higher than the first threshold, the at least one of the plurality of features based on at least one of the plurality of scores and a threshold.
14. The non-transitory computer-readable storage medium of claim 13 , wherein the attenuating includes:
identifying an audio artifact from among the plurality of features,
attenuating a volume of the audio artifact, and
buffering the at least one of the plurality of features.
15. The non-transitory computer-readable storage medium of claim 14 , wherein the operations include retrieving the at least one of the plurality of features from the buffer for insertion into an audio channel using time squeezing.
16. The non-transitory computer-readable storage medium of claim 14 , wherein the operations include receiving another set of audio segments from the audio capture device when the first probability does not exceed the first threshold.
17. The non-transitory computer-readable storage medium of claim 13 , wherein the operations include receiving another set of audio segments from the audio capture device when the second probability does not exceed the second threshold.