Methods and apparatus for signal quality analysis
View Patent ↗A non-intrusive objective speech quality assessment is performed on a degraded speech signal. The methods are well suited for systems where random and bursty packet losses may occur and/or packet stream regeneration may also occur prior to speech signal quality assessment. In one embodiment received packetized speech is analyzed to determine to an overall final signal quality score. A limited set of trained neural networks, e.g., 5, corresponding to different signal features, each determine a signal feature quality score. A trained joint quality score determination module determines a joint quality score based on the signal feature quality scores. Packet loss is estimated based on received packet header information and/or detected gap durations. The determined joint quality score is adjusted, based on estimated packet loss information obtained from examining the speech signal, network level statistics and/or codec parameters to generate the final quality score.
1. A method of measuring the quality of speech communicated using packets, the method comprising:
operating a plurality of N independent neural networks, each neural network processing one of N separate sets of speech signal feature measurements corresponding to a speech signal recovered from packets communicated via a communications network to generate from the processed set of speech signal feature measurements a signal feature quality score, each of the N different neural networks corresponding to a different one of the N sets of speech signal feature measurements, said N neural networks generating N signal feature quality scores, each one of the N signal feature quality scores corresponding to a different one of N speech signal features;
operating a joint quality score determination neural network to generate a joint signal feature quality score from said N signal feature quality scores;
adjusting said joint quality score based on packet loss information and codec type to generate a final overall quality score for said recovered speech signal wherein different types of codecs correspond to different packet loss robustness factors; and
wherein said N sets of separate speech signal feature measurements include at least two sets of speech signal measurements from a group of sets of speech signal feature measurements including: a set of signal to noise ratio measurements, a set of spectral clarity measurements, a set of linear prediction skew measurements, a set of linear prediction kurtosis measurements and a set of pitch average measurements.
2. The method of claim 1 ,
wherein said plurality of N neural networks includes at least a first neural network corresponding to a first speech signal feature and a second neural network corresponding to a second speech signal feature;
wherein said N sets of speech feature signal measurements include a first set of speech signal feature measurements generated from recovered speech signal and a second set of speech signal feature measurements generated from recovered speech signal the method further comprising:
wherein said operating a plurality of N neural networks includes:
operating the first neural network to generate a first signal feature quality score from the first set of speech signal feature measurements; and
operating the second neural network to generate a second signal feature quality score from the second set of speech signal feature measurements.
3. The method of claim 2 ,
wherein N is 5; and
wherein said N sets of speech signal feature measurements include speech signal measurements from the group of five sets of speech signal feature measurements including: i) a set of signal to noise ratio measurements, ii) a set of spectral clarity measurements, iii) a set of linear prediction skew measurements, iv) a set of linear prediction kurtosis measurements and v) a set of pitch average measurements.
4. The method of claim 1 , further comprising:
performing feature extraction operations on said recovered speech signal to generate speech signal feature measurements corresponding to different points in time, said feature extraction operations generating said N different sets of speech signal feature measurements.
5. The method of claim 1 , wherein said adjusted joint quality score is an approximation of a MOS score (mean opinion score) adjusted to take into consideration packet loss corresponding to signal portions which are non-silent signal portions.
6. The method of claim 1 , wherein said packet loss information includes an estimate of a number of packets lost in the communication of said speech signal via said communications network; and wherein said packet loss information includes packet loss location information.
7. The method of claim 6 , further comprising:
receiving said packets communicated via the communications network;
processing said packets to recover said speech signal; and
generating the estimate of the number of packets lost in the communication of said speech signal from the recovered speech signal.
8. The method of claim 7 , wherein generating the estimate of the number of packets lost in the communication of said speech signal includes:
detecting a falling edge corresponding to a signal energy drop in said recovered speech signal;
detecting a rising edge corresponding to a signal energy increase in said recovered speech signal; and
determining an amount of time between said detected falling edge and said detected rising edge during which the signal energy level of said recovered speech signal stays below an energy level threshold level.
9. The method of claim 8 , wherein generating the estimate of the number of packets lost in the communication of said speech signal further includes:
determining the estimate of the number of packets lost in the communication of the speech signal based on an estimated number of packets which would have been used to communicate a portion of said speech signal having a duration equal to the determined amount of time between said detected falling edge of said recovered speech signal and said detected rising edge of said recovered speech signal.
10. The method of claim 1 , further comprising:
prior to operating said N neural networks, training at least one of said N neural networks using a set of speech signal feature data and a corresponding known signal feature set quality score which was obtained through subjective human testing.
11. The method of claim 10 , further comprising:
prior to operating said joint quality score determination neural network to generate a joint quality score, training said joint quality score determination neural network using a known set of signal feature quality scores and a corresponding predetermined joint signal feature set quality score.
12. The method of claim 11 , wherein said predetermined joint quality scores are generated from ITU (International Telecommunication Union) Perceptual Evaluation of Speech Quality test P.563.
13. An apparatus for measuring the quality of speech communicated using packets, comprising:
a plurality of N independent neural networks, each neural network being configured to process one of N separate sets of speech signal feature measurements corresponding to a speech signal recovered from packets communicated via a communications network to generate from the processed set of speech signal feature measurements a signal feature quality score, each of the N different neural networks corresponding to a different one of the N sets of speech signal feature measurements, said N neural networks generating N signal feature quality scores, each one of the N signal feature quality scores corresponding to a different one of N speech signal features;
a joint quality score determination neural network configured to generate a joint signal feature quality score from said N signal feature quality scores;
a score adjustment module configured to adjust said joint quality score based on packet loss information and codec type to generate a final overall quality score for said recovered speech signal wherein different types of codecs correspond to different packet loss robustness factors; and
wherein said N sets of separate speech signal feature measurements include at least two sets of speech signal measurements from a group of sets of speech signal feature measurements including: a set of signal to noise ratio measurements, a set of spectral clarity measurements, a set of linear prediction skew measurements, a set of linear prediction kurtosis measurements and a set of pitch average measurements.
14. The apparatus of claim 13 ,
wherein said plurality of N neural networks includes at least a first neural network corresponding to a first speech signal feature and a second neural network corresponding to a second speech signal feature;
wherein said N sets of speech feature signal measurements include a first set of speech signal feature measurements generated from recovered speech signal and a second set of speech signal feature measurements generated from recovered speech signal; and
wherein said first neural network is configured to generate a first signal feature quality score from the first set of speech signal feature measurements, and said second neural network is configured to generate a second signal feature quality score from the second set of speech signal feature measurements.
15. The apparatus of claim 13 , further comprising:
a feature extraction module configured to perform feature extraction operations on said recovered speech signal to generate speech signal feature measurements corresponding to different points in time, said feature extraction module generating said N different sets of speech signal feature measurements.
16. The apparatus of claim 13 , wherein said adjusted joint quality score is an approximation of a MOS score (mean opinion score) adjusted to take into consideration packet loss corresponding to signal portions which are non-silent signal portions.
17. The apparatus of claim 13 , further comprising:
an interface including a receiver configured to receive said packets communicated via the communications network;
a processing module configure to process said packets to recover said speech signal; and
a packet loss estimate generation module configured to generate, from the recovered speech signal, an estimate of the number of packets lost in the communication of said speech signal via said communications network.