Determining the effects of new types of impairments on perceived quality of a voice service
View Patent ↗Empirically derived models are used to relate user perception to objectively measurable characteristics, under a criterion of nearly perfect co-variance between predicted MOS values and the variations in the values of the objectively measurable characteristics. In the example context of evaluating packet switched telephony service, the present invention eliminates the need to perform subjective tests to characterize each combination of codec type and transmission protocol.
1. One or more non-transitory computer-readable media having instructions stored thereon configured to cause a computing device to perform operations, the operations comprising:
obtaining objective data regarding a first performance characteristic of a first communication service;
generating a quality index for the first communication service based on the objective data;
determining an acceptable quality index for a second communication service based upon the quality index for the first communication service, where the second communication service exhibits a second performance characteristic not exhibited by the first communication service;
modifying the objective data creating modified objective data to reflect effects of the second performance characteristic using a model associated with the second communication service;
computing a modified quality index from the modified objective data; and
determining whether the second communication service meets the acceptable quality index based on the modified quality index.
2. The non-transitory computer-readable media of claim 1 , wherein the first performance characteristic comprises speech distortion.
3. The non-transitory computer-readable media of claim 1 , wherein the second performance characteristic comprises at least one of packet loss or packet delay.
4. The non-transitory computer-readable media of claim 1 , wherein the first communication service is provided by a circuit switched network and wherein the second communication service is provided by a packet switched network.
5. The non-transitory computer-readable media of claim 1 , wherein the operations further comprise:
establishing a communication session via the second communication service using a codec;
obtaining objective measurements of the first performance characteristic and the second performance characteristic;
transforming the objective measurements into subjective ratings reflective of the first performance characteristic;
altering the subjective ratings creating altered subjective ratings to reflect the effects of the second performance characteristic on the subjective ratings using an initial model;
computing an altered quality index from the altered subjective ratings; and
adjusting the initial model to improve a correlation between the altered quality index and the second performance characteristic.
6. The non-transitory computer-readable media of claim 5 , wherein the quality index comprises at least one of a mean opinion score or a percentage of calls or connections that elicit unusable, difficult, or irritating responses.
7. The non-transitory computer-readable media of claim 1 , wherein the first performance characteristic comprises one or more of low volume, noise, or echo.
8. A method performed by a processing device, the method comprising:
obtaining, by the processing device, objective data regarding a first performance characteristic of a first communication service;
generating, by the processing device, a quality index for the first communication service based on the objective data;
determining, by the processing device, an acceptable quality index for a second communication service based upon the quality index for the first communication service, where the second communication service exhibits a second performance characteristic not exhibited by the first communication service;
modifying, by the processing device, the objective data creating modified objective data to reflect effects of the second performance characteristic using a model associated with the second communication service;
computing, by the processing device, a modified quality index from the modified objective data; and
determining, by the processing device, whether the second communication service meets the acceptable quality index based on the modified quality index.