Determination of QOE in encrypted video streams using supervised learning

1. A method for determining quality of experience parameters of an encrypted video stream received at a client device, the method comprising:

extracting, from one or more encrypted video streams sent over a network from a content server to a plurality of client devices, a first instance of at least one stream-related feature associated with at least one of throughput of the one or more encrypted video streams, peak density of the one or more encrypted video streams, or quantity of the one or more encrypted video streams;

determining a first instance of at least one quality-related label of a plurality of quality-related labels based on applying a trained classifier to the first instance of the at least one stream-related feature extracted from the one or more encrypted video streams, the trained classifier being trained based on a training dataset, validated based on a validation dataset, and tested based on a test dataset, wherein each of the plurality of quality-related labels corresponds to a respective quality of experience parameter of the quality of experience parameters of the encrypted video stream received at the client device, and wherein the at least one quality-related label is associated with playback quality of the one or more encrypted video streams;

changing one or more first network parameters associated with the network to perform capacity enhancements at a radio access network used to connect the content server to a first subset of the plurality of client devices via the network or at a network core used to connect the content server to the first subset of the plurality of client devices via the network;

extracting, from the one or more encrypted video streams sent from the content server to the first subset of the plurality of client devices, a second instance of the at least one stream-related feature after the changing of the one or more first network parameters;

determining a second instance of the at least one quality-related label based on applying the trained classifier to the second instance of the at least one stream-related feature extracted after the changing of the one or more first network parameters;

analyzing the first instance of the at least one quality-related label determined before the changing of the one or more first network parameters and the second instance of the at least one quality-related label determined after the changing of the one or more first network parameters to determine a measurement of an impact of the changing of the one or more first network parameters on the quality of experience parameters of the encrypted video stream received at the first subset of the plurality of client devices; and

changing one or more second network parameters associated with the network used to connect the content server to a second subset of the plurality of client devices in view of the measurement of the impact of the changing of the one or more first network parameters on the quality of experience parameters of the encrypted video stream received at the first subset of the plurality of client devices, wherein the second subset is different from the first subset.

2. The method of claim 1 further comprising:

extracting, from the one or more encrypted video streams sent over the network from the content server to the plurality of client devices, a set of stream-related features;

extracting, from one or more client devices of the plurality of client devices, a set of quality-related labels corresponding to the quality of experience parameters of the encrypted video stream received at the one or more client devices; and

training a classifier using the training dataset to obtain the trained classifier, wherein the training dataset comprises the set of stream-related features and the set of quality-related labels.

3. The method of claim 2 , further comprising:

validating the trained classifier using the validation dataset; and

testing the trained classifier using the test dataset, wherein the validation dataset and the test dataset each comprise a respective set of stream-related features and a respective set of quality-related labels.

4. The method of claim 2 , wherein the one or more client devices comprise at least one of controlled user equipment (UEs) or UE simulators.

5. The method of claim 1 , wherein the at least one stream-related feature comprises at least one of:

an effective throughput;

a quartile of throughput;

at least one of a high, mid, or low peak density; or

a total number of the one or more encrypted video streams.

6. The method of claim 5 , wherein the at least one quality-related label comprises one or more of a rebuffering time percentage, or a streaming reproduction cut-off ratio.

7. A system for determining quality of experience parameters of an encrypted video stream received at a client device, the system comprising:

a plurality of client devices configured to receive over a network from a content server, one or more encrypted video streams;

a classifier trained using a supervised machine learning algorithm and a training data set, validated based on a validation dataset, and tested based on a test dataset, wherein the classifier is configured to be applied to at least one stream-related feature, extracted from the one or more encrypted video streams without obtaining data directly from the client device and without decrypting the one or more encrypted video streams, to determine at least one quality-related label corresponding to the quality of experience parameters of the encrypted video stream of the one or more encrypted video streams received at the client device, wherein the at least one stream-related feature is extracted from the one or more encrypted video streams via a passive tap into the network, wherein the at least one stream-related feature is associated with at least one of throughput of the one or more encrypted video streams, peak density of the one or more encrypted video streams, or quantity of the one or more encrypted video streams, and wherein the at least one quality-related label is associated with playback quality of the one or more encrypted video streams; and

a network operator component configured to:

receive the at least one quality-related label from the classifier;

change one or more first network parameters associated with the network to perform capacity enhancements at a radio access network used to connect the content server to a first subset of the plurality of client devices via the network or at a network core used to connect the content server to the first subset of the plurality of client devices via the network;

analyze the at least one quality-related label determined before and after changing the one or more first network parameters to determine a measurement of an impact of changing the one or more first network parameters on the quality of experience parameters of the encrypted video stream received at the first subset of the plurality of client devices; and

change one or more second network parameters associated with the network used to connect the content server to a second subset of the plurality of client devices in view of the measurement of the impact of changing the one or more first network parameters on the quality of experience parameters of the encrypted video stream received at the first subset of the plurality of client devices, wherein the second subset is different from the first subset.

8. The system of claim 7 , wherein the classifier is trained using the training data set comprising a set of stream-related features extracted from the one or more encrypted video streams and a set of quality-related labels received from one or more client devices of the plurality of client devices.

9. The system of claim 8 , wherein the one or more client devices comprise at least one of controlled user equipment (UEs) or UE simulators.

10. The system of claim 8 , wherein the set of stream-related features comprise at least one of:

an effective throughput;

a quartile of throughput; or

at least one of a high, mid, or low peak density; or a total number of the one or more encrypted video streams.

11. The system of claim 8 , wherein the set of quality-related labels comprises one or more of a rebuffering time percentage, or a streaming reproduction cut-off ratio.

12. The system of claim 7 , further comprising a feature extraction module comprising the passive tap that is configured to extract the at least one stream-related feature from the one or more encrypted video streams.

13. One or more computer-readable storage media having computer-readable instructions stored thereon, which, when executed by a processor, perform operations comprising:

extracting, from one or more encrypted video streams sent over a network from a content server to a plurality of client devices, a first instance of at least one stream-related feature associated with at least one of throughput of the one or more encrypted video streams, peak density of the one or more encrypted video streams, or quantity of the one or more encrypted video streams without obtaining data directly from the plurality of client devices and without decrypting the one or more encrypted video streams, wherein the extracting of the first instance of the at least one stream-related feature is via a passive tap into the network;

determining a first instance of at least one quality-related label of a plurality of quality-related labels based on applying a trained classifier to the first instance of the at least one stream-related feature extracted from the one or more encrypted video streams, the trained classifier being trained based on a training dataset, validated based on a validation dataset, and tested based on a test dataset, wherein each of the plurality of quality-related labels corresponds to a respective quality of experience parameter of quality of experience parameters of an encrypted video stream received at a client device, and wherein the at least one quality-related label is associated with playback quality of the one or more encrypted video streams;

changing one or more first network parameters associated with the network to perform capacity enhancements at a radio access network used to connect the content server to a first subset of the plurality of client devices via the network or at a network core used to connect the content server to the first subset of the plurality of client devices via the network;

extracting, from the one or more encrypted video streams sent from the content server to the first subset of the plurality of client devices, a second instance of the at least one stream-related feature after the changing of the one or more first network parameters;

determining a second instance of the at least one quality-related label based on applying the trained classifier to the second instance of the at least one stream-related feature extracted after the changing of the one or more first network parameters;

analyzing the first instance the at least one quality-related label determined before the changing of the one or more first network parameters and the second instance of the at least one quality-related label determined after the changing of the one or more first network parameters to determine a measurement of an impact of the changing of the one or more first network parameters on the quality of experience parameters of the encrypted video stream received at the first subset of the plurality of client devices; and

changing one or more second network parameters associated with the network used to connect the content server to a second subset of the plurality of client devices in view of the measurement of the impact of the changing of the one or more first network parameters on the quality of experience parameters of the encrypted video stream received at the first subset of the plurality of client devices, wherein the second subset is different from the first subset.

14. The one or more computer-readable storage media of claim 13 , wherein the operations further comprise:

extracting, from the one or more encrypted video streams sent over the network from the content server to the plurality of client devices, a set of stream-related features;

extracting, from one or more client devices of the plurality of client devices, a set of quality-related labels corresponding to the quality of experience parameters of the encrypted video stream received at the one or more client devices; and

training a classifier using the training dataset to obtain the trained classifier, wherein the training dataset comprises the set of stream-related features and the set of quality-related labels.

15. The one or more computer-readable storage media of claim 14 , wherein the operations further comprise:

validating the trained classifier using the validation dataset; and

testing the trained classifier using the test dataset, wherein the validation dataset and the test dataset each comprise a respective set of stream-related features and a respective set of quality-related labels.

16. The one or more computer-readable storage media of claim 14 , wherein the one or more client devices comprise at least one of controlled user equipment (UEs) or UE simulators.

17. The one or more computer-readable storage media of claim 14 , wherein:

the set of stream-related features comprises at least one of:

an effective throughput;

a quartile of throughput;

a high, mid and/or low peak density; or

a total number of the one or more encrypted video streams; and

the set of quality-related labels comprise at least one of:

a rebuffering time percentage;

or

a streaming reproduction cut-off ratio.

18. The method of claim 1 , wherein:

the extracting of the first instance of the at least one stream-related feature is without obtaining data directly from the client device and without decrypting the one or more encrypted video streams; and

the extracting of the first instance of the at least one stream-related feature is via a passive tap into the network.

19. The method of claim 18 , wherein:

the extracting of the second instance of the at least one stream-related feature is without obtaining data directly from the client device and without decrypting the one or more encrypted video streams; and

the extracting of the second instance of the at least one stream-related feature is via the passive tap into the network.

20. The method of claim 1 , wherein the measurement of the impact is a metric associated with a difference between the first instance of the at least one quality-related label and the second instance of the at least one quality-related label.