System and method for detecting portability of sentiment analysis system based on changes in a sentiment confidence score distribution

1. A computer-implemented method for facilitating portability analysis for a sentiment analysis model, the method comprising:

training, by a computer, a sentiment analysis model, which is based on a machine learning technique, on a reference data set of user reviews,

analyzing text of a first review to determine a sentiment expressed in the text of the first review by applying the sentiment analysis model to the first review, wherein the first review is in a first data set comprising a plurality of reviews;

computing a first confidence score that indicates an accuracy of the determined sentiment;

determining a first confidence score distribution for the first data set by applying the sentiment analysis model to a plurality of reviews in the first data set, wherein the first confidence score distribution incorporates the first confidence score;

determining a significance of changes between the first confidence score distribution and a benchmark confidence score distribution by obtaining a recall of a set of p-values derived from application of a Kolmogorov-Smirnov (K-S) test to the first confidence score distribution and the benchmark score distribution, wherein the benchmark confidence score distribution is determined from a benchmark data set for which the sentiment analysis model yields a high accuracy, wherein the first data set and the benchmark data set are non-identical data sets;

determining whether the sentiment analysis model is portable to the first data set based on a comparison of the recall to a portability threshold; and

displaying, via a display device, a recommendation indicating the portability of the sentiment analysis model to the first data set.

2. The method of claim 1 , wherein determining whether the sentiment analysis model is portable to the first data set comprises:

determining whether the significance of changes is less than the portability threshold; and

in response to the significance of changes being greater than or equal to the portability threshold, determining that the sentiment analysis model is portable to the first data set.

3. The method of claim 2 , wherein determining whether the sentiment analysis model is portable to the first data set further comprises, in response to the significance of changes being less than the portability threshold, determining that the sentiment analysis model needs retraining for the first data set.

4. The method of claim 2 , wherein determining whether the significance of changes is less than the portability threshold comprises:

comparing the recall with the portability threshold.

5. The method of claim 1 , wherein analyzing the text of the first review comprises:

determining how phrases in the first review correspond to one or more sentiments expressed in the first review; and

generating a tag indicating whether the first review expresses positive, negative, neutral, or mixed sentiment.

6. The method of claim 1 , wherein the benchmark data set comprises a set of reviews over a first period of time for a business entity, and wherein the first data set is one of:

a set of reviews over a second period of time for the same business entity;

a set of reviews for a second business entity in a same industry as the business entity; and

a set of reviews for a third business entity in a different industry than the business entity.

7. The method of claim 1 , further comprising: determining a first median score for the first confidence score distribution and a second median score for the benchmark confidence score distribution; and in response to the first median score being higher than the second median score, setting the first data set as the benchmark data set for a second data set.

8. A computer system for non-parametric correlation analysis, the system comprising:

a processor; and

a storage device storing instructions that when executed by the processor cause the processor to perform a method, the method comprising:

training, by a computer, a sentiment analysis model, which is based on a machine learning technique, on a reference data set of user reviews,

analyzing text of a first review to determine a sentiment expressed in the text of the first review by applying the sentiment analysis model to the first review, wherein the first review is in a first data set comprising a plurality of reviews;

computing a first confidence score that indicates an accuracy of the determined sentiment;

determining a first confidence score distribution for the first data set by applying the sentiment analysis model to a plurality of reviews in the first data set, wherein the first confidence score distribution incorporates the first confidence score;

determining a significance of changes between the first confidence score distribution and a benchmark confidence score distribution by obtaining a recall of a set of p-values derived from application of a Kolmogorov-Smirnov (K-S) test to the first confidence score distribution and the benchmark score distribution, wherein the benchmark confidence score distribution is determined from a benchmark data set for which the sentiment analysis model yields a high accuracy, wherein the first data set and the benchmark data set are non-identical data sets;

determining whether the sentiment analysis model is portable to the first data set based on a comparison of the recall to a portability threshold; and

displaying, via a display device, a recommendation indicating the portability of the sentiment analysis model to the first data set.

9. The computer system of claim 8 , wherein determining whether the sentiment analysis model is portable to the first data set comprises:

determining whether the significance of changes is less than the portability threshold; and

in response to the significance of changes being greater than or equal to the portability threshold, determining that the sentiment analysis model is portable to the first data set.

10. The computer system of claim 9 , wherein determining whether the sentiment analysis model is portable to the first data set further comprises, in response to the significance of changes being less than the portability threshold, determining that the sentiment analysis model needs retraining for the first data set.

11. The computer system of claim 9 , wherein determining whether the significance of changes is less than the portability threshold comprises:

comparing the recall with the portability threshold.

12. The computer system of claim 9 , wherein analyzing the text of the first review comprises:

determining how phrases in the first review correspond to one or more sentiments expressed in the first review; and

generating a tag indicating whether the first review expresses positive, negative, neutral, or mixed sentiment.

13. The computer system of claim 9 , wherein the benchmark data set comprises a set of reviews over a first period of time for a business entity, and wherein the first data set is one of:

a set of reviews over a second period of time for the same business entity,

a set of reviews for a second business entity in a same industry as the business entity; and

a set of reviews for a third business entity in a different industry than the business entity.

14. The computer system of claim 8 , wherein the method further comprises:

determining a first median score for the first confidence score distribution and a second median score for the benchmark confidence score distribution; and

in response to the first median score being higher than the second median score, setting the first data set as the benchmark data set for a second data set.

15. A non-transitory computer-readable storage medium storing instructions that when executed by a computer cause the computer to perform a method, the method comprising:

training, by a computer, a sentiment analysis model, which is based on a machine learning technique, on a reference data set of user reviews, analyzing text of a first review to determine a sentiment expressed in the text of the first review by applying the sentiment analysis model to the first review, wherein the first review is in a first data set comprising a plurality of reviews;

computing a first confidence score that indicates an accuracy of the determined sentiment;

determining a first confidence score distribution for the first data set by applying the sentiment analysis model to a plurality of reviews in the first data set, wherein the first confidence score distribution incorporates the first confidence score;

determining a significance of changes between the first confidence score distribution and a benchmark confidence score distribution by obtaining a recall of a set of p-values derived from application of a Kolmogorov-Smirnov (K-S) test to the first confidence score distribution and the benchmark score distribution, wherein the benchmark confidence score distribution is determined from a benchmark data set for which the sentiment analysis model yields a high accuracy, wherein the first data set and the benchmark data set are non-identical data sets;

determining whether the sentiment analysis model is portable to the first data set based on a comparison of the recall to a portability threshold; and

displaying, via a display device, a recommendation indicating the portability of the sentiment analysis model to the first data set.

16. The storage medium of claim 15 , wherein the benchmark data set comprises a set of reviews over a first period of time for a business entity, and wherein the first data set is one of:

a set of reviews over a second period of time for the same business entity;

a set of reviews for a second business entity in a same industry as the business entity; and

a set of reviews for a third business entity in a different industry than the business entity.

17. The storage medium of claim 15 , wherein determining whether the sentiment analysis model is portable to the first data set comprises:

determining whether the significance of changes is less than the portability threshold;

in response to the significance of changes being greater than or equal to the portability threshold, determining that the sentiment analysis model is portable to the first data set; and

in response to the significance of changes being less than the portability threshold, determining that the sentiment analysis model needs retraining for the first data set.

18. The storage medium of claim 15 , wherein the method further comprises:

determining a first median score for the first confidence score distribution and a second median score for the benchmark confidence score distribution; and

in response to the first median score being higher than the second median score, setting the first data set as the benchmark data set for a second data set.