Data analysis using natural language processing to obtain insights relevant to an organization
Methods and apparatuses for generating insights for improving an organization from unstructured and structured data. Natural language processing is employed to process the aggregated data from various data sources to create topics and the features that impact the topics. These topics are then used to generate recommendations to improve customer satisfaction with the organization.
1. A computer-implemented method for obtaining insights from unstructured data pertaining to an organization, comprising:
storing information in a standardized format about a set of topics in a network-based non-transitory storage devices having a collection of scores stored thereon;
providing remote access to users over a network so any one of the users can update information about at least one of the topics in real time through a graphical user interface, wherein the one of the users provides the updated information in a non-standardized comment, wherein the non-standardized comment includes an overall score and unstructured data including text;
converting, by a server, the non-standardized information into a standardized format by:
processing the unstructured data using natural language processing to generate a set of topics;
identifying a polarity and a severity of the topics in each individual comment normalizing the severity for each topic of the topics by a multivariate analysis of all topics in the comment against the overall score for the comment:
wherein a sentiment is associated with a polarity;
calculating a topic level factor for each individual topic by aggregating each individual topic by the polarity and normalized severity across all users;
and adjusting the topic level factor based upon the total number of times a given topic is mentioned in the comments and multiplied by a weight based upon the severity of the sentiment and the topic input from the specific organization;
ranking the topics, from highest to lowest, based upon their adjusted calculated topic factor identifying a set number of the highest ranked topics; and identifying features that are correlated to the identified highest ranked topics;
automatically generating a message containing the updated information about the topic by the server whenever updated information has been stored; and
transmitting the message to the organization over the computer network in real time, so that the organization has immediate access to up-to-date topic information.
2. The computer-implemented method of claim 1 , wherein the topics are generated using an organization specific.
3. The computer-implemented method of claim 1 , wherein the calculating the score for the comment is an average of the topics by severity and polarity.
4. The computer-implemented method of claim 1 , further comprising adjusting the severity for each topic, aggregated over all comments, by a number of occurrences, an average ranking of the organization, and feedback from the organization.
5. The computer-implemented method of claim 1 , wherein the identifying features is performed by identifying common topics in multiple comments.
6. The computer-implemented method of claim 1 , further comprising outputting the highest ranked topics and features correlated to the highest ranked topics to the organization.
7. The computer-implemented method of claim 1 , further comprising generating at least one recommendation to improve the identified features.
8. The computer-implemented method of claim 7 , wherein the recommendation is generated by a third party reviewer.
9. The computer-implemented method of claim 7 , wherein the recommendation is generated by the computer system comparing the identified features to a plurality of recommendations and then curating the plurality of recommendations by a third party reviewer.
10. A nonvolatile memory product which when implemented on a computer system causes the computer system to the steps of:
storing information in a standardized format about a set of topics in a network-based non-transitory storage devices having a collection of scores stored thereon;
providing remote access to users over a network so any one of the users can update information about at least one of the topics in real time through a graphical user interface, wherein the one of the users provides the updated information in a non-standardized comment, wherein the non-standardized comment includes an overall score and unstructured data including text;
converting, by a server, the non-standardized information into a standardized format by:
processing the unstructured data using natural language processing to generate a set of topics;
identifying a polarity and a severity of the topics in each individual comment normalizing the severity for each topic of the topics by a multivariate analysis of all topics in the comment against the overall score for the comment:
wherein a sentiment is associated with a polarity;
calculating a topic level factor for each individual topic by aggregating each individual topic by the polarity and normalized severity across all users;
and adjusting the topic level factor based upon the total number of times a given topic is mentioned in the comments and multiplied by a weight based upon the severity of the sentiment and the topic input from the specific organization;
ranking the topics, from highest to lowest, based upon their adjusted calculated topic factor identifying a set number of the highest ranked topics; and identifying features that are correlated to the identified highest ranked topics;
automatically generating a message containing the updated information about the topic by the server whenever updated information has been stored; and
transmitting the message to the organization over the computer network in real time, so that the organization has immediate access to up-to-date topic information.
11. The nonvolatile memory product of claim 10 , wherein the topics are generated using an organization specific.
12. The nonvolatile memory product of claim 10 , wherein the calculating the score for the comment is an average of the topics by severity and polarity.
13. The nonvolatile memory product of claim 10 , further comprising adjusting the severity for each topic, aggregated over all comments, by a number of occurrences, an average ranking of the organization, and feedback from the organization.
14. The nonvolatile memory product of claim 10 , wherein the identifying features is performed by identifying common topics in multiple comments.
15. The nonvolatile memory product of claim 10 , further comprising causing the computer system to perform the step of outputting the highest ranked topics and features correlated to the highest ranked topics to the organization.
16. The nonvolatile memory product of claim 10 , further comprising causing the computer system to perform the step of generating at least one recommendation to improve the identified features.
17. The nonvolatile memory product of claim 16 , wherein the recommendation is generated by a third party reviewer.
18. The nonvolatile memory product of claim 16 , wherein the recommendation is generated by the computer system comparing the identified features to a plurality of recommendations and then curating the plurality of recommendations by a third party reviewer.