Analysis and visualization tool with combined processing of structured and unstructured service event data
An apparatus comprises a processing platform configured to implement an analysis and visualization tool for combined processing of structured and unstructured service event data. The structured service event data comprises service event data stored in one or more structured data fields of a service events database, and the unstructured service event data includes documents comprising unstructured text data of the service events database. The analysis and visualization tool is associated with a clustering module that assigns each of the documents to one or more clusters corresponding to respective topics. The analysis and visualization tool comprises an interface that permits selection of one or more of the structured data fields, and a visualization module configured to generate at least one visualization as a function of the selected one or more structured data fields and particular ones of the cluster topics that relate to the selected one or more structured data fields.
1. An apparatus comprising:
a processing platform comprising one or more processing devices each comprising a processor coupled to a memory, the processing platform being configured for combined processing of structured and unstructured service event data;
the structured service event data comprising service event data stored in one or more structured data fields of a service events database;
the unstructured service event data including documents comprising unstructured text data of the service events database, the unstructured service event data comprising unstructured service request summaries including one or more problem summaries and one or more corresponding solution summaries;
wherein the processing platform is configured:
to apply preprocessing to the unstructured text data by:
constructing one or more term indexes of the unstructured text data based at least in part on associations between the unstructured service request summaries and structured data in one or more of the structured data fields of the service events database;
generating, for a domain comprising the documents, one or more in-domain dictionaries utilizing the one or more term indexes;
processing the one or more in-domain dictionaries to construct a topic model; and
automatically determining at least a subset of a plurality of topics automatically utilizing the topic model, as sets of related terms from the unstructured text data, without reference to a set of rules characterizing predefined topics;
to assign each of the documents to one or more of a plurality of clusters corresponding to respective topics;
to provide an interface configured to permit selection of one or more of the structured data fields; and
to generate at least one visualization as a function of the selected one or more structured data fields and particular ones of the cluster topics that relate to the selected one or more structured data fields.
2. The apparatus of claim 1 wherein selection of the one or more structured data fields via the interface activates a structured data filter which adapts the visualization in accordance with the selected one or more structured data fields.
3. The apparatus of claim 1 wherein the interface is further configured to permit text search of the structured data and further wherein said at least one generated visualization is modified based on a result of the text search.
4. The apparatus of claim 3 wherein utilization of the text search via the interface activates a text search filter which adapts the visualization in accordance with text entered in a text search field of the interface.
5. The apparatus of claim 1 wherein the generated visualization comprises a topic view visualization for the selected one or more structured data fields.
6. The apparatus of claim 5 wherein the topic view visualization comprises a topic histogram indicating a number of documents for each of a plurality of topics that relate to the one or more selected structured data fields.
7. The apparatus of claim 5 wherein the topic view visualization comprises at least one of a unigram view illustrating frequencies of respective topics in the selected one or more structured data fields and a bigram view illustrating frequencies of respective pairs of topics in the selected one or more structured data fields.
8. The apparatus of claim 1 wherein the generated visualization comprises a combined topic and structured data field view visualization for the selected one or more structured data fields.
9. The apparatus of claim 8 wherein the combined topic and structured data field view visualization comprises a topic histogram indicating a frequency of topic occurrence for each of a plurality of possible values of a given selected structured data field.
10. The apparatus of claim 9 wherein the given selected structured data field comprises one of a problem code field, a location field and a date field.
11. An information processing system comprising the apparatus of claim 1 .
12. A method comprising:
obtaining structured service event data comprising service event data stored in one or more structured data fields of a service events database;
obtaining unstructured service event data including documents comprising unstructured text data of the service events database, the unstructured service event data comprising unstructured service request summaries including one or more problem summaries and one or more corresponding solution summaries;
apply preprocessing to the unstructured text data by:
constructing one or more term indexes of the unstructured text data based at least in part on associations between the unstructured service request summaries and structured data in one or more of the structured data fields of the service events database;
generating, for a domain comprising the documents, one or more in-domain dictionaries utilizing the one or more term indexes;
processing the one or more in-domain dictionaries to construct a topic model; and
automatically determining at least a subset of a plurality of topics automatically utilizing the topic model, as sets of related terms from the unstructured text data, without reference to a set of rules characterizing predefined topics;
assigning each of the documents to one or more of a plurality of clusters corresponding to respective topics;
permitting selection of one or more of the structured data fields via an interface; and
generating at least one visualization as a function of the selected one or more structured data fields and particular ones of the cluster topics that relate to the selected one or more structured data fields;
wherein said obtaining structured and unstructured service event data, assigning, permitting and generating are performed by a processing platform comprising one or more processing devices.
13. The method of claim 12 wherein generating at least one visualization comprises generating a topic view visualization for the selected one or more structured data fields.
14. The method of claim 12 wherein generating at least one visualization comprises generating a combined topic and structured data field view visualization for the selected one or more structured data fields.
15. A non-transitory processor-readable storage medium having program code of one or more software programs embodied therein, wherein the program code when executed by at least one processing device of a processing platform causes the processing device:
to obtain structured service event data comprising service event data stored in one or more structured data fields of a service events database;
to obtain unstructured service event data including documents comprising unstructured text data of the service events database, the unstructured service event data comprising unstructured service request summaries including one or more problem summaries and one or more corresponding solution summaries;
to apply preprocessing to the unstructured text data by:
constructing one or more term indexes of the unstructured text data based at least in part on associations between the unstructured service request summaries and structured data in one or more of the structured data fields of the service events database;
generating, for a domain comprising the documents, one or more in-domain dictionaries utilizing the one or more term indexes;
processing the one or more in-domain dictionaries to construct a topic model; and
automatically determining at least a subset of a plurality of topics automatically utilizing the topic model, as sets of related terms from the unstructured text data, without reference to a set of rules characterizing predefined topics;
to assign each of the documents to one or more of a plurality of clusters corresponding to respective topics;
to permit selection of one or more of the structured data fields via an interface; and
to generate at least one visualization as a function of the selected one or more structured data fields and particular ones of the cluster topics that relate to the selected one or more structured data fields.
16. The processor-readable storage medium of claim 15 wherein the storage medium comprises at least one of an electronic memory and a storage disk.
17. The processor-readable storage medium of claim 15 wherein the program code when executed by at least one processing device further causes said processing device:
to generate said at least one visualization as at least one of a topic view visualization for the selected one or more structured data fields and a combined topic and structured data field view visualization for the selected one or more structured data fields.