Method for clustering automation and classification techniques
A method and system for use in an information retrieval system that provides related informational items that can be automatically classified and summarized in a hierarchical manner. The system comprises the steps and means for the presentation and analysis of mined data through the application of four distinct processes, namely, feature selection, clustering, classification and summarization. The implementation of each of these techniques is a single process without regard to order. Individual informational items are analyzed, and relevant features extracted. Individual groups of informational items, otherwise known as clusters, are identified, and rules are determined to allow the assignment of informational items to the previously mentioned groups. Descriptive and meaningful titles are then identified for the groups. Various methods of presenting searched information to an end user are also provided. A method of adaptive clustering is also provided.
1. A computer-implemented mapping method for categorizing a plurality of informational items in an information retrieval system, the method comprising the steps of;
clustering by identifying one or more groupings of the informational items into one or more clusters, wherein said clustering is extended by adding an adaptive method comprising the steps of:
application of said clustering,
statistical evaluation of said clustering output for quality,
usage of said statistics to alter the plurality of clustering parameters,
re-running of clustering algorithm, and
continuing this process until some stopping condition is met;
classifying said clusters by identifying rules to assign the informational items to a specific one of said clusters without utilizing information from externally provided hierarchical representations;
summarizing each of said clusters by determining human readable descriptive titles to uniquely identify each of said clusters; and
storing said human readable descriptive titles, wherein said clustering, classifying and summarizing are all performed on a given set of data in no particular order.
2. A general purpose computer programmed to perform the method recited in claim 1 .
3. The computer-implemented method, as recited in claim 1 , wherein said feature selection, adaptive clustering, classifying and summarizing are all performed on a given set of data in no particular order.
4. A general purpose computer programmed to perform the method recited in claim 3 .
5. A computer-implemented mapping method for categorizing a plurality of informational items in an information retrieval system, the method comprising the steps of:
clustering by identifying one or more groupings of the informational items into one or more clusters, wherein said clustering includes an extension to BIRCH clustering algorithm, said extension comprising;
incorporating into subsequent classification and summarization steps one or more key points that define a bin of information items;
augmenting said key points with ranking scores to better identify the most important items within a pool of informational items for subsequent classification and summarization;
classifying said clusters by identifying rules to assign the informational items to a specific one of said clusters without utilizing information from externally provided hierarchical representations; and
summarizing each of said clusters by determining human readable descriptive titles to uniquely identify each of said clusters.