Apparatus and method for categorical filtering of data

1. A computer readable storage medium, comprising executable instructions to:

retrieve a dataset from a data source wherein the dataset includes a first set of categories;

build a data structure that represents the dataset, wherein the data structure stores a plurality of attribute combinations and a plurality of counts wherein a representative count in the plurality of counts indicates how many times a corresponding attribute occurs in the dataset, wherein each attribute combination of the plurality of attribute combinations is selected from a record of the dataset and a partial record of the dataset;

calculate a first plurality of merit values for the first set of categories;

order the first set of categories based on a criterion;

return the first set of categories;

accept a filter;

copy a set of applicable branches from the data structure wherein an applicable branch of the set of applicable branches complies with the filter;

build a new data structure using the set of applicable branches, wherein the new data structure comprises a second set of categories;

calculate a second plurality of merit values from the second set of categories;

retrieve a filtered set of data from the data source; and

return the filtered set of data and the second set of categories.

2. The computer readable storage medium of claim 1 wherein a category comprises a set of attributes.

3. The computer readable storage medium of claim 1 wherein the data structure is parsed to derive metadata about the first set of categories.

4. The computer readable storage medium of claim 1 wherein the criterion is selected from descending merit, ascending merit, descending merit combined with another value derived from the dataset or associated metadata, and ascending merit combined with another value derived from the dataset or associated metadata.

5. The computer readable storage medium of claim 1 wherein:

each category in the first set of categories comprises a plurality of attributes;

the plurality of attributes in each category is returned in an order selected from descending frequency, ascending frequency, numerical, alphabetical, unordered and a user specified order.

6. The computer readable storage medium of claim 1 wherein the filter is specified by a user or a default setting.

7. The computer readable storage medium of claim 1 wherein the second set of categories does not include a category associated with the filter.

8. The computer readable storage medium of claim 1 wherein the dataset is contained in a single table.

9. The computer readable storage medium of claim 8 wherein the single table is an aggregation of data sources specified by a user.

10. A computer readable storage medium, comprising executable instructions to:

retrieve a dataset from a data source;

reorder the dataset by successively grouping on each category in a first set of categories;

build an enumeration tree;

calculate a plurality of merit values for the first set of categories;

determine a second set of categories wherein the merit values meet a criterion; and

return the second set of categories, wherein:

the dataset comprises a plurality of records, each record comprising a plurality of attributes;

the enumeration tree comprises a plurality of nodes comprising an attribute and a count;

the nodes of the enumeration tree are arranged in branches; and

the executable instruction to build an enumeration tree further comprising executable instructions to:

add a first branch to the enumeration tree by mapping each attribute from a first record to each node in the first branch;

store the attributes added to the enumeration tree in a previous branch, wherein the attributes of the previous branch correspond to attributes from a previous record;

add a further branch to the enumeration tree with executable instructions to:

compare the attributes of the previous record and the attributes of a current record;

increment the count for each node corresponding to an attribute of the common leading attributes;

map each remaining attribute in the current record to a new node in a sub-branch, wherein the sub-branch stems from a branch node, wherein the branch node corresponds to the last attribute in the common leading attributes.

11. The computer readable storage medium of claim 10 wherein a category comprises a set of attributes.

12. The computer readable storage medium of claim 10 wherein an enumeration tree node stores a set of variables selected from at least one of an attribute ID, a parent node ID and a nodal attribute count.

13. The computer readable storage medium of claim 12 wherein the attribute ID refers to a temporary table storing an ID-attribute pair for each attribute in each category in the set of categories.

14. The computer readable storage medium of claim 10 wherein the enumeration tree stores a plurality of attribute combinations and a plurality of counts wherein a representative count in the plurality of counts indicates how many times a corresponding attribute occurs in the dataset.

15. The computer readable storage medium of claim 14 , wherein each attribute combination of the plurality of attribute combinations is selected from a record of the dataset and a partial record of the dataset.

16. The computer readable storage medium of claim 10 further comprising executable instructions to:

accept a filter;

copy a set of applicable branches from the enumeration tree;

build a new enumeration tree using the set of applicable branches;

calculate a new plurality of merit values;

determine a new set of categories wherein the merit value meets a new criterion;

retrieve a filtered dataset from the data source; and

return the filtered dataset and the new set of categories.