Generating a Venn diagram using a columnar database management system
Venn diagrams are computed for a given plurality of input sets. The process of computing the Venn diagrams is executed on columnar database systems for efficient execution. The computation of various subsets of the Venn diagrams is performed by determining subsets of various combinations of the input sets and computing set differences of the intersection sets. The process orders the execution of various steps of computing the subsets for the Venn diagram in an order that reduces the number of times an input set is loaded. Information describing various subsets of a Venn diagram is used to render the Venn diagram for display, for example, on a client device.
1 . A method, comprising:
encapsulating, by a client device, a data mining request in a file such that the file contains the data mining request, wherein the data mining request is for generating a Venn diagram, wherein the data mining request indicates data sets;
sending, by the client device, the file containing the data mining request over a network to a columnar database management system having a processor and a memory, the columnar database management system utilizing a columnar database structure to:
generate a truth table based on data stored in the columnar database structure, wherein the truth table contains combinations of the data sets, each entry in the truth table corresponding to one of the combinations, the truth table having a column for totaling a number of trues in a respective combination of the data sets, each true of the number of trues representing presence of a data set in the respective combination;
rank the combinations of the data sets in the truth table based on the number of trues in each respective combination of the data sets;
determine, for each respective combination of the ranked combinations in an order specified by a descending number of trues in the ranked combinations, an intersection set common to the data sets of the respective combination and a cardinality of the intersection set;
store the cardinality in the truth table as a partial result for the respective combination;
determine, based on the partial result for the respective combination, a respective final result for the respective combination; and
based on the respective final results thus determined, generate the Venn diagram requested by the client device;
receiving, by the client device, the Venn diagram from the columnar database management system; and
presenting, by the client device, the Venn diagram on a display.
2 . The method according to claim 1 , wherein the file has a message format, a text format, or an extensible markup language format.
3 . The method according to claim 1 , wherein the sending comprises making an application programming interface call by the client device to the columnar database management system.
4 . The method according to claim 1 , wherein the sending comprises sending, from the client device to the columnar database management system, the file through an interactive console on the client device.
5 . The method according to claim 1 , wherein each data set of the data sets comprises a set of elements and where the cardinality refers to a number of elements in the intersection set.
6 . The method according to claim 1 , wherein the columnar database management system loads a first combination of the data sets into the memory, sorts the truth table based on the number of trues, and determines, based at least on the truth table thus sorted, an intersection set that, once the intersection set is loaded into the memory, the first combination of the data sets and the intersection set form a second combination of the data sets that is a superset of the first combination of the data sets.
7 . The method according to claim 1 , wherein each entry of the truth table is associated with a binary value and wherein positions of bits in the binary value correspond to the data sets.
8 . A system, comprising:
a processor;
a non-transitory computer-readable medium; and
instructions stored on the non-transitory computer-readable medium and translatable by the processor for:
encapsulating a data mining request in a file such that the file contains the data mining request, wherein the data mining request is for generating a Venn diagram, wherein the data mining request indicates data sets;
sending the file containing the data mining request over a network to a columnar database management system having a processor and a memory, the columnar database management system utilizing a columnar database structure to:
generate a truth table based on data stored in the columnar database structure, wherein the truth table contains combinations of the data sets, each entry in the truth table corresponding to one of the combinations, the truth table having a column for totaling a number of trues in a respective combination of the data sets, each true of the number of trues representing presence of a data set in the respective combination;
rank the combinations of the data sets in the truth table based on the number of trues in each respective combination of the data sets;
determine, for each respective combination of the ranked combinations in an order specified by a descending number of trues in the ranked combinations, an intersection set common to the data sets of the respective combination and a cardinality of the intersection set;
store the cardinality in the truth table as a partial result for the respective combination;
determine, based on the partial result for the respective combination, a respective final result for the respective combination; and
based on the respective final results thus determined, generate the Venn diagram requested by the client device;
receiving the Venn diagram from the columnar database management system; and
presenting the Venn diagram on a display.
9 . The system of claim 8 , wherein the file has a message format, a text format, or an extensible markup language format.
10 . The system of claim 8 , wherein the sending comprises making an application programming interface call by the client device to the columnar database management system.
11 . The system of claim 8 , wherein the sending comprises sending, from the client device to the columnar database management system, the file through an interactive console on the client device.
12 . The system of claim 8 , wherein each data set of the data sets comprises a set of elements and where the cardinality refers to a number of elements in the intersection set.
13 . The system of claim 8 , wherein the columnar database management system loads a first combination of the data sets into the memory, sorts the truth table based on the number of trues, and determines, based at least on the truth table thus sorted, an intersection set that, once the intersection set is loaded into the memory, the first combination of the data sets and the intersection set form a second combination of the data sets that is a superset of the first combination of the data sets.
14 . The system of claim 8 , wherein each entry of the truth table is associated with a binary value and wherein positions of bits in the binary value correspond to the data sets.
15 . A computer program product comprising a non-transitory computer-readable medium storing instructions translatable by a processor for:
encapsulating a data mining request in a file such that the file contains the data mining request is for generating a Venn diagram, wherein the data mining request indicates data sets;
sending the file containing the data mining request over a network to a columnar database management system having a processor and a memory, wherein the columnar database management system utilizing a columnar database structure to:
generate a truth table based on data stored in the columnar database structure, wherein the truth table contains combinations of the data sets, each entry in the truth table corresponding to one of the combinations, the truth table having a column for totaling a number of trues in a respective combination of the data sets, each true of the number of trues representing presence of a data set in the respective combination;
rank the combinations of the data sets in the truth table based on the number of trues in each respective combination of the data sets;
determine, for each respective combination of the ranked combinations in an order specified by a descending number of trues in the ranked combinations, an intersection set common to the data sets of the respective combination and a cardinality of the intersection set;
store the cardinality in the truth table as a partial result for the respective combination;
determine, based on the partial result for the respective combination, a respective final result for the respective combination; and
based on the respective final results thus determined, generate the Venn diagram requested by the client device;
receiving the Venn diagram from the columnar database management system; and
presenting the Venn diagram on a display.
16 . The computer program product of claim 15 , wherein the file has a message format, a text format, or an extensible markup language format.
17 . The computer program product of claim 15 , wherein the sending comprises making an application programming interface call by the client device to the columnar database management system.
18 . The computer program product of claim 15 , wherein the sending comprises sending, from the client device to the columnar database management system, the file through an interactive console on the client device.
19 . The computer program product of claim 15 , wherein each data set of the data sets comprises a set of elements and where the cardinality refers to a number of elements in the intersection set.
20 . The computer program product of claim 15 , wherein the columnar database management system loads a first combination of the data sets into the memory, sorts the truth table based on the number of trues, and determines, based at least on the truth table thus sorted, an intersection set that, once the intersection set is loaded into the memory, the first combination of the data sets and the intersection set form a second combination of the data sets that is a superset of the first combination of the data sets.