Fuzzy data operations
A method for clustering data elements stored in a data storage system includes reading data elements from the data storage system. Clusters of data elements are formed with each data element being a member of at least one cluster. At least one data element is associated with two or more clusters. Membership of the data element belonging to respective ones of the two or more clusters is represented by a measure of ambiguity. Information is stored in the data storage system to represent the formed clusters.
1. A method for joining data elements from two or more datasets stored in at least one data storage system, the method performed by one or more processors and including:
determining matches between one or more variants of objects in first data elements from a first dataset and objects in second data elements from a second dataset, with a variant being in accordance with a variant relation between one or more of the objects in the first data elements from the first dataset and one or more of the objects in the second data elements from the second dataset;
evaluating respective second data elements having respective objects determined as matches;
joining at least one of the first data elements from the first dataset with at least one of the second data elements from the second dataset to produce a third data element, with the joining based on the evaluation of the respective second data elements; and
outputting a third dataset including at least a portion of the first data set combined with at least a portion of the second dataset, with the third dataset including one or more third data elements produced by joining.
2. The method of claim 1 , wherein the variant relation between a first object and a second object corresponds to a value of a function representing a distance between the first object and the second object being below a predetermined threshold.
3. The method of claim 2 , wherein the variant relation is not an equivalence relation.
4. The method of claim 2 , further including determining a match between an object in one of the first data elements from the first dataset and an object in one of the second data elements in the second dataset by determining that the variant relation holds between the object in the one of the first data elements and the object in the one of the second data elements.
5. The method of claim 2 , further including determining a match between an object in one of the first data elements from the first dataset and an object in one of the second data elements in the second dataset by determining: that the variant relation holds between the object in the one of the first data elements and an object in a third data element in the first dataset, and that the variant relation holds between the object in the third data element and the object in the one of the second data elements.
6. The method of claim 1 , wherein evaluating respective second data elements having respective objects determined as matches includes comparison of objects in the respective second data elements other than the respective objects determined as matches.
7. A system for joining data elements from two or more datasets stored in at least one data storage system, the system including:
means for determining matches between one or more variants of objects in first data elements from a first dataset and objects in second data elements from a second dataset, with a variant being in accordance with a variant relation between one or more of the objects in the first data elements from the first dataset and one or more of the objects in the second data elements from the second dataset;
means for evaluating respective second data elements having respective objects determined as matches;
means for joining at least one of the first data elements from the first dataset with at least one of the second data elements from the second dataset to produce a third data element, with the joining based on the evaluation of the respective second data elements; and
means for outputting a third dataset including at least a portion of the first data set combined with at least a portion of the second dataset, with the third dataset including one or more third data elements produced by joining.
8. A computer-readable hardware storage device storing a computer program for joining data elements from two or more datasets stored in at least one data storage system, the computer program including instructions for causing a computer to:
determine matches between one or more variants of objects in first data elements from a first dataset and objects in second data elements from a second dataset, with a variant being in accordance with a variant relation between one or more of the objects in the first data elements from the first dataset and one or more of the objects in the second data elements from the second dataset;
evaluate respective second data elements having respective objects determined as matches;
join at least one of the first data elements from the first dataset with at least one of the second data elements from the second dataset to produce a third data element, with the joining based on the evaluation of the respective second data elements; and
output a third dataset including at least a portion of the first data set combined with at least a portion of the second dataset, with the third dataset including one or more third data elements produced by joining.
9. The computer-readable hardware storage device of claim 8 , wherein the variant relation between a first object and a second object corresponds to a value of a function representing a distance between the first object and the second object being below a predetermined threshold.
10. The computer-readable hardware storage device of claim 9 , wherein the variant relation is not an equivalence relation.
11. The computer-readable hardware storage device of claim 9 , further including instructions for causing the computer to determine a match between an object in one of the first data elements from the first dataset and an object in one of the second data elements in the second dataset by determining that the variant relation holds between the object in the one of the first data elements and the object in the one of the second data elements.
12. The computer-readable hardware storage device of claim 9 , further including instructions for causing the computer to determine a match between an object in one of the first data elements from the first dataset and an object in one of the second data elements in the second dataset by determining that the variant relation holds between the object in the one of the first data elements and an object in a third data element in the first dataset, and that the variant relation holds between the object in the third data element and the object in the one of the second data elements.
13. The computer-readable hardware storage device of claim 8 , wherein the instructions for causing the computer to evaluate respective second data elements having respective objects determined as matches includes comparison of objects in the respective second data elements other than the respective objects determined as matches.
14. A system including:
a computer; and
computer-readable hardware storage device storing a computer program for joining data elements from two or more datasets stored in at least one data storage system, the computer program including instructions for causing the computer to:
determine matches between one or more variants of objects in first data elements from a first dataset and objects in second data elements from a second dataset, with a variant being in accordance with a variant relation between one or more of the objects in the first data elements from the first dataset and one or more of the objects in the second data elements from the second dataset;
evaluate respective second data elements having respective objects determined as matches;
join at least one of the first data elements from the first dataset with at least one of the second data elements from the second dataset to produce a third data element, with the joining based on the evaluation of the respective second data elements; and
output a third dataset including at least a portion of the first data set combined with at least a portion of the second dataset, with the third dataset including one or more third data elements produced by joining.
15. The system of claim 14 , wherein the variant relation between a first object and a second object corresponds to a value of a function representing a distance between the first object and the second object being below a predetermined threshold.
16. The system of claim 15 , wherein the variant relation is not an equivalence relation.
17. The system of claim 15 , further including instructions for causing the computer to determine a match between an object in one of the first data elements from the first dataset and an object in one of the second data elements in the second dataset by determining that the variant relation holds between the object in the one of the first data elements and the object in the one of the second data elements.
18. The system of claim 15 , further including instructions for causing the computer to determine a match between an object in one of the first data elements from the first dataset and an object in one of the second data elements in the second dataset by determining that the variant relation holds between the object in the one of the first data elements and an object in a third data element in the first dataset, and that the variant relation holds between the object in the third data element and the object in the one of the second data elements.
19. The system of claim 14 , wherein the instructions for causing the computer to evaluate respective second data elements having respective objects determined as matches includes comparison of objects in the respective second data elements other than the respective objects determined as matches.