Combination of datasets with use of a dataset filter

1. A method to join a first source dataset with a second source dataset, the method comprising:

receiving, by a processor, a request to join the first source dataset with the second source dataset based on a condition, wherein the condition is associated with an attribute of the first source dataset and the second source dataset;

determining, by the processor, a first range of a first set of attribute values, wherein the first set of attribute values corresponds to the first source dataset, and wherein the first set of attribute values is associated with the attribute;

determining, by the processor, a second range of a second set of attribute values, wherein the second set of attribute values corresponds to the second source dataset, and wherein the second set of attribute values is associated with the attribute;

comparing, by the processor, the first range with the second range to produce a comparison result;

determining, by the processor, a presence or absence of an overlap range between the first range and the second range based on the comparison result;

producing, by the processor, a filtered dataset based on the determination of the presence or absence of the overlap range and based on another comparison, wherein the another comparison, associated with the condition, comprises comparing each attribute value of the first set of attribute values with each attribute value of the second set of attribute values, and

wherein the filtered dataset includes:

at least a portion of the first source dataset based on a determination of the presence of the overlap range and when the condition is satisfied;

at least another portion of the first source dataset based on a determination of the absence of the overlap range and when the condition is satisfied; and

zero data based on the determination of the absence of the overlap range and when the condition is not satisfied;

transforming, by a map unit, the produced filtered dataset into a first mapped dataset based on the condition, wherein transforming using the produced filtered dataset facilitates reduction of an amount of input to the map unit and an amount of output from the map unit, thereby enabling reduction of overhead and processing power;

transforming, by the map unit, the second source dataset into a second mapped dataset based on the condition;

transforming, by a reduce unit, the first mapped dataset into a first reduced dataset based on the condition;

transforming, by the reduce unit, the second mapped dataset into a second reduced dataset based on the condition; and

combining, by a combiner unit, the first reduced dataset with the second reduced dataset to facilitate joining of the first source dataset with the second source dataset based on the condition that is associated with the attribute of the first source dataset and the second source dataset, in a social media platform.

2. The method of claim 1 , wherein the filtered dataset is a first filtered dataset, the method further comprising:

producing, by the processor, a second filtered dataset in response to the determination of the presence of the overlap range, wherein the second filtered dataset includes at least a portion of the second source dataset; and

transforming, by the map unit, the second filtered dataset into the second mapped dataset based on the condition.

3. The method of claim 1 , wherein the filtered dataset is a first filtered dataset, the method further comprising:

producing, by the processor, a second filtered dataset in response to the determination of the absence of the overlap range, wherein the second filtered dataset includes at least a portion of the second source dataset; and

transforming, by the map unit, the second filtered dataset into the second mapped dataset based on the condition.

4. The method of claim 1 , wherein the condition includes an indication of an inequality relation.

5. The method of claim 1 , wherein determining the first range of the first set of attribute values and determining the second range of the second set of attribute values comprises:

determining, by the processor, a first minimum value of the first set of attribute values of the first range;

determining, by the processor, a first maximum value of the first set of attribute values of the first range;

determining, by the processor, a second minimum value of the second set of attribute values of the second range; and

determining, by the processor, a second maximum value of the second set of attribute values of the second range.

6. A system effective to join a first source dataset with a second source dataset, the system comprising:

a memory configured to store the first source dataset and the second source dataset, wherein the first source dataset corresponds to a first set of attribute values of an attribute, and wherein the second source dataset corresponds to a second set of attribute values of the attribute;

a filter unit configured to be in communication with the memory;

a map unit configured to be in communication with the memory and the filter unit;

a reduce unit configured to be in communication with the memory and the map unit; and

a combiner unit configured to be in communication with the reduce unit;

the filter unit being configured to:

receive a request to join the first source dataset with the second source dataset based on a condition, wherein the condition is associated with the attribute;

determine a first range of the first set of attribute values;

determine a second range of the second set of attribute values;

compare the first range with the second range to produce a comparison result;

determine a presence or absence of an overlap range between the first range and the second range based on the comparison result;

produce a first filtered dataset based on the determination of the presence or absence of the overlap range,

wherein the first filtered dataset includes:

at least a portion of the first source dataset based on a determination of the presence of the overlap range and when the condition is satisfied;

at least another portion of the first source dataset based on a determination of the absence of the overlap range and when the condition is satisfied; and

zero data based on the determination of the absence of the overlap range and when the condition is not satisfied; and

send the first filtered dataset to the map unit;

the map unit being configured to:

transform the produced first filtered dataset into a first mapped dataset based on the condition, wherein the transformation using the produced first filtered dataset facilitates reduction of an amount of input to the map unit and an amount of output from the map unit, thereby enabling reduction of overhead and processing power;

transform the second source dataset into a second mapped dataset based on the condition; and

send the first and second mapped datasets to the reduce unit;

the reduce unit being configured to:

transform the first mapped dataset into a first reduced dataset;

transform the second mapped dataset into a second reduced dataset; and

send the first and the second reduced datasets to the combiner unit; and

the combiner unit being configured to combine the first reduced dataset with the second reduced dataset, to facilitate joining of the first source dataset with the second source dataset, based on the condition that is associated with the attribute, in a social media platform.

7. The system of claim 6 , wherein the filter unit is further configured to

produce a second filtered dataset in response to the determination of the presence of the overlap range, wherein the produced second filtered dataset includes at least a portion of the second source dataset; and

wherein the map unit is further configured to transform the produced second filtered dataset into the second mapped dataset based on the condition.

8. The system of claim 6 , wherein the filter unit is further configured to

produce a second filtered dataset in response to the determination of the absence of the overlap range, wherein the produced second filtered dataset includes at least a portion of the second source dataset; and

wherein the map unit is further configured to transform the produced second filtered dataset into the second mapped dataset based on the condition.

9. The system of claim 6 , wherein the condition includes an indication of an inequality relation.

10. The system of claim 6 , wherein the determination of the first range of the first set of attribute values and the determination of the second range of the second set of attribute values comprises:

determine a first minimum value of the first set of attribute values of the first range;

determine a first maximum value of the first set of attribute values of the first range;

determine a second minimum value of the second set of attribute values of the second range; and

determine a second maximum value of the second set of attribute values of the second range.

11. A system effective to process a request to join a first source dataset with a second source dataset in a social media platform, the system comprising:

a memory configured to store a filter instruction;

a processor configured to be in communication with the memory; and

a node configured to be in communication with the processor, the processor being configured to:

receive the request to join the first source dataset with the second source dataset based on a condition, wherein the condition is associated with an attribute of the first source dataset and the second source dataset, wherein the first source dataset corresponds to a first set of attribute values of the attribute, and wherein the second source dataset corresponds to a second set of attribute values of the attribute;

execute the filter instruction to generate a first command to determine a first range of the first set of attribute values;

execute the filter instruction to generate a second command to determine a second range of the second set of attribute values;

execute the filter instruction to generate a third command to compare the first range with the second range to produce a comparison result;

execute the filter instruction to generate a fourth command to produce a first filtered dataset based on the comparison result, wherein the first filtered dataset includes at least a portion of the first dataset, wherein production of the first filtered dataset is based on a first comparison, and wherein the first comparison, associated with the condition, comprises comparison of each attribute value of the first set of attribute values with each attribute value of the second set of attribute values;

execute the filter instruction to generate a fifth command to produce a second filtered dataset based on the comparison result, wherein the second filtered dataset includes at least a portion of the second dataset, and wherein production of the second filtered dataset is based on a second comparison, and wherein the second comparison, associated with the condition, comprises comparison of each attribute value of the first set of attribute values with each attribute value of the second set of attribute values; and

send the first, second, third, fourth, and fifth commands to the node, wherein an execution of the first, second, third, fourth, and fifth commands facilitate reduction of overhead and processing power of the node.

12. The system of claim 11 , wherein the node comprises:

a filter unit;

a map unit configured to be in communication with the filter unit;

a reduce unit configured to be in communication with the map unit; and

a combiner unit configured to be in communication with the reduce unit;

the filter unit of the node being configured to:

execute the first command to determine the first range of the first set of attribute values;

execute the second command to determine the second range of the second set of attribute values;

execute the third command to compare the first range with the second range to produce the comparison result;

execute the fourth command to produce the first filtered dataset based on the comparison result;

execute the fifth command to produce the second filtered dataset based on the comparison result; and

send the first and second filtered datasets to the map unit of the node;

the map unit of the node being configured to:

transform the first filtered dataset into a first mapped dataset based on the condition;

transform the second filtered dataset into a second mapped dataset based on the condition; and

send the first and second mapped datasets to the reduce unit of the node;

the reduce unit of the node being configured to:

transform the first mapped dataset into a first reduced dataset;

transform the second mapped dataset into a second reduced dataset; and

send the first and second reduced datasets to the combiner unit; and

the combiner unit being configured to combine the first reduced dataset with the second reduced dataset to join the first source dataset with the second source dataset.

13. The system of claim 12 , wherein the node is further configured to send the joined first source dataset and second source dataset to the processor.

14. The system of claim 11 , wherein the condition includes an indication of an inequality relation.

15. The system of claim 11 , wherein the determination of the first range of the first set of attribute values and the determination of the second range of the second set of attribute values comprises:

determine a first minimum value of the first set of attribute values of the first range;

determine a first maximum value of the first set of attribute values of the first range;

determine a second minimum value of the second set of attribute values of the second range; and

determine a second maximum value of the second set of attribute values of the second range.

16. The system of claim 11 , wherein the node comprises a node memory, and wherein the processor is further configured to:

divide a raw dataset into the first source dataset and the second source dataset;

store the first source dataset and the second source dataset in the node memory of the node;

in response to the receipt of the request, identify the node based on the first source dataset and the second source dataset being stored in the node memory of the node; and

send the first, second, third, fourth, and fifth commands to the node in response to the identification of the node.

17. The method of claim 1 , further comprising:

determining, by the map unit, a particular number of partitions of a mapped dataset, which is associated with the first mapped dataset and the second mapped dataset, based on partition instructions received from the processor, wherein the partition instructions include instructions to minimize processing time and resource usage, and wherein the particular number of the partitions of the mapped dataset is based on the processing time and the resource usage.

18. The system of claim 6 , wherein the filter unit is configured to produce the first filtered dataset based on a number of occurrences of each attribute value of the first set of attribute values and the second set of attribute values.