Parallel data stream processing method, parallel data stream processing system, and storage medium

1. A non-transitory computer-readable storage medium having stored therein a data stream processing program that causes a computer to execute a process comprising:

extracting a relationship between properties included in events in a data stream from the data stream itself, definitions of queries for processing the data stream, or both of the data stream and the definitions of the queries;

specifying a common partitioning key common to a subset of the queries based on partitioning keys in the definitions of the queries, the relationship between the properties, and a probability of the relationship;

grouping the subset of the queries into a partitioning group based on the common partitioning key; and

assigning the subset of the queries belonging to the partitioning group to each of parallel nodes provided for parallel distributed processing,

wherein, when functions in the definitions of the queries are used as the partition keys, the specifying the common partitioning key includes

when a first function defined in a first query of the queries has a first period length and a second function defined in a second query of the queries following the first query has a second period length, obtaining a third function having a third period length that is a least common multiple of the first period length and the second period length;

calculating a ratio of a number of events caused to be transferred between the parallel nodes due to misalignment between the first period length and the second period length to a number of all events in the data stream input to the parallel nodes based on conditions that the first query and the second query are grouped into the same partitioning group and assigned to each of the parallel nodes using the third function as the common partitioning key; and

when the calculated ratio is less than or equal to a first threshold, specifying the third function as the common partitioning key.

2. The non-transitory computer-readable storage medium as claimed in claim 1 , wherein

the extracting the relationship between the properties includes

storing multiple input events input to one of the queries, and

identifying two properties from the properties in the stored input events, the two properties having one of a 1-to-N relationship and a 1-to-1 relationship (N is an integer) at a probability greater than a second threshold; and

the common partitioning key is specified based on the identified two properties and the probability of the relationship between the two properties.

3. The non-transitory computer-readable storage medium as claimed in claim 1 , wherein

the extracting the relationship between the properties includes

when a partitioning key is defined as a function in one of the queries, identifying the function and properties related to the function; and

the common partitioning key is specified based on the function and the properties related to the function.

4. A method executed by a computer, the method comprising:

extracting a relationship between properties included in events in a data stream from the data stream itself, definitions of queries for processing the data stream, or both of the data stream and the definitions of the queries;

specifying a common partitioning key common to a subset of the queries based on partitioning keys in the definitions of the queries, the relationship between the properties, and a probability of the relationship;

grouping the subset of the queries into a partitioning group based on the common partitioning key; and

assigning the subset of the queries belonging to the partitioning group to each of parallel nodes provided for parallel distributed processing,

wherein, when functions in the definitions of the queries are used as the partition keys, the specifying the common partitioning key includes

when a first function defined in a first query of the queries has a first period length and a second function defined in a second query of the queries following the first query has a second period length, obtaining a third function having a third period length that is a least common multiple of the first period length and the second period length;

calculating a ratio of a number of events caused to be transferred between the parallel nodes due to misalignment between the first period length and the second period length to a number of all events in the data stream input to the parallel nodes based on conditions that the first query and the second query are grouped into the same partitioning group and assigned to each of the parallel nodes using the third function as the common partitioning key; and

when the calculated ratio is less than or equal to a first threshold, specifying the third function as the common partitioning key.

5. The method as claimed in claim 4 , wherein

the extracting the relationship between the properties includes

storing multiple input events input to one of the queries, and

identifying two properties from the properties in the stored input events, the two properties having one of a 1-to-N relationship and a 1-to-1 relationship (N is an integer) at a probability greater than a second threshold; and

the common partitioning key is specified based on the identified two properties and the probability of the relationship between the two properties.

6. The method as claimed in claim 4 , wherein

the extracting the relationship between the properties includes

when a partitioning key is defined as a function in one of the queries, identifying the function and properties related to the function; and

the common partitioning key is specified based on the function and the properties related to the function.

7. A system, comprising:

a processor that executes a process including

extracting a relationship between properties included in events in a data stream from the data stream itself, definitions of queries for processing the data stream, or both of the data stream and the definitions of the queries;

specifying a common partitioning key common to a subset of the queries based on partitioning keys in the definitions of the queries, the relationship between the properties, and a probability of the relationship;

grouping the subset of the queries into a partitioning group based on the common partitioning key; and

assigning the subset of the queries belonging to the partitioning group to each of parallel nodes provided for parallel distributed processing,

wherein, when functions in the definitions of the queries are used as the partition keys, the specifying the common partitioning key includes

when a first function defined in a first query of the queries has a first period length and a second function defined in a second query of the queries following the first query has a second period length, obtaining a third function having a third period length that is a least common multiple of the first period length and the second period length;

calculating a ratio of a number of events caused to be transferred between the parallel nodes due to misalignment between the first period length and the second period length to a number of all events in the data stream input to the parallel nodes based on conditions that the first query and the second query are grouped into the same partitioning group and assigned to each of the parallel nodes using the third function as the common partitioning key; and

when the calculated ratio is less than or equal to a first threshold, specifying the third function as the common partitioning key.

8. The system as claimed in claim 7 , wherein

the extracting the relationship between the properties includes

storing multiple input events input to one of the queries, and

identifying two properties from the properties in the stored input events, the two properties having one of a 1-to-N relationship and a 1-to-1 relationship (N is an integer) at a probability greater than a second threshold; and

the common partitioning key is specified based on the identified two properties and the probability of the relationship between the two properties.

9. The system as claimed in claim 7 , wherein

the extracting the relationship between the properties includes

when a partitioning key is defined as a function in one of the queries, identifying the function and properties related to the function; and

the common partitioning key is specified based on the function and the properties related to the function.