System for changing rules for data pipeline reading using trigger data from one or more data connection modules
Provided is an operating method of a system including a first data processing module, a second data processing module, and one or more data connection modules that transfer data between the first data processing module and the second data processing module. In an operational state where the first data processing module and the second data processing module can process data input thereto, the first data processing module inputs trigger data to any of the one or more data connection modules, and in response to the second data processing module receiving the trigger data via the one or more data connection modules, the second data processing module changes a reading rule for reading data from the one or more data connection modules. Also provided are a system and a computer program product.
1. A system including a first data processing module, a second data processing module, and one or more data connection modules that transfer data between the first data processing module and the second data processing module, the system comprising:
a processor; and
one or more computer readable mediums collectively including instructions that, when executed by the processor, cause the processor to:
in an operational state where the first data processing module and the second data processing module can process data input thereto, cause the first data processing module to input trigger data to the one or more data connection modules; and
in response to the second data processing module receiving the trigger data via the one or more data connection modules, cause the second data processing module to change a reading rule for reading data from the one or more data connection modules;
wherein before the second data processing module receives the trigger data via the one or more data connection modules, the second data processing module is informed in advance concerning the reading rule to be used by the second data processing module in response to the trigger data.
2. The system according to claim 1 , wherein after the first data processing module has input the trigger data to the one or more data connection modules, the first data processing module changes a writing rule for writing data to the one or more data connection modules.
3. The system according to claim 1 , wherein:
when the first data processing module is in the operational state, a new data connection module that transfers data between the first data processing module and the second data processing module is added to the one or more existing data connection modules;
the first data processing module inputs the trigger data to the one or more existing data connection modules, and in response to the trigger data having been input, inputs data to the new data connection module; and
the second data processing module reads the data according to the reading rule including the new data connection module as a data reading target, on a condition that the second data processing module has read the trigger data from all of the one or more existing data connection modules.
4. The system according to claim 3 , wherein whether or not to add the new data connection module is determined based on an amount of data accumulated in the one or more existing data connection modules.
5. The system according to claim 3 , wherein whether or not to add the new data connection module is determined based on the data input to the first data processing module.
6. The system according to claim 5 , wherein when the new data connection module is created based on the data input to the first data processing module, data capacity of the new data connection module is controlled based on data input to a driver of the system.
7. The system according to claim 5 , wherein when the new data connection module is created based on the data input to the first data processing module, a type of the new data connection module is controlled based on the data input to the first data processing module.
8. The system according to claim 1 , wherein the first data processing module inputs the trigger data including information concerning the reading rule to the one or more data connection modules.
9. The system according to claim 1 , wherein an early-stage data processing module that processes data is provided in the system before the first data processing module; and
wherein the trigger data is input to the one or more data connection modules from the first data processing module, without passing through the early-stage data processing module.
10. A computer program product comprising a computer readable storage medium having program instructions embodied therewith, the program instructions executable by a computer to cause the computer to:
in an operational state where a first data processing module and a second data processing module can process data input thereto, cause the first data processing module to input trigger data to one or more data connection modules that transfer data between the first data processing module and the second data processing module; and
in response to the second data processing module receiving the trigger data via the one or more data connection modules, cause the second data processing module to change a reading rule for reading data from the one or more data connection modules;
wherein before the second data processing module receives the trigger data via the one or more data connection modules, the second data processing module is informed in advance concerning the reading rule to be used by the second data processing module in response to the trigger data.