Method and system for transforming input data streams
View Patent ↗A system and method for processing an input data stream in a first data format of a plurality of first data formats to an output data stream in a second data format of a plurality of second data formats. A plurality of input connector modules receive respective input data streams and at least one input queue stores the received input data streams. A plurality of job threads is operatively connected to the at least one input queue, each job thread formatting a stored input data stream to produce an output data stream. At least one output queue stores the output data streams from the plurality of job threads. A plurality of output connector modules is operatively connected to the at least one output queue, the output connector modules supplying respective output data streams.
1. A system for transforming input data streams comprising:
a physical input connector;
a physical output connector;
a processing system coupled to the physical input connector and the physical output connector, the processing system including a computer program product to:
read an electronic input data stream received at the physical input connector, wherein input data in the input data stream is of a first format; and
in a same thread:
identify one or more events based on a detected pattern or a detected sequence in the input data stream;
create a message for each identified event, the message containing data from the event;
create output data of a second format from the message, the output data containing the data from the message, wherein the second format is a different format from the first format; and
provide an output data stream via the physical output connector, the output data stream comprising the output data.
2. The system of claim 1 , wherein the output data of the second format is created by transforming the data of the message based on one or more communication rules, each communication rule defining how the data is to be transformed.
3. The system of claim 2 , wherein at least one communication rule is selected based on a destination of the output data stream.
4. The system of claim 1 , wherein the thread services one or more corresponding input queues.
5. The system of claim 4 , wherein the input data stream is associated with one or more jobs on one or more of the corresponding input queues.
6. The system of claim 1 , wherein the message has one or more fields populated from the data of the event.
7. The system of claim 1 , wherein the one or more events are ordered for execution.
8. The system of claim 1 wherein the output data stream is a page layout, a record based text file, eXtensible Markup Language (XML), an email, a text message, or a meta record.
9. A computer program product comprising a non-transitory computer readable medium storing a set of computer readable instructions, the set of computer readable instructions comprising instructions executable to:
read an electronic input data stream of file data received over a network at a physical input, wherein input data in the input data stream is of a first format; and
in a same thread:
identify one or more events based on a detected pattern or a detected sequence in the input data stream;
create a message for each identified event, the message containing data from the event;
create output data of a second format from the message, the output data containing the data from the message, wherein the second format is a different format from the first format; and
provide an output data stream via a physical output connector, the output data stream comprising the output data.
10. The computer program product of claim 9 , wherein the output data of the second format is created by transforming the data of the message based on one or more communication rules, each communication rule defining how the data is to be transformed.
11. The computer program product of claim 10 , wherein at least one communication rule is selected based on a destination of the output data stream.
12. The computer program product of claim 9 , wherein the thread services one or more corresponding input queues.
13. The computer program product of claim 12 , wherein the input data stream is associated with one or more jobs on one or more of the corresponding input queues.
14. The computer program product of claim 9 , wherein the message has one or more fields populated from the data of the event.
15. The computer program product of claim 9 , wherein the one or more events are ordered for execution.
16. The computer program product of claim 9 , wherein the output data stream is a page layout, a record based text file, eXtensible Markup Language (XML), an email, a text message, or a meta record.
17. A method for processing a data stream in a network environment, comprising:
reading an electronic input data stream of file data received over a network at a physical input, wherein input data in the input data stream is of a first format; and
in a same thread:
identifying one or more events based on a detected pattern or a detected sequence in the input data stream;
creating a message for each identified event, the message containing data from the event;
creating output data of a second format from the message, the output data containing the data from the message, wherein the second format is a different format from the first format; and
providing an output data stream via a physical output connector, the output data stream comprising the output data.
18. The method of claim 17 , wherein the output data of the second format is created by transforming the data of the message based on one or more communication rules, each communication rule defining how the data is to be transformed.
19. The method of claim 18 , wherein at least one communication rule is selected based on a destination of the output data stream.
20. The method of claim 17 , wherein the thread services one or more corresponding input queues.
21. The method of claim 20 , wherein the input data stream is associated with one or more jobs on one or more of the corresponding input queues.