Service oriented architecture for a transformation function in a data integration platform

1. A method executed in a facility, the method comprising:

providing a data transformation module for a data integration function in a data integration platform, wherein the data integration function provides real time data integration of data between a plurality of data sources and wherein the data transformation module transforms data in the plurality of data sources from a first form to a second form;

providing an interface for the data transformation module;

identifying, using a processor, the data transformation module as a service in a registry of services;

responsive to identifying the data transformation module in the registry of services, using the interface to access the data transformation module as the service in a services oriented architecture;

providing a smart client for the service;

identifying the smart client as a service in the registry of services;

downloading the smart client to a client;

sending a request for the service on the interface in accordance with a set of parameters for the smart client; and

responsive to a request for the service on the interface, executing the data integration function in real time, wherein the data integration function supports a batch topology, a real time topology, and a combination of the batch topology and the real time topology;

pipelining one or more data integration transactions for delivery to the data integration function; and

inserting an end-of-wave marker between one or more pipelined data integration transactions to separate processing the pipelined data integration transactions into a set of distinct transactions.

2. The method of claim 1 , wherein the data integration function comprises a metadata management function and wherein the facility is a metadata facility.

3. The method of claim 2 , wherein the metadata management function is a member of a metadata management system, wherein the metadata management system allows for a modification of metadata in the plurality of data sources.

4. The method of claim 3 , wherein the metadata management function further comprises a metabroker, an atomic data repository, and a migration engine.

5. The method of claim 1 , wherein the data transformation module cleanses a plurality of data during a transformation of the plurality of data, and wherein cleansing the plurality of data comprises automatically correcting inaccuracies in the plurality of data.

6. The method of claim 1 , wherein the set of parameters cause the smart client to buffer the request for the service and send the request in a batch.

7. The method of claim 6 , wherein the set of parameters further cause the smart client to select a binding to access the service from a set of available bindings on which the service is available, wherein a selection of the binding determines a coupling type used in accessing the service.

8. The method of claim 1 , further comprising:

executing the data integration function in real time through a set of stages to form a data output; and

exposing the data output to a user in real time as a service using the graphical user interface.

9. The method of claim 1 , wherein the step of pipelining the one or more data integration transactions for the delivery to the data integration function comprises:

delivering the one or more data integration transactions to the data integration function while the data integration function is already running.

10. The method of claim 9 , wherein the data integration function is a first data integration function and further comprising:

delivering the one or more data integration transactions to a second data integration function, wherein the second data integration function is run when the first data integration function is at a maximum capacity.

11. The method of claim 10 , wherein the first data integration function providing real time data integration of the data between the plurality of data sources comprising:

responsive to identifying the end of wave marker between the one or more pipelined data integration transactions, flushing a data integration transaction in the one or more pipelined data integration transaction to the first data integration function.

12. The method of claim 11 , wherein the end-of-wave marker triggers immediate execution of the data integration function without waiting for a batch process.

13. The method of claim 1 , wherein the set of parameters cause the smart client to buffer the request for the service and send the request in a batch.

14. The method of claim 1 , wherein data in the request is cleansed by the data transformation module.

15. A system comprising:

a memory;

a data transformation module for a data integration function in a data integration platform, wherein the data integration function provides real time data integration of data between a plurality of data sources, wherein the data transformation module transforms data in the plurality of data sources from a first form to a second form, wherein the data transformation module pipelines one or more data integration transactions for delivery to the data integration function, wherein the data transformation module inserts an end-of-wave marker between one or more pipelined data integration transactions to separate processing the pipelined data integration transactions into a set of distinct transactions, wherein the data transformation module is in the memory, wherein the data integration function supports a batch topology, a real time topology, and a combination of the batch topology and the real time topology;

a registry of services, wherein the registry is in the memory;

an interface for the data transformation module, wherein the interface is in the memory;

a smart client for the service;

wherein the data transformation module is identified in the registry as a service;

wherein the data transformation module can be accessed as the service in a services oriented architecture in real time; and

wherein the smart client is downloaded to a client and sends a request to the service in accordance with a set of parameters for the smart client.

16. The system of claim 15 , wherein the data integration function comprises a metadata management function.

17. The system of claim 16 , wherein the metadata management function is a member of a metadata management system, wherein the metadata management system allows for a modification of metadata in the plurality of data sources.

18. The system of claim 17 , wherein the metadata management function further comprises a metabroker, an atomic data repository, and a migration engine.

19. The system of claim 15 , further comprising:

executing the data integration function in real time through a set of stages to form a data output; and

exposing the data output to a user in real time as a service using the graphical user interface.

20. The system of claim 15 , wherein the data integration function supports a batch topology, a real time topology, and a combination of the batch topology and the real time topology.

21. The system of claim 20 , further comprising:

pipelining one or more data integration transactions for delivery to the data integration function.

22. The system of claim 21 , further comprising:

inserting an end-of-wave marker between one or more pipelined data integration transactions to separate processing the pipelined data integration transactions into a set of distinct transactions.

23. The system of claim 22 , wherein the end-of-wave marker triggers immediate execution of the data integration function without waiting for a batch process.