Granted Patent B2

US 9,256,472 · App. 13/545,243 · Granted Feb 9, 2016

Abstract

Claims (31)

1. A system for improved batch processing of data comprising:

a commodity hardware-based software framework;

one or more programming functions on the commodity hardware-based software framework that replace programming on one or more of a mainframe computer and a distributed platform, wherein the programming on the one or more of the mainframe computer and the distributed platform is part of an application running on the one or more of the mainframe computer and the distributed platform;

a distributed file system that stores data on at least one computer node; and

one or more software tools configured to move data from a data source on the one or more of the mainframe computer and the distributed platform to the distributed file system,

wherein the one or more software tools are configured to move data from the distributed file system to the data sources on the one or more of the mainframe computer and the distributed platform, wherein data that would have been processed by the programming on the one or more of the mainframe computer and the distributed platform is instead processed by the one or more programming functions executed on the commodity hardware-based software framework, and wherein the data processed by the one or more programming functions is sent back to the application running on the one or more of the mainframe computer and the distributed platform.

2. The system of claim 1 , wherein the commodity hardware-based software framework processes the data at a lower cost than the one or more of the mainframe computer and the distributed platform.

3. The system of claim 1 , wherein the one or more programming functions are created using one or more of a PIG computer language, a HIVE migration tool, and a MapReduce program.

4. The system of claim 1 , wherein the one or more programming functions are created without altering application logic.

5. The system of claim 1 , wherein the one or more programming functions are designed so that node failures are automatically handled by the framework.

6. The system of claim 1 , wherein the distributed file system is designed so that node failures are automatically handled by the framework.

7. The system of claim 1 , wherein the data sources comprise one or more of the following: a relational management database system (RDBMS), a virtual sequential access method (VSAM), an information management system (IMS), an adaptable data base system (ADABAS), a statistical analysis system (SAS), and other data files created by third party tools or packages.

8. The system of claim 1 , further comprising a storage cluster that is configured to provide archival and backup storage for application recovery.

9. The system of claim 1 , wherein the distributed platform is a legacy distributed platform.

10. A method for improved batch processing of data comprising:

migrating computer software for batch processing from a mainframe computer or a distributed platform to a commodity hardware-based software framework;

replacing one or more programming functions on the mainframe computer or the distributed platform with software that runs on the commodity hardware-based software framework, wherein the one or more programming functions on the mainframe computer or the distributed platform are part of an application running on the mainframe computer or the distributed platform, wherein data that would have been processed by the one or more programming functions on the mainframe computer or the distributed platform is instead processed by the software running on the commodity hardware-based software framework;

storing data on computer nodes in a distributed file system;

moving the data from data sources on the mainframe computer or the distributed platform to the distributed file system using one or more software tools;

moving the data from the distributed file system to the data sources on the mainframe computer or the distributed platform using the one or more software tools;

processing the data by the software running on the commodity hardware-based software framework instead of processing the data by the one or more programming functions on the mainframe or the distributed platform; and

sending back the data processed by the software running on the commodity hardware-based software framework to the application running on the mainframe computer or the distributed platform.

11. The method of claim 10 , wherein the commodity hardware-based software framework is configured to process the data at a lower cost than the mainframe computers or distributed platforms.

12. The method of claim 10 , wherein the one or more programming functions are replaced using one or more of a PIG computer language, a HIVE migration tool, and a MapReduce program.

13. The method of claim 10 , wherein the one or more programming functions are replaced without altering application logic.

14. The method of claim 10 , wherein the one or more programming functions are replaced so that node failures are automatically handled by the framework.

15. The method of claim 10 , wherein the distributed file system is designed so that node failures are automatically handled by the framework.

16. The method of claim 10 , wherein the data sources comprise one or more of the following: a relational management database system (RDBMS), a virtual sequential access method (VSAM), an information management system (IMS), an adaptable data base system (ADABAS), a statistical analysis system (SAS), and other data files created by third party tools or packages.

17. The method of claim 10 , further comprising moving the data from the distributed file system to a storage cluster that is configured to provide archival and backup storage.

18. The method of claim 17 , further comprising identifying batch processing components in legacy applications.

19. The method of claim 10 , further comprising identifying high MIPS legacy applications or applications with a larger batch processing window.

Assignments (18)