Method and system for dynamically collecting data for checkpoint tuning and reduce recovery time
View Patent ↗A system and method for checkpoint tuning in a computer environment comprising a processor coupled to a statistical database and at least one database or table manager and data log files. The data log files store data relating to the operations of the database or table manager. The processor executes a module that captures data comprising the number of operations, the time taken by the checkpoint process, folded journal entries, long running transactions, and the ratio of WRITE:READ operations. The processor then stores the captured data and processes the collected data, generating statistics therefrom. The statistical data is stored in the statistical database. The system selects checkpoint intervals based on the data statistics; and implements the checkpoint intervals.
1. A method for checkpoint tuning in a computer environment comprising:
executing a process concurrently with a database manager, wherein said process;
monitors said database manager;
dynamically captures data comprising:
the number of operations,
the number of operations between at least two checkpoints,
the time taken by the checkpoint process,
folded journal entries,
unresolved long running transactions, the unresolved long running transactions are operations which span across multiple loggers and fail to terminate upon a logger change, and
the ratio of WRITE:READ operations; and
stores said collected data;
processing said collected data and generating statistics therefrom;
selecting checkpoint intervals based on said data statistics; and
implementing said checkpoint intervals.
2. The method of claim 1 wherein said number of operations is the total of all WRITE and READ operations for the persistence data.
3. The method of claim 1 wherein said time taken by the checkpoint process is the time that other jobs are locked out from writing to the LOG files and WRITE and READ operations are blocked.
4. The method of claim 1 wherein said folded journal entries are log records of persistent data occupying multiple physical records in a log file.
5. The method of claim 1 wherein said long running transactions are determined by capturing, at the time of said logger change, the transaction ID maintained by the underlying software;
scanning backwards through the logger chain; and
determining the point where each transaction originated.
6. A method for checkpoint tuning in a computer environment comprising:
executing a batch program to collect data from data LOG files associated with a database manager, said collected data comprising:
the total number of operations,
the number of operations between at least two checkpoints,
the time taken by the checkpoint process,
folded journal entries, the folded journal entries are a log record of persistent data occupying multiple physical records in the data LOG files,
long running transactions, the long running transactions are operations which span across multiple data LOG files and fails to end upon a LOG file change, and
the ratio of WRITE:READ operations;
storing said collected data;
processing said data and generating statistics therefrom;
selecting checkpoint intervals based on said data statistics; and
setting said checkpoint intervals.
7. The method of claim 6 wherein said number of data operations is the total of all WRITE and READ operations for the persistence data.
8. The method of claim 6 wherein said time taken by the checkpoint process is the time that other jobs are locked out from writing to the data LOG files and WRITE and READ operations are blocked.
9. The method of claim 6 wherein said long running transactions are determined by capturing, at the time of said LOG file change, the transaction ID maintained by the underlying software;
scanning backwards through the LOG file chain; and
determining the point where each transaction originated.
10. A system for checkpoint tuning in a computer environment comprising:
a statistical database;
at least one queue manager journal receiver, said queue manager journal receiver storing data regarding the activities of said queue manager; and
a processor coupled to said statistical database, said processor executes a module that captures data comprising:
the number of queue operations,
the time taken by the checkpoint process,
folded journal entries, the folded journal entries are a log record of persistent data occupying multiple physical records in data LOG files,
long running transactions, the long running transactions are operations which span across multiple data LOG files and fails to end upon a LOG file change, and
the ratio of PUT:GET operations;
processes said collected data and generating statistics therefrom;
stores said statistical data in said statistical database;
selects checkpoint intervals based on said data statistics; and
implements said checkpoint intervals.
11. The system of claim 10 wherein said module is executed concurrently with a queue manager;
monitors said queue manager; and
dynamically captures data directly from said queue manager.
12. The system of claim 10 wherein said module executes a batch program to collect data from said at least one journal receiver associated with a queue manager.
13. The system of claim 11 wherein said number of queue operations comprise the number of queue operations between at least two checkpoints.
14. The system of claims 12 wherein said number of queue operations comprise the number of queue operations between at least two checkpoints.