Granted Patent B2

US 12,430,364 · App. 17/809,572 · Granted Sep 30, 2025

Inventors: Xue Bo (ChangPing district, CN); Xiaobo Wang (Beijing, CN); He Fang Zhang (Beijing, CN); Yu Liu (Beijing, CN); Peng Hui Jiang (Beijing, CN)

Assignee: International Business Machines Corporation

G06F16/283G06F16/2282G06F16/2379G06F16/2462

Abstract

Claims (57)

1. A computer-implemented method, comprising:

extracting statistics for each columnar table from an online analytical processing (OLAP) database catalog, wherein the statistics include optimizer information, retrieval paths indicating access by index, and sequential access;

building a transaction table map from transaction log records of an online transaction processing (OLTP) database, wherein the transaction table map includes the transaction log records for each transaction, along with a transaction identifier, timestamps indicating a longevity of the transaction, wherein the longevity is an elapsed time since a start of the transaction, an operation identifier, and a counter of log records in a transaction;

identifying transactions from the transaction table map with the counter or the longevity exceeding a predefined threshold without the transaction having issued a commit or rollback as qualified transactions;

transforming each qualified transaction into an external table load streaming transaction;

transmitting each the transformed transaction comprising the external table load streaming transaction to an OLAP database, wherein the OLAP database catalog stores statistics for the OLAP database;

replaying the transformed transaction at the OLAP database, further comprising:

receiving the transformed transaction at the OLAP database;

creating a uniquely identified named pipe for each external load statement in the transformed transaction, wherein the named pipe is uniquely identified with a combination of transaction identifier from the transformed transaction and table identifier from the OLAP database catalog, and the named pipe replaces the placeholders in the load statement; and

executing the transformed transaction.

2. The computer-implemented method of claim 1 , wherein the statistics identify a transaction that is a candidate for transformation.

3. The computer-implemented method of claim 1 , wherein the transforming the transaction further comprises:

creating an external table load statement, comprising an insert operation, an identifier of a data delimiter, a placeholder naming an external table; and

following the creating of the external table load statement, inserting a plurality of data delimited data values from the transaction, and a commit statement.

4. The computer-implemented method of claim 1 , wherein based on the transaction not exceeding the counter or the longevity:

sending the log records of the transaction directly to the OLAP system without transformation as an as-is transaction;

sorting, at the OLAP database, the log records of the as-is transaction by table and timestamp within table; and

replaying the log records at the OLAP database.

5. A computer program product, the computer program product comprising a non-transitory tangible storage device having program code embodied therewith, the program code executable by a processor of a computer to perform a method, the method comprising:

extracting statistics for each columnar table from an online analytical processing (OLAP) database catalog, wherein the statistics include optimizer information, retrieval paths indicating access by index, and sequential access;

building a transaction table map from transaction log records of an online transaction processing (OLTP) database, wherein the transaction table map includes the transaction log records for each transaction, along with a transaction identifier, timestamps indicating a longevity of the transaction, wherein the longevity is an elapsed time since a start of the transaction, an operation identifier, and a counter of log records in a transaction;

identifying transactions from the transaction table map with the counter or the longevity exceeding a predefined threshold without the transaction having issued a commit or rollback as qualified transactions;

transforming each qualified transaction into an external table load streaming transaction;

transmitting each transformed transaction comprising the external table load streaming transaction to an OLAP database, wherein the OLAP database catalog stores statistics for the OLAP database;

replaying the transformed transaction at the OLAP database, further comprising:

receiving the transformed transaction at the OLAP database;

creating a uniquely identified named pipe for each external load statement in the transformed transaction, wherein the named pipe is uniquely identified with a combination of transaction identifier from the transformed transaction and table identifier from the OLAP database catalog, and the named pipe replaces the placeholders in the load statement; and

executing the transformed transaction.

6. The computer program product of claim 5 , wherein the statistics identify a transaction that is a candidate for transformation.

7. The computer program product of claim 5 , wherein the transforming the transaction further comprises:

creating an external table load statement, comprising an insert operation, an identifier of a data delimiter, a placeholder naming an external table; and

following the creating of the external table load statement, inserting a plurality of data delimited data values from the transaction, and a commit statement.

8. The computer-implemented method of claim 1 , wherein based on the transaction not exceeding the counter or the longevity;

sending the log records of the transaction directly to the OLAP system without transformation as an as-is transaction;

sorting, at the OLAP database, the log records of the as-is transaction by table and timestamp within table; and

replaying the log records at the OLAP database.

9. The computer program product of claim 5 , wherein the enhanced compiler executable code output is based on granularity of inner computation unit requirements of individual threads, and wherein the enhanced scheduler dispatches based on granularity of the thread.

10. A computer system, comprising:

one or more processors;

a memory coupled to at least one of the processors;

a set of computer program instructions stored in the memory and executed by at least one of the processors in order to perform actions of:

extracting statistics for each columnar table from an online analytical processing (OLAP) database catalog, wherein the statistics include optimizer information, retrieval paths indicating access by index, and sequential access;

building a transaction table map from transaction log records of an online transaction processing (OLTP) database, wherein the transaction table map includes the transaction log records for each transaction, along with a transaction identifier, timestamps indicating a longevity of the transaction, wherein the longevity is an elapsed time since a start of the transaction, an operation identifier, and a counter of log records in a transaction;

identifying transactions from the transaction table map with the counter or the longevity exceeding a predefined threshold without the transaction having issued a commit or rollback as qualified transactions;

transforming each qualified transaction into an external table load streaming transaction;

transmitting each transformed transaction comprising the external table load streaming transaction to an OLAP database, wherein the OLAP database catalog stores statistics for the OLAP database;

replaying the transformed transaction at the OLAP database, further comprising:

receiving the transformed transaction at the OLAP database;

creating a uniquely identified named pipe for each external load statement in the transformed transaction, wherein the named pipe is uniquely identified with a combination of transaction identifier from the transformed transaction and table identifier from the OLAP database catalog, and the named pipe replaces the placeholders in the load statement; and

executing the transformed transaction.

11. The computer system of claim 10 , wherein the statistics identify a transaction that is a candidate for transformation.

12. The computer system of claim 10 , wherein the transforming the transaction further comprises:

creating an external table load statement, comprising an insert operation, an identifier of a data delimiter, a placeholder naming an external table; and

following the creating of the external table load statement, inserting a plurality of data delimited data values of the transaction, and a commit statement.

13. The computer system of claim 10 , wherein:

a transaction not exceeding the counter or the longevity is an as-is transaction;

the log records of an as-is transaction are sent directly to the OLAP database without transformation, sorted by table and timestamp within the transaction, and replayed at the OLAP database.

Assignments (1)

Continuity (1)

References Cited (26)