Query execution against an in-memory standby database
Techniques related to query execution against an in-memory standby database are disclosed. A first database includes PF data stored on persistent storage in a persistent format. The first database is accessible to a first database server that converts the PF data to a mirror format to produce MF data that is stored within volatile memory. The first database server receives, from a second database server, one or more change records indicating one or more transactions performed against a second database. The one or more change records are applied to the PF data, and a reference timestamp is advanced from a first to a second timestamp. The first database server invalidates any MF data that is changed by a subset of the one or more transactions that committed between the first and second timestamps.
1. A method comprising:
maintaining, on persistent storage, a first database that is accessible to a first database server, wherein said first database is a standby for a second database that is accessible at a second database server;
wherein said first database includes a set of persistent format data (PF data), stored on said persistent storage, in a persistent format;
converting said set of PF data within said first database to a mirror format to produce a set of mirror format data (MF data) that is accessible to said first database server;
storing said set of MF data within volatile memory;
wherein said mirror format is different from and independent of said persistent format;
receiving, from the second database server by said first database server, a plurality of change records that specify changes to one or more data blocks in a second database, said changes to said one or more data blocks in said second database being made by a plurality of transactions performed against said second database;
applying, by said first database server, at least said plurality of change records that specify changes to said one or more data blocks in said second database to said set of PF data within said first database;
wherein applying, by said first database server, at least said plurality of change records includes committing a batch of transactions, said batch of transactions including said plurality of transactions;
after applying, by said first database server, at least said plurality of change records:
advancing a reference timestamp from a first timestamp to a second timestamp;
when advancing said reference timestamp, invalidating any of said set of MF data that is changed by said batch of transactions that committed between said first timestamp and said second timestamp;
comparing said reference timestamp to a query time as of which said first database server is to execute a query; and
when said reference timestamp is greater than or equal to said query time, delaying execution of said query; and
wherein said method is performed by one or more computing devices.
2. The method of claim 1 , wherein said applying, by said first database server, at least said plurality of change records that specify changes to said one or more data blocks in said second database is performed via parallel processes.
3. The method of claim 2 , wherein said plurality of change records are allocated among said parallel processes based on data block addresses of PF data within said first database affected by said plurality of change records.
4. The method of claim 2 , wherein said reference timestamp is a consensus timestamp of said parallel processes.
5. The method of claim 1 , further comprising, prior to invalidating any of said set of MF data that is changed by any of said batch of transactions that committed between said first timestamp and said second timestamp, recording, in a transaction log, one or more operations for each transaction of any of said batch of transactions that changes any of said set of MF data.
6. The method of claim 5 , wherein recording, in said transaction log, one or more operations for each transaction of any said batch of transactions that changes any of said set of MF data is performed via parallel processes that are each allocated a different memory address range of said transaction log.
7. The method of claim 1 , wherein invalidating any of said set of MF data that is changed by any of said batch of transactions that committed between said first timestamp and said second timestamp comprises setting, in a bitmap, one or more bits that correspond to said any of said set of MF data.
8. The method of claim 1 , wherein invalidating any of said set of MF data that is changed by any of said batch of transactions that committed between said first timestamp and said second timestamp comprises traversing a commit log that indicates which of said batch of transactions have committed.
9. The method of claim 1 , wherein receiving, from the second database server by said first database server, said plurality of change records comprises merging said plurality of change records from multiple instances of said second database server.
10. One or more non-transitory storage media storing a set of instructions which, when processed by one or more computing devices, cause:
maintaining, on persistent storage, a first database that is accessible to a first database server, wherein said first database is a standby for a second database that is accessible at a second database server;
wherein said first database includes a set of persistent format data (PF data), stored on said persistent storage, in a persistent format;
converting said set of PF data within said first database to a mirror format to produce a set of mirror format data (MF data) that is accessible to said first database server;
storing said set of MF data within volatile memory;
wherein said mirror format is different from and independent of said persistent format;
receiving, from the second database server by said first database server, a plurality of change records that specify changes to one or more data blocks in a second database, said changes to said one or more data blocks in said second database being made by a plurality of transactions performed against said second database;
applying, by said first database server, at least said plurality of change records that specify changes to said one or more data blocks in said second database to said set of PF data within said first database;
wherein applying, by said first database server, at least said plurality of change records includes committing a batch of transactions, said batch of transactions including said plurality of transactions;
after applying, by said first database server, at least said plurality of change records:
advancing a reference timestamp from a first timestamp to a second timestamp;
when advancing said reference timestamp, invalidating any of said set of MF data that is changed by said batch of transactions that committed between said first timestamp and said second timestamp;
comparing said reference timestamp to a query time as of which said first database server is to execute a query; and
when said reference timestamp is greater than or equal to said query time, delaying execution of said query.
11. The one or more non-transitory storage media of claim 10 , wherein said applying, by said first database server, at least said plurality of change records that specify changes to said one or more data blocks in said second database is performed via parallel processes.
12. The one or more non-transitory storage media of claim 11 , wherein said plurality of change records are allocated among said parallel processes based on data block addresses of PF data within said first database affected by said plurality of change records.
13. The one or more non-transitory storage media of claim 11 , wherein said reference timestamp is a consensus timestamp of said parallel processes.
14. The one or more non-transitory storage media of claim 10 , wherein said set of instructions further comprises instructions, which when executed by said one or more computing devices, cause, prior to invalidating any of said set of MF data that is changed by any of said batch of transactions that committed between said first timestamp and said second timestamp, recording, in a transaction log, one or more operations for each transaction of any of said batch of transactions that changes any of said set of MF data.
15. The one or more non-transitory storage media of claim 14 , wherein said set of instructions further comprises instructions, which when executed by said one or more computing devices, cause recording, in said transaction log, one or more operations for each transaction of any said batch of transactions that changes any of said set of MF data to be performed via parallel processes that are each allocated a different memory address range of said transaction log.
16. The one or more non-transitory storage media of claim 10 , wherein invalidating any of said set of MF data that is changed by any of said batch of transactions that committed between said first timestamp and said second timestamp comprises setting, in a bitmap, one or more bits that correspond to said any of said set of MF data.
17. The one or more non-transitory storage media of claim 10 , wherein invalidating any of said set of MF data that is changed by any of said batch of transactions that committed between said first timestamp and said second timestamp comprises traversing a commit log that indicates which of said batch of transactions have committed.
18. The one or more non-transitory storage media of claim 10 , wherein receiving, from the second database server by said first database server, said plurality of change records comprises merging said plurality of change records from multiple instances of said second database server.