Methods and systems for hardware acceleration of database operations and queries for a versioned database based on multiple hardware accelerators
Embodiments of the present invention provide a hardware accelerator that assists a host database system in processing its queries. The hardware accelerator comprises special purpose processing elements that are capable of receiving database query/operation tasks in the form of machine code database instructions, execute them in hardware without software, and return the query/operation result back to the host system.
1. A computer-implemented method for hardware-based snapshot versioning of a database, said method comprising:
determining, by one or more processors of a computer, snapshots of data in the database at intervals;
determining, by the one or more processors of the computer, what data of the database is valid for each of the determined snapshots by using multiple validity bitmaps respectively representative of the snapshots, wherein each one of the multiple validity bitmaps includes a plurality of bits, and wherein each bit of the plurality of bits in each one the multiple validity bitmaps corresponds to a row of data of a data table of the database;
identifying, by the one or more processors of the computer, differences in the data of the snapshots based on comparing the multiple validity bitmaps with each other;
determining, by the one or more processors of the computer, based on the identified differences of the multiple validity bitmaps, a set of snapshot bitmap vectors that takes data of one snapshot to the data of another snapshot of the snapshots;
querying, by the one or more processors of the computer, for data of the database during loading of other data based on multi-version concurrency control;
determining, by the one or more processors of the computer, timestamps for the snapshots of data by using the set of snapshot bitmap vectors;
forming, by the one or more processors of the computer, a query plan for a database query of the database based on version filtering using the set of snapshot bitmap vectors; and
executing the formed query plan for the database query by one or more query processing modules using one or more database machine code (MOP) executed in hardware to perform one or more particular tasks of the database query.
2. The method of claim 1 , wherein identifying differences in the data of the snapshots comprises tracking committed versus uncommitted rows in the database.
3. The method of claim 1 , further comprising accelerating snapshot refresh via a database machine code instruction.
4. The method of claim 1 , further comprising determining different snapshots by creating versions of one validity bitmap of the multiple validity bitmaps.
5. The method of claim 4 , wherein differences in each version of the one validity bitmap of the multiple validity bitmaps are determined to take the one validity bitmap of the multiple validity bitmaps from one snapshot to another snapshot.
6. The method of claim 5 , wherein the differences between each version of the one validity bitmap of the multiple validity bitmaps are used to change a validity bitmap from one snapshot to another snapshot.
7. A computing device, comprising:
a non-transitory computer readable storage medium; and
one or more physical processors configured to:
determine snapshots of data in a database at intervals, wherein the snapshots of the data in the database includes snapshots of multiple rows of data of a data table of the database;
determine what data of the database is valid for each of the determined snapshots by using multiple validity bitmaps respectively representative of the snapshots, wherein each one of the multiple validity bitmaps includes a plurality of bits, and wherein each bit of the plurality of bits in each one the multiple validity bitmaps corresponds to a row of data of a data table of the database;
identify, by the one or more processors of the computer, differences in the data of the snapshots based on comparing the multiple validity bitmaps with each other;
determine a set of changes that takes data of one snapshot to the data of another snapshot of the snapshots;
query for data of the database during loading of other data of the database based on multi-version concurrency control;
determine timestamps for the snapshots of data by using the set of snapshot bitmap vectors;
form a query plan for a database query of the database based on version filtering using the snapshot bitmap vectors; and
execute the formed query plan for the database query by one or more query processing modules using one or more database machine code (MOP) executed in hardware to perform one or more particular tasks of the query.
8. The computing device of claim 7 , wherein identifying differences in the data of the snapshots comprises tracking committed versus uncommitted rows in the database.
9. The computing device of claim 8 , wherein the one or more physical processors are further configured to: accelerate snapshot refresh via a database machine code instruction.
10. The computing device of claim 9 , wherein the one or more physical processors are further configured to: determine different snapshots by creating versions of one validity bitmap of the multiple validity bitmaps.
11. The computing device of claim 10 , wherein differences in each version of the one validity bitmap of the multiple validity bitmaps are determined to take the one validity bitmap of the multiple validity bitmaps from one snapshot to another snapshot.
12. The computing device of claim 10 , wherein the differences between each version of the one validity bitmap of the multiple validity bitmaps are used to change a validity bitmap from one snapshot to another snapshot.
13. A non-transitory computer readable storage medium storing at least executable instructions for a method to be executed by a computer that when executed:
determines snapshots of data in a database at intervals, wherein the snapshots of the data in the database includes snapshots of multiple rows of data of a data table of the database;
determines what data of the database is valid for each of the determined snapshots by using multiple validity bitmaps respectively representative of the snapshots, wherein each one of the multiple validity bitmaps includes a plurality of bits, and wherein each bit of the plurality of bits in each one the multiple validity bitmaps corresponds to a row of data of a data table of the database;
identifies, by the one or more processors of the computer, differences in the data of the snapshots based on comparing the multiple validity bitmaps with each other;
determines a set of changes that takes data of one snapshot to the data of another snapshot of the snapshots;
queries for data of the database during loading of other data of the database based on multi-version concurrency control;
determines timestamps for the snapshots of data by using the set of snapshot bitmap vectors;
forms a query plan for a database query of the database based on version filtering using the snapshot bitmap vectors; and
executes the formed query plan for the database query by one or more query processing modules using one or more database machine code (MOP) executed in hardware to perform one or more particular tasks of the query.
14. The non-transitory computer readable storage medium of claim 13 , wherein identifying differences in the data of the snapshots comprises tracking committed versus uncommitted rows in the database.
15. The non-transitory computer readable storage medium of claim 14 , wherein the executable instructions when executed: further accelerates snapshot refresh via a database machine code instruction.