Method of performing transactional and analytical data processing using a data structure
Disclosed herein is a computer-implemented method of processing a query using a columnstore comprising a plurality of segments, the method comprising: receiving a query comprising a parameter; determining a key corresponding to the parameter; identifying a mapping structure, from a plurality of mapping structures, relating to said key, wherein each mapping structure corresponds to a respective segment of the columnstore; interrogating the identified mapping structure to determine a value corresponding to the key, wherein the value identifies an entry of an index table of a plurality of index tables that corresponds to the respective segment, wherein the entry of the index table identifies a row of the respective segment of the columnstore relating to the parameter; and retrieving data relating to the parameter from a data source based on data stored in the entry of the index table.
1. A computer-implemented method of updating data, the method comprising:
obtaining a columnstore comprising a plurality of columns and a plurality of rows, wherein the columnstore is segmented into a plurality of segments and each of the plurality of columns is segmented into a plurality of column segments, wherein each segment of the columnstore comprises a plurality of column segments of the respective columns and each column segment of a respective column comprises a plurality of rows;
receiving a query comprising a parameter;
determining a key corresponding to the parameter;
identifying a mapping structure, from a plurality of mapping structures, relating to said key, wherein the identified mapping structure corresponds to a respective segment of the columnstore;
interrogating the identified mapping structure to determine a value corresponding to the key, wherein the value identifies an entry of an index table of a plurality of index tables that corresponds to a first said respective segment of the columnstore, wherein the identified entry of the index table identifies a row of a first column segment of the first said respective segment of the columnstore that contains a first entry containing data matching a value of the parameter of the query and the row having other entries in different one or more column segments, respectively;
moving the identified row to a rowstore;
manipulating the data of the first entry in the rowstore based upon an update request;
activating an access lock on an identifier of the row whilst the data of the first entry is manipulated in the rowstore, whereby rows other than the identified row for which the access lock has been activated are available for access such that other concurrent queries are allowed to access data within entries of other rows in the segment of the columnstore from which the identified row was moved;
releasing the access lock after the manipulating is completed; and
in response to the manipulating, updating a status of the identified row to indicate that updates to the retrieved data of the first entry of the identified row are located in the rowstore.
2. The computer-implemented method of claim 1 , comprising generating the rowstore comprising a plurality of rows including the identified row of the columnstore.
3. The computer-implemented method of claim 1 , comprising creating a column segment comprising the data stored within the rowstore, the column segment to be inserted into the columnstore.
4. The computer-implemented method of claim 1 , comprising storing the rowstore in a random access memory.
5. The computer-implemented method of claim 1 , further comprising:
merging the plurality of mapping structures to form a single mapping structure.
6. The computer-implemented method of claim 1 , comprising generating a data structure relating to the columnstore and for use in processing a query according to claim 1 , the generating comprising:
generating the mapping structure and the index table for the respective segment of the columnstore, wherein the mapping structure and the index table comprise at least one entry, the at least one entry of the respective mapping structure comprising a mapping between a key and a value identifying the at least one entry of the corresponding index table comprising a mapping between the key and a row in the respective segment of the columnstore.
7. The computer-implemented method of claim 1 , further comprising:
for each entry of the identified row in a column segment of the different one or more column segments that corresponds to the query:
determining an offset within the respective column segment based on a location of the row in the segment; and
retrieving, from the respective column segment and based on the respective offset, data of the entry.
8. A non-transitory computer readable medium comprising instructions that, when executed by a processor of a computer, cause a computer comprising the processor to carry out the method of claim 1 .
9. The computer-implemented method of claim 3 , wherein the column segment is created after a predetermined period of time or a criterion relating to a predetermined size of the rowstore has been met.
10. A database system comprising a database having data stored in a columnstore, a processor, and memory comprising processor readable instructions, which when executed on the processor cause the processor to:
obtain a columnstore comprising a plurality of columns and a plurality of rows, wherein the columnstore is segmented into a plurality of segments and each of the plurality of columns is segmented into a plurality of column segments, wherein each segment of the columnstore comprises a plurality of column segments of the respective columns and each column segment of a respective column comprises a plurality of rows;
receive a query comprising a parameter;
determine a key corresponding to the parameter;
identify a mapping structure, from a plurality of mapping structures, relating to said key, wherein the identified mapping structure corresponds to a respective segment of the columnstore;
interrogate the identified mapping structure to determine a value corresponding to the key, wherein the value identifies an entry of an index table of a plurality of index tables that corresponds to a first said respective segment of the columnstore, wherein the identified entry of the index table identifies a row of a first column segment of the first said respective segment of the columnstore that contains a first entry containing data matching a value of the parameter of the query and the row having other entries in different one or more column segments, respectively;
move the identified row to a rowstore;
manipulate the data of the first entry in the rowstore based upon an update request;
activate an access lock on an identifier of the row whilst the data of the first entry is manipulated in the rowstore, whereby rows other than the identified row for which the access lock has been activated are available for access such that other concurrent queries are allowed to access data within entries of other rows in the segment of the columnstore from which the identified row was moved;
release the access lock after the manipulating is completed; and
in response to the manipulation, update a status of the identified row to indicate that updates to the retrieved data of the first entry of the identified row are located in the rowstore.
11. The database system of claim 10 , wherein the processor readable instructions, when executed, further cause the processor to:
generate the rowstore comprising a plurality of rows including the identified row of the columnstore.
12. The database system of claim 10 , wherein the processor readable instructions, when executed, further cause the processor to:
create a column segment comprising the data stored within the rowstore, the column segment to be inserted into the columnstore.
13. The database system of claim 10 , wherein the processor readable instructions, when executed, further cause the processor to:
store the rowstore in a random access memory.
14. The database system of claim 10 , wherein the processor readable instructions, when executed, further cause the processor to:
merge the plurality of mapping structures to form a single mapping structure.
15. The database system of claim 10 , wherein the processor readable instructions, when executed, further cause the processor to:
generate a mapping structure and an index table for each of the plurality of segments of the columnstore, wherein each mapping structure and each index table comprise at least one entry, each entry of the respective mapping structure comprising a mapping between a key and a value identifying at least one entry of the corresponding index table comprising a mapping between the key and a row in a respective segment of the plurality of segments.
16. The database system of claim 10 , wherein the processor readable instructions, when executed, further cause the processor to:
for each entry of the identified row in a column of the different one or more column segments that corresponds to the query:
determine an offset within the respective column based on a location of the row in the segment; and
retrieve, from the respective column and based on the respective offset, data of the entry.
17. The database system of claim 12 , wherein the column segment is created after a predetermined period of time or a criterion relating to a predetermined size of the rowstore has been met.