Method and apparatus for reading data maintained in a tree data structure
View Patent ↗The present disclosure provides a method of reading data maintained in a tree data structure, such as B+ tree, using near data processing (NDP) in a cloud native database. According to embodiments, a desired LSN will be used in NDP page reads on the master computing node (e.g. master SQL node). When the master computing node (e.g. master SQL node) reads the regular page, the maximum desired LSN (e.g. the latest page version number) for that regular page will be used. Embodiments use features of the desired LSN and page locking, wherein correct versions of pages can be obtained by using the desired LSN associated with a page, in combination with page locking, and can enable the reading of a consistent tree structure and achieve good read/write concurrency.
1. A method for obtaining one or more pages in response to a query using near data processing (NDP) in a cloud native database, the method comprising:
receiving the query, the query including information indicative of one or more required pages, the information further indicative of a version of each of the one or more required pages;
scanning a general buffer pool to identify one or more of the required pages;
upon identification of one or more of the required pages in the general buffer pool, copying the identified one or more required pages into a private buffer pool portion of the general buffer pool assigned to the query;
upon identification of no further required pages in the general buffer pool, sending a request to one or more storage nodes for one or more of the required pages remaining to be identified;
receiving the one or more remaining required pages; and
copying the received one or more remaining required pages into the private buffer pool.
2. The method of claim 1 , further comprising applying a shared lock to the identified one or more required pages prior to copying the identified one or more required pages.
3. The method of claim 2 , further comprising releasing the shared lock once the identified one or more required pages are copied.
4. The method of claim 1 , wherein the version of the one or more required pages is defined by a log sequence number (LSN).
5. The method of claim 4 , wherein LSN maximum defines a latest version.
6. The method of claim 1 , further comprising applying a shared lock on a root of a B+ tree associated with the one or more required pages.
7. The method of claim 6 , further comprising releasing the shared lock on the root of the B+ tree.
8. The method of claim 1 , further comprising copying the received one or more remaining required pages into the general buffer pool.
9. A device supporting near data processing (NDP) in a cloud native database, the device comprising:
a network interface for receiving data from and transmitting data to devices connected to a network;
a processor; and
machine readable memory storing machine executable instructions which when executed by the processor configure the device to:
receive a query, the query including information indicative of one or more required pages, the information further indicative of a version of each of the one or more required pages;
scan a general buffer pool to identify one or more of the required pages;
upon identification of one or more of the required pages in the general buffer pool, copy the identified one or more required pages into a private buffer pool portion of the buffer pool assigned to the query;
upon identification of no further required pages in the general buffer pool, send a request to one or more storage nodes for one or more of the required pages remaining to be identified;
receive the one or more remaining required pages; and
copy the received one or more remaining required pages into the private buffer pool.
10. A method of reading data in a data tree using near data processing (NDP) in a cloud native database, the method comprising:
applying a shared page lock on a root and internal pages across the data tree, in a top-down manner, until reaching P0, the P0 referring to a page at a level immediately above the leaf level of the data tree;
acquiring a desired log sequence number (LSN) while holding the shared page lock on the P0;
after acquiring the desired LSN, for each extracted child page:
allocating a page from a free list of a buffer pool to a NDP cache area of the buffer pool, the NDP cache area designated for a specific query,
upon allocating the page to the NDP cache area, determining whether the child page is found at a regular page area of the buffer pool, and
securing the child page to the page allocated in the NDP cache area based on the determination;
releasing the shared page lock applied on the P0;
processing the pages allocated to the NDP cache area; and
upon completion of processing, releasing each of the pages allocated to the NDP cache area back to the free list of the buffer pool.
11. The method of claim 10 , further comprising:
upon completion of processing the pages allocated to the NDP cache area, determining a next-to-visit page based on whether the last child page is in the NDP cache area or in the regular page area.
12. The method of claim 11 , wherein the last child page is in the NDP cache area, and the next-to-visit page is determined based upon another search on the data tree initiated using the last record on the last child page.
13. The method of claim 11 , wherein the last child page is in the regular page area, the next-to-visit page is determined based on a next page identifier (ID) of the last child page.
14. The method of claim 10 , wherein the child page is found at a regular page area of the buffer pool, the securing the child page includes:
for each child page:
applying shared page lock on the child page;
copying the child page found at the regular page area to the page allocated in the NDP cache area; and
releasing the shared page lock on the child page.
15. The method of claim 10 , wherein the child page is unfound at a regular page area of the buffer pool, the securing the child page includes:
submitting, to a storage node, an asynchronous NDP request on the child page using the desired LSN.
16. The method of claim 15 , wherein multiple child pages are submitted to a storage node via one input/output (IO) request.
17. The method of claim 10 , wherein the desired LSN is valid only for a period of reading the child pages of the P0.
18. The method of claim 10 , wherein allocating and releasing of the pages allocated to the NDP cache area are solely under control of the query that requests the NDP.
19. The method of claim 10 , wherein the data tree is a B+ tree.
20. A device supporting near data processing (NDP) in a cloud native database, comprising:
a network interface for receiving data from and transmitting data to devices connected to a network;
a processor; and
machine readable memory storing machine executable instructions which when executed by the processor configure the device to:
apply shared page lock on root and internal pages across the data tree, in a top-down manner, until reaching P0, the P0 referring to a page at a level immediately above the leaf level of the data tree;
acquire a desired log sequence number (LSN) while holding the shared page lock on the P0;
after acquiring the desired LSN, for each extracted child page:
allocate a page from a free list of a buffer pool to a NDP cache area of the buffer pool, the NDP cache area designated for a specific query,
upon allocating the page to the NDP cache area, determine whether the child page is found at a regular page area of the buffer pool, and
secure the child page to the page allocated in the NDP cache area based on the determination;
release the shared page lock applied on the P0;
process the pages allocated to the NDP cache area; and
upon completion of processing, release each of the pages allocated to the NDP cache area back to the free list of the buffer pool.