Systems and methods for database management using append-only storage devices
View Patent ↗An apparatus is disclosed. The apparatus may be implemented in a database node or a storage node and includes one or more processors and memory storing instructions for causing the processor to perform a number of operations. Responsive to a page write request, the apparatus determines an identifier corresponding to the requested page, sends component blocks corresponding to the page to a storage node for appending to an append-only storage log of the storage node, receives the physical location of the stored component blocks, and associates the physical storage location of the stored component blocks with the logical identifier corresponding to the page.
1. A method comprising:
receiving, by an apparatus, a write request instructing appending a component block in a storage device, wherein the component block includes a delta over a most recent complete version of a page, the most recent complete version of the page stored in an append-only storage log in the storage device;
sending, by the apparatus to the storage device, the component block; and
receiving, by the apparatus from the storage device, information about a physical location of the component block appended to the append-only storage log in the storage device;
recording, by the apparatus, an association between a logical identifier of the page and the information about the physical location of the component block;
receiving, by the apparatus, a read request for the page;
determining, by the apparatus, the information about the physical location of the component block based on the logical identifier of the page and the association;
retrieving, by the apparatus from the storage device, the component block using the information about the physical location of the component block;
retrieving, by the apparatus from the storage device, the most recent complete version of the page using the logical identifier of the page;
constructing, by the apparatus, a current version of the page based on the most recent complete version of the page and the component block; and
returning, by the apparatus, the current version of the page.
2. The method of claim 1 , wherein the component block is stored outside the most recent complete version of the page.
3. The method of claim 1 , wherein the component block and the most recent complete version of the page are retrieved in parallel.
4. The method of claim 1 , the constructing comprising:
constructing, by the apparatus, the current version of the page based on the most recent complete version of the page and a plurality of component blocks, the plurality of component blocks including the component block.
5. The method of claim 1 , further comprising:
deleting, by the apparatus, the most recent complete version of the page after the current version of the page is constructed.
6. The method of claim 1 , wherein the component block and the most recent complete version of the page coexist in the append-only storage log of the storage device for a period of time.
7. An apparatus comprising:
at least one processor; and
a computer readable storage medium storing programming, the programming including instructions that, when executed by the at least one processor, cause the apparatus to perform:
receiving a write request instructing appending a component block in a storage device, wherein the component block includes a delta over a most recent complete version of a page, the most recent complete version of the page stored in an append-only storage log in the storage device;
sending, to the storage device, the component block; and
receiving, from the storage device, information about a physical location of the component block appended to the append-only storage log in the storage device;
recording an association between a logical identifier of the page and the information about the physical location of the component block;
receiving a read request for the page;
determining the information about the physical location of the component block based on the logical identifier of the page and the association;
retrieving, from the storage device, the component block using the information about the physical location of the component block;
retrieving, from the storage device, the most recent complete version of the page using the logical identifier of the page;
constructing a current version of the page based on the most recent complete version of the page and the component block; and
returning the current version of the page.
8. The apparatus of claim 7 , wherein the component block is stored outside the most recent complete version of the page.
9. The apparatus of claim 7 , wherein the component block and the most recent complete version of the page are retrieved in parallel.
10. The apparatus of claim 7 , the constructing comprising:
constructing the current version of the page based on the most recent complete version of the page and a plurality of component blocks, the plurality of component blocks including the component block.
11. The apparatus of claim 7 , the instructions further causing the apparatus to perform:
deleting the most recent complete version of the page after the current version of the page is constructed.
12. The apparatus of claim 7 , wherein the component block and the most recent complete version of the page coexist in the append-only storage log of the storage device for a period of time.
13. A non-transitory computer-readable medium having instructions stored thereon that, when executed by an apparatus, cause the apparatus to perform operations, the operations comprising:
receiving a write request instructing appending a component block in a storage device, wherein the component block includes a delta over a most recent complete version of a page, the most recent complete version of the page stored in an append-only storage log in the storage device;
sending, to the storage device, the component block; and
receiving, from the storage device, information about a physical location of the component block appended to the append-only storage log in the storage device;
recording an association between a logical identifier of the page and the information about the physical location of the component block;
receiving a read request for the page;
determining the information about the physical location of the component block based on the logical identifier of the page and the association;
retrieving, from the storage device, the component block using the information about the physical location of the component block;
retrieving, from the storage device, the most recent complete version of the page using the logical identifier of the page;
constructing a current version of the page based on the most recent complete version of the page and the component block; and
returning the current version of the page.
14. The non-transitory computer-readable medium of claim 13 , wherein the component block is stored outside the most recent complete version of the page.
15. The non-transitory computer-readable medium of claim 13 , wherein the component block and the most recent complete version of the page are retrieved in parallel.
16. The non-transitory computer-readable medium of claim 13 , the constructing comprising:
constructing the current version of the page based on the most recent complete version of the page and a plurality of component blocks, the plurality of component blocks including the component block.
17. The non-transitory computer-readable medium of claim 13 , the operations further comprising:
deleting the most recent complete version of the page after the current version of the page is constructed.