DATA TRANSITION IN HIGHLY PARALLEL DATABASE MANAGEMENT SYSTEM
A method by one or more processing units of a database management system includes transitioning storage of a collection of data from storage in a set of first temperature storage nodes of the database management system to storage in a set of second temperature storage nodes of the database management system based on one or more criteria associated with the collection of data, where each first temperature storage node of the set of first temperature storage nodes is paired with a corresponding second temperature storage node of the set of second temperature storage nodes to form a set of parallel node pairs. The method further includes, after determining the collection of data has been stored in the set of second temperature storage nodes, deleting the collection of data in the set of first temperature storage nodes.
1 . A method for execution by one or more processing units of a database management system, the method comprising:
transitioning storage of a collection of data from storage in a set of first temperature storage nodes of the database management system to storage in a set of second temperature storage nodes of the database management system based on one or more criteria associated with the collection of data, wherein each first temperature storage node of the set of first temperature storage nodes is paired with a corresponding second temperature storage node of the set of second temperature storage nodes to form a set of parallel node pairs; and
after determining the collection of data has been stored in the set of second temperature storage nodes, deleting the collection of data in the set of first temperature storage nodes.
2 . The method of claim 1 , wherein a criterion of the one or more criteria comprises:
a data transition rule.
3 . The method of claim 1 , wherein a criterion of the one or more criteria comprises:
an age of the collection of data.
4 . The method of claim 1 , wherein a criterion of the one or more criteria comprises:
a least recently used criterion.
5 . The method of claim 1 , wherein a criterion of the one or more criteria comprises:
a data access pattern.
6 . The method of claim 5 further comprises:
periodically determining whether the data access pattern compares unfavorably to a data access pattern threshold; and
when determining the data access pattern compares unfavorably to the data access pattern threshold, determining to perform the transitioning.
7 . The method of claim 1 further comprises
determining an age of the collection of data exceeds an age threshold; and
transitioning the storage based on the age of the collection of data exceeding the age threshold.
8 . The method of claim 1 further comprises:
determining the collection of data has been stored in the set of second temperature nodes; and
deleting the collection of data by the set of first temperature storage nodes.
9 . The method of claim 8 , wherein the determining the collection of data has been stored comprises:
receiving a confirmation of a successful write from the set of second temperature storage nodes.
10 . The method of claim 1 , wherein the transitioning storage of the collection of data in parallel is via remote direct memory access links.
11 . The method of claim 1 further comprises:
prior to the transitioning storage:
determining the collection of data is of a first temperature, wherein the first temperature indicates an expected access rate of the collection of data is above a first access rate threshold; and
storing the collection of data in the set of first temperature storage nodes.
12 . The method of claim 1 further comprises:
determining to transition the collection of data from storage in the set of second temperature storage nodes to storage in a set of third temperature nodes based on the one or more criteria associated with the collection of data, wherein each second temperature storage node of the set of second temperature storage nodes is paired with a corresponding third temperature storage node of the set of third temperature storage nodes to form a second set of parallel node pairs; and
transitioning storage of the collection of data, by each second parallel node pair of the second set of parallel node pairs, from storage in the set of second temperature storage nodes to storage in the set of third temperature storage nodes.
13 . The method of claim 12 further comprises:
determining to transition the collection of data from storage in the set of third temperature storage nodes to storage in a set of fourth temperature nodes based on the one or more criteria associated with the collection of data, wherein each third temperature storage node of the set of third temperature storage nodes is paired with a corresponding fourth temperature storage node of the set of fourth temperature storage nodes to form a third set of parallel node pairs; and
transitioning storage of the collection of data, by each third parallel node pair of the third set of parallel node pairs, from storage in the set of third temperature storage nodes to storage in the set of fourth temperature storage nodes.
14 . The method of claim 13 , wherein storage devices of the set of second temperature storage nodes comprise non-volatile random access memory.
15 . The method of claim 13 , wherein storage devices of the set of third temperature storage nodes comprise serial advanced technology attachment (SATA) solid state drives (SSD) memory.
16 . The method of claim 13 , wherein storage devices of the set of fourth temperature storage nodes comprise archival storage devices.
17 . The method of claim 1 , wherein storage devices of the set of first temperature storage nodes comprise dynamic random access memory.
18 . The method of claim 1 , wherein storage devices of the set of first temperature storage nodes comprise flash memory.
19 . The method of claim 18 , wherein storage devices of the set of second temperature storage nodes comprise non-volatile random access memory.