Method for creating an in-memory physical dictionary for data compression
View Patent ↗Some aspects of the invention provide methods for creating an in-memory physical dictionary for data compression. To that end, in accordance with aspects of the present invention, a new heuristic is defined for converting each of the plurality of logical nodes into a corresponding physical node forming a plurality of physical nodes; then place each of the physical nodes into the physical dictionary while traversing the dictionary tree in descending visit count order. Each physical node is placed in its nearest ascendant's cache-line with sufficient space. If there is no space in any of the ascendant's cache-line, then the physical node is placed into a new cache-line, unless a pre-defined packing threshold has been reached, in which case the physical node is placed in the first available cache-line.
1. A method for creating an in-memory physical dictionary corresponding to a logical dictionary tree comprising a plurality of logical nodes for data compression comprising the steps of:
converting each of the plurality of logical nodes into a corresponding physical node forming a plurality of physical nodes;
placing a root physical node of the plurality of physical nodes into the physical dictionary; and
placing each of the remaining plurality of physical nodes into the physical dictionary while traversing the dictionary tree in descending visit count order;
wherein placing the root physical node into the physical dictionary comprises the steps of:
placing the root physical node into a new cache-line if a packing threshold has not been reached; and
placing the root physical node into the first available space in any cache-line if the packing threshold has been reached.
2. A method for creating an in-memory physical dictionary corresponding to a logical dictionary tree comprising a plurality of logical nodes for data compression comprising the steps of:
converting each of the plurality of logical nodes into a corresponding physical node forming a plurality of physical nodes;
placing a root physical node of the plurality of physical nodes into the physical dictionary; and
placing each of the remaining plurality of physical nodes into the physical dictionary while traversing the dictionary tree in descending visit count order;
wherein placing each of the remaining plurality of physical nodes into the physical dictionary while traversing the dictionary tree in descending visit count order comprises the steps of:
placing a physical node into the physical dictionary if not a root node; and
placing the children nodes of the physical node into the physical dictionary, if any, in order of descending visit count.
3. The method of claim 2 , wherein the step of placing a physical node into the physical dictionary comprises the steps of:
examining the cache-lines of the ascendant nodes of the physical node in ascending order until either sufficient space is found or the root node is reached;
placing the physical node into the cache-line of the first ascendant node found with sufficient cache-line space; and
placing the physical node into a secondary cache-line if no ascendant node is found with sufficient cache-line space.
4. The method of claim 3 , wherein the secondary cache-line is a new cache-line if a packing threshold has not been reached; and the secondary cache-line is any cache-line with a first available space if the packing threshold has been reached.
5. The method of claim 4 , wherein the packing threshold is defined as the instantaneous proportion of physical dictionary storage occupied by physical nodes.