Expanding variable sub-column widths as needed to store data in memory
A set of data storage values is received. It is determined that a data storage value in the set will not fit in an available memory segment including variable data column widths based at least in part on data sizes specified in a plurality of segment layout maps. A memory segment is selected for which a column width of a column will be expanded. A column width of the selected memory segment is expanded. A segment layout map corresponding to the selected memory segment is updated. The set of data storage values is stored in the selected memory segment.
1. A system, comprising:
a processor configured to:
select a first column included in an available memory segment to store a data storage value, wherein:
the first column comprises a first sub-column that stores data of a type associated with the data storage value; and
a corresponding variable column width of the first sub-column is not more than a maximum allowable column width associated with the first column;
determine that increasing a data size associated with the corresponding variable column width of the first sub-column to accommodate a size associated with the data storage value causes a column width of the first column to exceed the maximum allowable column width associated with the first column; and
in response to the determination, store the data storage value to a second column of the available memory segment that includes a second sub-column that has a corresponding variable column width, wherein a data size associated with the corresponding variable column width of the second sub-column can be increased to the size associated with the data storage value; and
a memory coupled to the processor and configured to provide the processor with instructions.
2. The system of claim 1 , wherein the data size associated with the corresponding variable column width of the first sub-column is stored as a mask value in a segment layout map associated with the available memory segment.
3. The system of claim 1 , wherein to increase the data size associated with the corresponding variable column width of the first sub-column to accommodate the size associated with the data storage value comprises to update a mask value associated with the data size associated with the corresponding variable column width of the first sub-column that is stored in a segment layout map associated with the available memory segment.
4. The system of claim 1 , wherein the processor is further configured to determine that the data size associated with the corresponding variable column width of the second sub-column is to be at least expanded to accommodate the size associated with the data storage value.
5. The system of claim 1 , wherein the processor is further configured to select the second column of the available memory segment based at least in part on a determination that the data size associated with the corresponding variable column width of the second sub-column is to be at least expanded to accommodate the size associated with the data storage value.
6. The system of claim 1 , wherein to store the data storage value comprises to add, to the available memory segment, a row comprising the data storage value.
7. The system of claim 1 , wherein the processor is further configured to update a mask value associated with the second sub-column stored in a segment layout map associated with the available memory segment to be at least the size associated with the data storage value.
8. The system of claim 1 , wherein the processor is further configured to:
determine that the available memory segment is full; and
divide the available memory segment into a plurality of memory segments.
9. The system of claim 1 , wherein the processor is further configured to divide the available memory segment into a plurality of memory segments.
10. The system of claim 9 , wherein to divide the available memory segment into the plurality of memory segments includes to:
determine a first set of rows and a second set of rows included in the available memory segment based at least in part on clustering; and
divide the available memory segment into the plurality of memory segments comprising a first memory segment including the first set of rows and a second memory segment including the second set of rows.
11. The system of claim 9 , wherein the processor is further configured to combine a first memory segment from the plurality of memory segments with a second memory segment from the plurality of memory segments.
12. The system of claim 1 , wherein the processor is further configured to:
divide the available memory segment into a plurality of memory segments; and
combine a first memory segment of the plurality of memory segments with a second memory segment divided from a separate memory segment.
13. The system of claim 1 , wherein to store the data storage value further comprises to:
determine that storing the data storage value in the available memory segment would result in the available memory segment filling up;
divide the available memory segment into a plurality of memory segments based at least in part on the determination; and
store the data storage value in one of the plurality of memory segments.
14. The system of claim 1 , wherein the processor is further configured to:
receive a read request including one or more of the following: a column identifier, a row identifier, an offset, and a mask value; and
read a requested data storage value from the available memory segment based at least in part on one or more of the column identifier, the row identifier, the offset, and the mask value.
15. A method, comprising:
selecting a first column included in an available memory segment to store a data storage value, wherein:
the first column comprises a first sub-column that stores data of a type associated with the data storage value; and
a corresponding variable column width of the first sub-column is not more than a maximum allowable column width associated with the first column;
determining that increasing a data size associated with the corresponding variable column width of the first sub-column to accommodate a size associated with the data storage value causes a column width of the first column to exceed the maximum allowable column width associated with the first column; and
in response to the determination, storing the data storage value to a second column of the available memory segment that includes a second sub-column that has a corresponding variable column width, wherein a data size associated with the corresponding variable column width of the second sub-column can be increased to the size associated with the data storage value.
16. The method of claim 15 , wherein increasing the data size associated with the corresponding variable column width of the first sub-column to accommodate the size associated with the data storage value comprises updating a mask value associated with the data size associated with the corresponding variable column width of the first sub-column that is stored in a segment layout map associated with the available memory segment.
17. The method of claim 15 , further comprising determining that the data size associated with the corresponding variable column width of the second sub-column is to be at least expanded to accommodate the size associated with the data storage value.
18. The method of claim 15 , further comprising selecting the second column of the available memory segment based at least in part on a determination that the data size associated with the corresponding variable column width of the second sub-column is to be at least expanded to accommodate the size associated with the data storage value.
19. The method of claim 15 , further comprising updating a mask value associated with the second sub-column stored in a segment layout map associated with the available memory segment to be at least the size associated with the data storage value.
20. A computer program product, the computer program product being embodied in a tangible non-transitory computer readable storage medium and comprising computer instructions for:
selecting a first column included in an available memory segment to store a data storage value, wherein:
the first column comprises a first sub-column that stores data of a type associated with the data storage value; and
a corresponding variable column width of the first sub-column is not more than a maximum allowable column width associated with the first column;
determining that increasing a data size associated with the corresponding variable column width of the first sub-column to accommodate a size associated with the data storage value causes a column width of the first column to exceed the maximum allowable column width associated with the first column; and
in response to the determination, storing the data storage value to a second column of the available memory segment that includes a second sub-column that has a corresponding variable column width, wherein a data size associated with the corresponding variable column width of the second sub-column can be increased to the size associated with the data storage value.