Expanding variable sub-column widths as needed to store data in memory
A set of data storage values is received. It is determined that a data storage value in the set will not fit in an available memory segment including variable data column widths based at least in part on data sizes specified in a plurality of segment layout maps. A memory segment is selected for which a column width of a column will be expanded. A column width of the selected memory segment is expanded. A segment layout map corresponding to the selected memory segment is updated. The set of data storage values is stored in the selected memory segment.
1. A system, comprising:
a processor configured to:
select a column included in a memory segment to store a data storage value, wherein:
the column comprises a sub-column that stores data of a type associated with the data storage value; and
a corresponding variable column width of the sub-column is not more than
a maximum allowable column width associated with the column;
determine that a data size associated with the corresponding variable column width of the sub-column is less than a size associated with the data storage value;
determine that the data size associated with the corresponding variable column width of the sub-column can be increased to accommodate the size associated with the data storage value without causing a column width of the column to exceed the maximum allowable column width associated with the column;
increase the data size associated with the corresponding variable column width of the sub-column to accommodate the size associated with the data storage value; and
store the data storage value in the sub-column; and
a memory coupled to the processor and configured to provide the processor with instructions.
2. The system of claim 1 , wherein the data size associated with the corresponding variable column width of the sub-column comprises a mask size associated with the corresponding variable column width of the sub-column.
3. The system of claim 1 , wherein the column comprises a plurality of rows each comprising vectors of data storage values.
4. The system of claim 1 , wherein to store the data storage value in the sub-column comprises to add to the memory segment, a row comprising the data storage value.
5. The system of claim 1 , wherein the processor is further configured to update a segment layout map including by updating one or more of column identifiers, offset values, and mask values based at least in part on the increased data size associated with the corresponding variable column width of the sub-column.
6. The system of claim 1 , wherein the processor is further configured to:
determine that the memory segment is full; and
divide the memory segment into a plurality of memory segments.
7. The system of claim 6 , wherein to store the data storage value includes to store the data storage value in one of the plurality of memory segments.
8. The system of claim 6 , wherein to store the data storage value includes to store the data storage value prior to the dividing of the memory segment.
9. The system of claim 1 , wherein the processor is further configured to divide the memory segment into a plurality of memory segments.
10. The system of claim 9 , wherein to divide the memory segment includes to:
determine a first set of rows and a second set of rows included in the memory segment; and
divide the memory segment into a plurality of memory segments comprising a first memory segment including the first set of rows and a second memory segment including the second set of rows.
11. The system of claim 9 , wherein to divide the memory segment includes to combine a first memory segment divided from the plurality of memory segments with a second memory segment divided from the plurality of memory segments.
12. The system of claim 1 , wherein the processor is further configured to:
divide the memory segment into a plurality of memory segments; and
combine a first memory segment of the plurality of memory segments with a second memory segment divided from a separate memory segment.
13. The system of claim 1 , wherein to store the data storage value comprises to:
determine that storing the data storage value in the memory segment would result in the memory segment filling up;
divide the memory segment into a plurality of memory segments based at least in part on the determination; and
store the data storage value in one of the plurality of memory segments.
14. A method, comprising:
selecting a column included in a memory segment to store a data storage value, wherein:
the column comprises a sub-column that stores data of a type associated with the data storage value; and
a corresponding variable column width of the sub-column is not more than a maximum allowable column width associated with the column;
determining that a data size associated with the corresponding variable column width of the sub-column is less than a size associated with the data storage value;
determining that the data size associated with the corresponding variable column width of the sub-column can be increased to accommodate the size associated with the data storage value without causing a column width of the column to exceed the maximum allowable column width associated with the column;
increasing the data size associated with the corresponding variable column width of the sub-column to accommodate the size associated with the data storage value; and
storing the data storage value in the sub-column.
15. The method of claim 14 , further comprising:
determining that the memory segment is full; and
dividing the memory segment into a plurality of memory segments.
16. The method of claim 15 , wherein storing the data storage value includes storing the data storage value in one of the plurality of memory segments.
17. The method of claim 15 , wherein storing the data storage value includes storing the data storage value prior to the dividing of the memory segment.
18. The method of claim 14 , further comprising:
dividing the memory segment into a plurality of memory segments; and
combining a first memory segment of the plurality of memory segments with a second memory segment divided from a separate memory segment.
19. The method of claim 14 , wherein storing the data storage value comprises:
determining that storing the data storage value in the memory segment would result in the memory segment filling up;
dividing the memory segment into a plurality of memory segments based at least in part on the determination; and
storing the data storage value in one of the plurality of memory segments.
20. A computer program product, the computer program product being embodied in a tangible non-transitory computer readable storage medium and comprising computer instructions for:
selecting a column included in a memory segment to store a data storage value, wherein:
the column comprises a sub-column that stores data of a type associated with the data storage value; and
a corresponding variable column width of the sub-column is not more than a maximum allowable column width associated with the column;
determining that a data size associated with the corresponding variable column width of the sub-column is less than a size associated with the data storage value;
determining that the data size associated with the corresponding variable column width of the sub-column can be increased to accommodate the size associated with the data storage value without causing a column width of the column to exceed the maximum allowable column width associated with the column;
increasing the data size associated with the corresponding variable column width of the sub-column to accommodate the size associated with the data storage value; and
storing the data storage value in the sub-column.