Methods and systems for bidirectional indexing using indexlets
In an aspect, provided is a method comprising receiving a data model, partitioning a first table in the data model into a first plurality of blocks of rows, generating a first plurality of indexlets, the first plurality of indexlets comprising a first plurality bidirectional indexes, each of the first plurality of bidirectional indexes being generated based on a corresponding one of the first plurality of blocks of rows.
1. A method comprising:
receiving a query associated with a data model comprising at least one table, wherein the query is based on at least one user interface selection, and wherein the at least one user interface selection is associated with at least one attribute, of a plurality of attributes, stored in the at least one table;
partitioning the at least one table into a first plurality of blocks of rows;
generating a first plurality of indexlets for the at least one table, wherein each indexlet of the first plurality of indexlets corresponds to one of the first plurality of blocks of rows,
determining, based on a query associated with at least one attribute stored in the at least one table, a portion of the data model associated with the query;
generating, based on the first plurality of indexlets, a global attribute list comprising a plurality of entries and associated with the plurality of attributes, wherein each entry of the plurality of entries associated with the at least one attribute comprises a hash reference to one of the first plurality of indexlets and wherein each indexlet of the first plurality of indexlets indicates, for each row of the first plurality of blocks of rows associated with the corresponding indexlet, that a value associated with the at least one attribute is stored, or not stored, in the corresponding row of the at least one table; and
causing, based on the hash references within the global attribute list and the first plurality of indexlets, the portion of the data model to be loaded into memory of a computing device, wherein the computing device is associated with the at least one user interface selection.
2. The method of claim 1 , wherein the data model is sequentially loaded into the memory of the computing device.
3. The method of claim 1 , wherein the data model comprises a size that exceeds a capacity of the memory of the computing device.
4. The method of claim 1 , further comprising processing the query.
5. The method of claim 1 , wherein each indexlet, of the first plurality of indexlets, comprises a column to indicate which of the plurality of attributes are associated with the at least one user interface selection and which of the plurality of attributes are not associated with the at least one user interface selection.
6. The method of claim 1 , further comprising:
partitioning a second tableof the data model into a second plurality of blocks of rows; and
generating a second plurality of indexlets based on the second plurality of blocks of rows.
7. The method of claim 1 , further comprising generating an attribute-to-attribute index.
8. The method of claim 7 , wherein the attribute-to-attribute index comprises a plurality of associations between a first plurality of attributes in the at least one table and a second plurality of attributes in a second table of the data model.
9. The method of claim 1 , wherein partitioning the at least one table in the data model into the first plurality of blocks of rows comprises: partitioning, based on a predefined block size, the at least one table in the data model into the first plurality of blocks of rows.
10. The method of claim 1 , wherein the first plurality of indexlets facilitate parallelized operations applied to the data model.
11. An apparatus comprising:
one or more processors; and
memory storing processor-executable instructions, that when executed by the one or more processors, cause the apparatus to:
receive a query associated with a data model comprising at least one table, wherein the query is based on at least one user interface selection, and wherein the at least one user interface selection is associated with at least one attribute, of a plurality of attributes, stored in the at least one table;
partition the at least one table into a first plurality of blocks of rows;
generate a first plurality of indexlets for the at least one table, wherein each indexlet of the first plurality of indexlets corresponds to one of the first plurality of blocks of rows;
determining, based on a query associated with at least one attribute stored in the at least one table, a portion of the data model associated with the query;
generate, based on the first plurality of indexlets, a global attribute list comprising a plurality of entries and associated with the plurality of attributes, wherein each entry of the plurality of entries associated with the at least one attribute comprises a hash reference to one of the first plurality of indexlets and wherein each indexlet of the first plurality of indexlets indicates, for each row of the first plurality of blocks of rows associated with the corresponding indexlet, that a value associated with the at least one attribute is stored, or not stored, in the corresponding row of the at least one table; and
cause, based on the hash references within the global attribute list and the first plurality of indexlets, the portion of the data model to be loaded into memory of a computing device, wherein the computing device is associated with the at least one user interface selection.
12. The apparatus of claim 11 , wherein the data model is sequentially loaded into the memory of the computing device.
13. The apparatus of claim 11 , wherein the data model comprises a size that exceeds a capacity of the memory of the computing device.
14. The apparatus of claim 11 , wherein the processor-executable instructions further cause the apparatus to process the query.
15. The apparatus of claim 11 , wherein each indexlet, of the first plurality of indexlets, comprises a column to indicate which of the plurality of attributes are associated with the at least one user interface selection and which of the plurality of attributes are not associated with the at least one user interface selection.
16. The apparatus of claim 11 , wherein the processor-executable instructions further cause the apparatus to:
partition a second table of the data model into a second plurality of blocks of rows; and
generate a second plurality of indexlets based on the second plurality of blocks of rows.
17. The apparatus of claim 11 , wherein the processor-executable instructions further cause the apparatus to generate an attribute-to-attribute index.
18. The apparatus of claim 17 , wherein the attribute-to-attribute index comprises a plurality of associations between a first plurality of attributes in the at least one table and a second plurality of attributes in a second table of the data model.
19. The apparatus of claim 11 , wherein the processor-executable instructions that cause the apparatus to partition the at least one table in the data model into the first plurality of blocks of rows further cause the apparatus to partition, based on a predefined block size, the at least one table in the data model into the first plurality of blocks of rows.
20. The apparatus of claim 11 , wherein the first plurality of indexlets facilitate parallelized operations applied to the data model.