Using self-maintaining structure information for faster data access
A method, a system, and a computer program product for accessing data. A schema representing a structure of an object in a plurality of objects stored in a storage location is generated. Each object includes one or more data elements. Each schema identifies one or more data elements of the object, an offset location of each data element of the object, and a value of each data element of the object. A query requesting access to one or more data elements is received. A generated schema in a plurality of generated schemas representing the queried object is identified. The elements are accessed using the identified generated schema, and retrieved.
1 . A computer-implemented method, comprising:
generating a schema representing a structure of an object in a plurality of objects stored in a storage location, each object in the plurality of objects including one or more data elements, the schema identifying, for the object in the plurality of objects, one or more data elements of the object, an offset location of each of the one or more data elements of the object, and a value of each of the one or more data elements of the object;
receiving a query requesting access to at least one of the one or more data elements of the object in the plurality of objects;
identifying a generated schema in a plurality of generated schemas representing the object;
mapping the identified generated schema to a binary representation of the structure of the object, such that each row in the schema is mapped to an offset field in the binary representation, and each offset field is mapped to a value field in the binary representation;
accessing, using the binary representation of the structure of the object mapped to the identified generated schema, the one or more data elements of the object in the plurality of objects; and
retrieving the one or more data elements of the object in the plurality of objects, wherein the generated schema identifies, using a skip list offset location in the object, one or more skip lists permitting skipping at least a portion of the one or more data elements of the object during the accessing and the retrieving, wherein the one or more skip lists are incorporated into a beginning of the binary representation of the structure of the object.
2 . The method according to claim 1 , wherein the generated schema is a fixed schema, the fixed schema preventing modification of the schema.
3 . The method according to claim 1 , wherein the generated schema is a mixed schema, the mixed schema allowing modification of the schema.
4 . The method according to claim 3 , further comprising adding identification of one or more another data elements of the object, an offset location of each another data element of the object, and a value of each another data element of the object.
5 . The method according to claim 4 , wherein the accessing includes accessing the one or more another data elements of the object, and the retrieving includes retrieving the one or more another data elements of the object.
6 . The method according to claim 1 , wherein the generated schema identifies one or more offset location levels of each data element in the one or more data elements of the object.
7 . The method according to claim 1 , wherein the receiving further comprises
generating a query execution plan for the received query;
determining, at a query compilation time of the received query, an offset location of a first data element in the one or more data elements of the object;
storing, in the generated query execution plan of the received query, the determined offset location of the first data element; and
storing the generated query execution plan;
the method further comprising
receiving another query to access the first data element; and
accessing, using the stored query execution plan, the first data element.
8 . The method according to claim 1 , wherein the object in the plurality of objects does not include an object schema.
9 . A system comprising:
at least one programmable processor; and
a non-transitory machine-readable medium storing instructions that, when executed by the at least one programmable processor, cause the at least one programmable processor to perform operations comprising:
generating a schema representing a structure of an object in a plurality of objects stored in a storage location, each object in the plurality of objects including one or more data elements, the schema identifying, for the object in the plurality of objects, one or more data elements of the object, an offset location of each of the one or more data elements of the object, and a value of each of the one or more data elements of the object;
receiving a query requesting access to at least one of the one or more data elements of the object in the plurality of objects;
identifying a generated schema in a plurality of generated schemas representing the object;
mapping the identified generated schema to a binary representation of the structure of the object, such that each row in the schema is mapped to an offset field in the binary representation, and each offset field is mapped to a value field in the binary representation;
accessing, using the binary representation of the structure of the object mapped to the identified generated schema, the one or more data elements of the object in the plurality of objects; and
retrieving the one or more data elements of the object in the plurality of objects, wherein the generated schema identifies, using a skip list offset location in the object, one or more skip lists permitting skipping at least a portion of the one or more data elements of the object during the accessing and the retrieving, wherein the one or more skip lists are incorporated into a beginning of the binary representation of the structure of the object.
10 . The system according to claim 9 , wherein the generated schema is a fixed schema, the fixed schema preventing modification of the schema.
11 . The system according to claim 9 , wherein the generated schema is a mixed schema, the mixed schema allowing modification of the schema.
12 . The system according to claim 11 , wherein the operations further comprise adding identification of one or more another data elements of the object, an offset location of each another data element of the object, and a value of each another data element of the object.
13 . The system according to claim 12 , wherein the accessing includes accessing the one or more another data elements of the object, and the retrieving includes retrieving the one or more another data elements of the object.
14 . The system according to claim 9 , wherein the generated schema identifies one or more offset location levels of each data element in the one or more data elements of the object.
15 . The system according to claim 9 , wherein the receiving further comprises
generating a query execution plan for the received query;
determining, at a query compilation time of the received query, an offset location of a first data element in the one or more data elements of the object;
storing, in the generated query execution plan of the received query, the determined offset location of the first data element; and
storing the generated query execution plan;
the operations further comprising
receiving another query to access the first data element; and
accessing, using the stored query execution plan, the first data element.
16 . The system according to claim 9 , wherein the object in the plurality of objects does not include an object schema.
17 . A computer program product comprising a non-transitory machine-readable medium storing instructions that, when executed by at least one programmable processor, cause the at least one programmable processor to perform operations comprising:
generating a schema representing a structure of an object in a plurality of objects stored in a storage location, each object in the plurality of objects including one or more data elements, the schema identifying, for the object in the plurality of objects, one or more data elements of the object, an offset location of each of the one or more data elements of the object, and a value of each of the one or more data elements of the object;
receiving a query requesting access to at least one of the one or more data elements of the object in the plurality of objects;
identifying a generated schema in a plurality of generated schemas representing the object;
mapping the identified generated schema to a binary representation of the structure of the object, such that each row in the schema is mapped to an offset field in the binary representation, and each offset field is mapped to a value field in the binary representation;
accessing, using the binary represent of the structure of the object mapped to the identified generated schema, the one or more data elements of the object in the plurality of objects; and
retrieving the one or more data elements of the object in the plurality of objects, wherein the generated schema identifies, using a skip list offset location in the object, one or more skip lists permitting skipping at least a portion of the one or more data elements of the object during the accessing and the retrieving, wherein the one or more skip lists are incorporated into a beginning of the binary representation of the structure of the object.