Benchmarking JSON document stores
A method may include generating, based at least on a schema configuration of one or more document stores, a plurality of data in a JavaScript Object Notation (JSON) format for storage at the document stores. One or more queries may be generated to match the plurality of data stored at the document stores. The one or more queries may be distributed for execution at the document stores by a scalable quantity of concurrently operating worker nodes. One or more performance metrics for the execution of the one or more queries at the one or more document stores may be generated. Performance improvements at the document stores may be applied based on the performance metrics. Related systems and computer program products are also provided.
1. A system, comprising:
at least one data processor; and
at least one memory storing instructions which, when executed by the at least one data processor, cause operations comprising:
generating, based at least on a schema configuration of one or more document stores, a plurality of data in a JavaScript Object Notation (JSON) format for storage at the one or more document stores;
generating, based at least on the schema configuration, a query configuration, and a feature matrix of the one or more document stores, one or more queries to match the plurality of data stored at the one or more document stores, wherein the query configuration includes a first object describing an element used in a project clause of a query, and wherein a function selected randomly from the feature matrix is applied to the projected element;
distributing the one or more queries for execution at the one or more document stores by a scalable quantity of concurrently operating worker nodes;
generating one or more performance metrics during the execution of the one or more queries at the one or more document stores; and
applying, based at least on the one or more performance metrics, one or more performance improvements during subsequent query processing at the one or more document stores, wherein the one or more performance improvements include avoiding unnecessary unnesting of JSON objects.
2. The system of claim 1 , wherein the one or more queries are distributed to a first client associated with a first document store for distribution to the scalable quantity of concurrently operating worker nodes associated with a second client.
3. The system of claim 1 , wherein the one or more queries are further distributed to a second client associated with a second document store for distribution to the scalable quantity of concurrently operating worker nodes associated with the second client.
4. The system of claim 1 , wherein the schema configuration includes a first object describing a fixed portion of a document comprising the plurality of data by at least specifying one or more elements required to appear in the document.
5. The system of claim 4 , wherein the schema configuration further includes a second object describing one or more variations in the document, and wherein the one or more variations include variations in a quantity of nested objects, a quantity of keys associated with each nested object, and/or a size of a value associated with each key.
6. The system of claim 1 , wherein the operations further comprise generating random sub-documents in order to increase a size and complexity of a single document.
7. The system of claim 6 , wherein the random sub-documents include variations in a quantity of nested objects, a quantity of keys inside each object, and a size of a value associated with each key.
8. The system of claim 1 , wherein the query configuration further includes a second object describing a where clause of the query that includes a conjunctive combination or a disjunctive combination of multiple predicates with one or more user specified or randomly selected filters.
9. The system of claim 8 , wherein the query configuration further includes a third object describing the element and a nesting depth of the element.
10. The system of claim 1 , wherein the one or more queries include at least one structured query language (SQL) query.
11. The system of claim 1 , wherein the operations further comprise:
generating a single index or a compound index for the execution of the one or more queries at the one or more document stores.
12. The system of claim 1 , wherein the plurality of data includes custom data and/or existing data associated with the one or more document stores.
13. The system of claim 1 , wherein the one or more queries include custom queries and/or existing queries associated with the one or more document stores.
14. The system of claim 1 , wherein the one or more document stores include a JSON document store and/or a relational database with a JSON extension.
15. The system of claim 1 , wherein the one or more performance metrics include a first performance metric of the one or more queries being executed at the one or more document stores by a first quantity of concurrently operating worker nodes and a second performance metric of the one or more queries being executed at the one or more document stores by a second quantity of concurrently operating worker nodes.
16. The system of claim 1 , wherein the plurality of data include documents having various nesting levels, and wherein the one or more performance metrics include a first performance metric of the one or more queries being executed on documents having a first quantity of nesting levels and a second performance metric of the one or more queries being executed on documents having a second quantity of nesting levels.
17. The system of claim 1 , wherein the one or more performance metrics include a first performance metric of the one or more queries being executed with an index, and wherein the one or more metrics further include a second performance metric of the one or more queries being executed without the index.
18. The system of claim 1 , wherein the one or more performance improvements comprise performing a pre-filter operation on a plurality of elements in a JSON array prior to performing an unnest operation on the plurality of elements in the JSON array.
19. The system of claim 1 , wherein the plurality of data and the one or more queries are further generated in accordance with one or more configurable scaling factors associated with data complexity, query complexity, data types, result sizes, and quantity of concurrent users.
20. A computer-implemented method, comprising:
generating, based at least on a schema configuration of one or more document stores, a plurality of data in a JavaScript Object Notation (JSON) format for storage at the one or more document stores;
generating, based at least on the schema configuration, a query configuration, and a feature matrix of the one or more document stores, one or more queries to match the plurality of data stored at the one or more document stores, wherein the query configuration includes a first object describing an element used in a project clause of a query, and wherein a function selected randomly from the feature matrix is applied to the projected element;
distributing the one or more queries for execution at the one or more document stores by a scalable quantity of concurrently operating worker nodes;
generating one or more performance metrics during the execution of the one or more queries at the one or more document stores; and
applying, based at least on the one or more performance metrics, one or more performance improvements during subsequent query processing at the one or more document stores, wherein the one or more performance improvements include avoiding unnecessary unnesting of JSON objects.