Systems and methods of sharing a database across multiple deployments and services
Techniques for implementing a non-relational database that makes efficient use of collections within the database. For one or more collections, two or more sub-collections can be created for storing documents. Each collection can be configured as a single partition entity or a partitioned entity within the database. Each sub-collection is identified by a sub-collection identifier. If the collection is configured as a partitioned entity, then a partition key can be determined for documents to be accessed in the collection. The partition key can be extended with the sub-collection ID to form a compound property (sub-collection ID, partition key) that determines placements of the respective documents in the identified sub-collection across partitions of the collection. If the collection is configured as a single partition entity, then a field for the partition key is ignored and the respective documents are placed in the identified sub-collection within the single partition of the collection.
1. A method of implementing a non-relational database that makes more efficient use of a limited number of collections within a database storage resource, the non-relational database being hosted by a database server, the method comprising:
creating a collection of one or more documents in the non-relational database, the collection being created on a single partition of the database server;
creating a plurality of sub-collections within the collection so as to increase storage capacity of the non-relational database without an increase in a limited number of collections of the database storage resource, the plurality of sub-collections being created on the single partition of the database server;
generating a sub-collection identifier (ID) for a sub-collection from among the plurality of sub-collections, the sub-collection including a respective document from among the one or more documents of the collection, the sub-collection ID including a plurality of predetermined identifiers (IDs) that identify the sub-collection from among the plurality of sub-collections;
modifying a format of the respective document to include a plurality of document fields containing the plurality of predetermined IDs, respectively;
receiving a query to access the respective document of the collection, the query including a plurality of query fields containing the plurality of predetermined IDs, respectively, that identify the sub-collection within the collection;
identifying the sub-collection within the collection using the plurality of predetermined IDs contained in the plurality of query fields, respectively; and
accessing, from the non-relational database, the respective document of the identified sub-collection on the single partition of the database server.
2. The method of claim 1 further comprising:
inserting the plurality of predetermined IDs into the plurality of document fields, respectively, of the format of the respective document.
3. The method of claim 2 further comprising:
obtaining the plurality of predetermined IDs from the plurality of document fields, respectively, of the format of the respective document, thereby obtaining the sub-collection ID for the sub-collection.
4. The method of claim 1 wherein the respective document has an associated document schema, wherein a respective second document among the one or more documents has an associated second document schema, wherein the second document schema is different from the document schema of the respective document, and wherein the method further comprises:
generating a second sub-collection ID for a second sub-collection from among the plurality of sub-collections, the second sub-collection including the respective second document, the second sub-collection ID including a plurality of predetermined second IDs that identify the second sub-collection from among the plurality of sub-collections;
modifying a format of the respective second document to include a plurality of second document fields containing the plurality of predetermined second IDs relating to the second sub-collection, respectively;
receiving a second query to access the respective second document of the collection, the second query including a plurality of second query fields containing the plurality of predetermined second IDs, respectively, that identify the second sub-collection within the collection; and
accessing, from the non-relational database, the respective second document of the second sub-collection on the single partition of the database server based on the plurality of predetermined second IDs contained in the plurality of second query fields, respectively.
5. The method of claim 1 wherein the plurality of predetermined IDs that identify the sub-collection include a first predetermined ID corresponding to a specified deployment of a software application, a second predetermined ID corresponding to a specified microservice of the software application, and a third predetermined ID corresponding to a specified type of database object associated with the specified microservice.
6. A method of implementing a non-relational database that makes more efficient use of a limited number of collections within a database storage resource, the non-relational database being hosted by a database server, the method comprising:
creating a collection of one or more documents in the non-relational database, the collection being created across a plurality of partitions of the database server;
creating a plurality of sub-collections within the collection so as to increase storage capacity of the non-relational database without an increase in a limited number of collections of the database storage resource, a portion of the plurality of sub-collections being created on a single partition from among the plurality of partitions of the database server;
generating a sub-collection identifier (ID) for a sub-collection from among the portion of the plurality of sub-collections, the sub-collection including a respective document from among the one or more documents of the collection, the sub-collection ID including a plurality of predetermined identifiers (IDs) that identify the sub-collection from among the plurality of sub-collections;
determining a partition key for the respective document, a value of the partition key being hosted by a respective one of the plurality of partitions of the database server;
extending the partition key to include the plurality of predetermined IDs relating to the sub-collection and the partition key for the respective document;
modifying a format of the respective document to include a plurality of document fields containing the plurality of predetermined IDs, respectively, and an additional document field containing the partition key;
receiving a query to access the respective document of the collection, the query including a plurality of query fields containing the plurality of predetermined IDs, respectively, that identify the sub-collection within the collection, and an additional query field containing the partition key;
identifying the sub-collection within the collection using the plurality of predetermined IDs contained in the plurality of query fields, respectively, and the partition key contained in the additional query field; and
accessing, from the non-relational database, the respective document of the identified sub-collection on the respective one of the plurality of partitions of the database server.
7. The method of claim 6 further comprising:
inserting the partition key into the additional document field.
8. The method of claim 7 further comprising:
inserting the plurality of predetermined IDs into the plurality of document fields, respectively, of the format of the respective document.
9. The method of claim 8 further comprising:
obtaining the partition key for the respective document from the additional document field.
10. The method of claim 9 further comprising:
obtaining the value of the partition key.
11. The method of claim 10 further comprising:
obtaining the plurality of predetermined IDs from the plurality of document fields, respectively, thereby obtaining the sub-collection ID for the sub-collection.
12. The method of claim 10 further comprising:
accessing the respective document in the identified sub-collection of the collection within the respective one of the plurality of partitions hosting the value of the partition key.
13. A system for implementing a non-relational database that makes more efficient use of a limited number of collections within a database storage resource, comprising:
a database server including storage processing circuitry and a data storage resource, the data storage resource containing the non-relational database,
wherein the storage processing circuitry is operative:
to create a collection of one or more documents in the non-relational database, the collection being created on a single partition of the database server;
to create a plurality of sub-collections within the collection so as to increase storage capacity of the non-relational database without an increase in a limited number of collections of the database storage resource, the plurality of sub-collections being created on the single partition of the database server;
to generate a sub-collection identifier (ID) for a sub-collection from among the plurality of sub-collections, the sub-collection including a respective document from among the one or more documents of the collection, the sub-collection ID including a plurality of predetermined identifiers (IDs) that identify the sub-collection from among the plurality of sub-collections; and
to modify a format of the respective document to include a plurality of document fields containing the plurality of predetermined IDs, respectively;
to receive a query to access the respective document of the collection, the query including a plurality of query fields containing the plurality of predetermined IDs, respectively, that identify the sub-collection within the collection;
to identify the sub-collection within the collection using the plurality of predetermined IDs contained in the plurality of query fields, respectively; and
to access, from the non-relational database, the respective document of the identified sub-collection on the single partition of the database server.
14. The system of claim 13 wherein the at least one memory of the development client contains a wrapper library having an application programming interface (API) and a plurality of predetermined functions, and wherein the at least one processor of the development client is further operative to execute the at least one computer program out of the at least one memory:
to generate the query for the respective document as an API call to a respective function from among the plurality of predetermined functions of the wrapper library.
15. A computer program product having non-transitory computer readable media that stores at least one set of instructions to implement a non-relational database that makes more efficient use of a limited number of collections within a database storage resource, the at least one set of instructions, when carried out by computerized circuitry, causing the computerized circuitry to perform a method of:
creating a collection of one or more documents in the non-relational database, the collection being created on a single partition of the database server;
creating a plurality of sub-collections within the collection so as to increase storage capacity of the non-relational database without an increase in a limited number of collections of the database storage resource, the plurality of sub-collections being created on the single partition of the database server;
generating a sub-collection identifier (ID) for a sub-collection from among the plurality of sub-collections, the sub-collection including a respective document from among the one or more documents of the collection, the sub-collection ID including a plurality of predetermined identifiers (IDs) that identify the sub-collection from among the plurality of sub-collections;
modifying a format of the respective document to include a plurality of document fields containing the plurality of predetermined IDs, respectively;
receiving a query to access the respective document of the collection, the query including a plurality of query fields containing the plurality of predetermined IDs, respectively, that identify the sub-collection within the collection;
identifying the sub-collection within the collection using the plurality of predetermined IDs contained in the plurality of query fields, respectively; and
accessing, from the non-relational database, the respective document of the identified sub-collection on the single partition of the database server.