Method and systems for flexible and scalable databases
View Patent ↗Methods and systems for utilizing a database are disclosed. The methods and systems determine a key representative of a storage location of first RDF data in a NoSQL database. In addition, the methods and systems read the first RDF data in the NoSQL database using the key. The methods and systems also write second RDF data derived from the first RDF data into a second database stored in memory. The methods and systems may also modify the second RDF data, and write third RDF data derived from the modified second RDF data into the NoSQL database.
1. A system for scaling indexed data with scalable databases, the system comprising:
a memory storing a set of instructions; and
one or more processers configured to execute the set of instructions to perform one or more operations, the one or more operations comprising:
storing indexed data in a scalable database, the scalable database comprising a Not Only Structured Query Language (NoSQL) database,
determining a received new job initiation requires the indexed data stored in the scalable database,
determining, when the new job initiation requires data stored in the scalable database, a key representing a storage location of first indexed data in the scalable database, wherein the first indexed data is compressed,
reading the first indexed data in the scalable database using the key,
writing second indexed data derived from the first indexed data into an in-memory database, wherein the second indexed data is uncompressed, wherein the second indexed data comprises a set of indices, and wherein the data stored in the set of indices is derived from decompressing the first indexed data,
modifying the second indexed data, and
writing third indexed data derived from the modified second indexed data into the scalable database.
2. The system of claim 1 , wherein the operations further comprise deleting the second indexed data or the modified second indexed data after writing the third indexed data into the scalable database.
3. The system of claim 2 , wherein deleting the second indexed data or the modified second indexed data is performed in response to detecting that the new job initiation is complete.
4. The system of claim 1 , wherein the third indexed data is compressed.
5. The system of claim 1 , wherein the operations further comprise:
locking on the modified second indexed data to prevent access to the modified second indexed data; and
releasing the lock after writing the third indexed data.
6. The system of claim 1 , wherein the operations further comprise determining that the second indexed data references data not yet loaded in the in-memory database.
7. The system of claim 6 , wherein the operations further comprise:
determining a second key representing a storage location of the data not yet loaded in the scalable database;
reading the data not yet loaded in the scalable database using the second key; and
writing fourth indexed data derived from the data not yet loaded in the scalable database into the in-memory database.
8. The system of claim 1 , wherein the first indexed data, the second indexed data, and the third indexed data each comprise Resource Description Framework (RDF) data.
9. A method for scaling indexed data with scalable databases, the method comprising one or more processors to perform the steps of:
storing indexed data in a scalable database, the scalable database comprising a Not Only Structured Query Language (NoSQL) database;
determining a received new job initiation requires the indexed data stored in the scalable database;
determining, when the new job initiation requires data stored in the scalable database, a key representing a storage location of first indexed data in the scalable database, wherein the first indexed data is compressed;
reading the first indexed data in the scalable database using the key;
writing second indexed data derived from the first indexed data into an in-memory database, wherein the second indexed data is uncompressed, wherein the second indexed data comprises a set of indices, and wherein the data stored in the set of indices is derived from decompressing the first indexed data;
modifying the second indexed data; and
writing third indexed data derived from the modified second indexed data into the scalable database.
10. The method of claim 9 , further comprising deleting the second indexed data or the modified second indexed data after writing the third indexed data into the scalable database.
11. The method of claim 10 , wherein deleting the second indexed data or the modified second indexed data is performed in response to detecting that the new job initiation is complete.
12. The method of claim 9 , wherein the third indexed data is compressed.
13. The method of claim 9 , farther comprising:
locking on the modified second indexed data to prevent access to the modified second indexed data; and
releasing the lock after writing the third indexed data.
14. The method of claim 9 , further comprising determining that the second indexed data references data not yet loaded in the in-memory database.
15. The method of claim 14 , further comprising:
determining a second key representing a storage location of the data not yet loaded in the scalable database;
reading the data not yet loaded in the scalable database using the second key; and
writing fourth indexed data derived from the data not yet loaded in the scalable database into the in-memory database.
16. The method of claim 9 , wherein the first indexed data, the second indexed data, and the third indexed data each comprise Resource Description Framework (RDF) data.
17. A non-transitory, computer readable medium storing instructions for execution by one or more processors to perform a method for scaling indexed data with scalable databases, the method comprising:
storing indexed data in a scalable database, the scalable database comprising a Not Only Structured Query Language (NoSQL) database;
determining a received new job initiation requires the indexed data stored in the scalable database;
determining, when the new job initiation requires data stored in the scalable database, a key representing a storage location of first indexed data in the scalable database, wherein the first indexed data is compressed;
reading the first indexed data in the scalable database using the key;
writing, second indexed data derived from the first indexed data into an in-memory database, wherein the second indexed data is uncompressed, wherein the second indexed data comprises a set of indices, and wherein the data stored in the set of indices is derived from decompressing the first indexed data;
modifying the second indexed data; and
writing third indexed data derived from the modified second indexed data into the scalable database.
18. The system of claim 1 , wherein the operations further comprise decompressing the first indexed data to derive the second indexed data.
19. The system of claim 8 , wherein the first indexed data comprises a compressed RDF graph, and wherein the compressed RDF graph is grouped into a set of topics, wherein each topic is associated with a set of sub-topics.
20. The method of claim 9 , further comprising decompressing the first indexed data to derive the second indexed data.