Reading own writes using context objects in a distributed database
A context object is created when a write operation is initiated. The client application or user performs a write operation to a table and receives a context object which has information on all the tablets that are impacted by writes by the client application. The context object may contain a list describing what key ranges the client application has updated. As such, only that small portion of what has been touched needs to be looked for and this typically only includes a small subset of tablets compared to all the tablets associated with that table. This leads to a small verification cost when checking only impacted tablets in the cluster. The only portion of the table in the database that needs to be verified is the one or more portions that were updated and nothing else.
1. A method comprising:
requesting, by a client application, a write operation to one or more tablets associated with a table of a distributed database;
receiving at the client application a context object associated with the write operation, wherein the context object is generated by the one or more tablets;
subsequent to a request for the write operation, requesting, by the client application, a read operation;
determining whether an indexing operation associated with the write operation is completed based on the context object; and
using the context object to reduce a number of tablets of the read operation during verification of indexing operations to an intersection of tablets involved in the read operation and tablets involved in the write operation.
2. The method of claim 1 , wherein the context object comprises a list describing a key range updated by the client application.
3. The method of claim 1 , wherein the context object comprises timestamp data associated with a latest timestamp of an update to any of the one or more tablets.
4. The method of claim 1 , wherein the context object comprises a timestamp for each key range updated by the write operation.
5. The method of claim 1 , further comprising:
receiving a confirmation of the write operation for a respective tablet and a timestamp associated with the write operation for each respective tablet of the one or more tablets.
6. The method of claim 1 , further comprising:
asynchronously writing records of a respective tablet to one or more secondary indexes; and
maintaining a minimum timestamp of a pending secondary index write operation for the respective tablet.
7. The method of claim 6 , further comprising:
determining that the write operation has propagated to the secondary index by testing the minimum timestamp against a timestamp associated with the write operation.
8. The method of claim 1 , further comprising:
determining that an indexing operation associated with the write operation is completed based on the context object; and
completing the read operation.
9. The method of claim 1 , further comprising:
determining that an indexing operation associated with the write operation is not completed based on the context object; and
delaying the read operation.
10. The method of claim 1 , further comprising:
receive, from the one or more tablets, a statistical estimate of an indexing delay in the one or more tablets; and
query the one or more tablets to determine if the write operation has been indexed, wherein the one or more tablets are queried at a time based on the time delay.
11. A system for performing write operations in a distributed database comprising:
a processing unit; and
a memory containing instructions that, when executed by the processing unit, cause the processing unit to:
request, by a client application, a write operation to one or more tablets associated with a table of a distributed database;
receive at the client application a context object associated with the write operation, wherein the context object is generated by the one or more tablets;
subsequent to a request for the write operation, request, by the client application, a read operation;
determine whether an indexing operation associated with the write operation is completed based on the context object;
receive, from the one or more tablets, a statistical estimate of an indexing delay in the one or more tablets; and
query the one or more tablets to determine if the write operation has been indexed, wherein the one or more tablets are queried at a time based on the time delay.
12. The system of claim 11 , wherein the instructions further cause the processing unit to:
pass contents of the context object to a second client application.
13. The system of claim 12 , wherein the second client application uses the context object to perform a test for index currency of the one or more tablets.
14. The system of claim 11 , wherein the instructions further cause the processing unit to:
request a second write operation; and
receive a second context object associated with the second write operation.
15. The system of claim 14 , wherein the instructions further cause the processing unit to:
combine the context object and the second context object to test an indexing status of the write request and the second write request using a single test.
16. The system of claim 11 , wherein each of the one or more tablets is associated with a replica tablet and wherein each replica tablet is associated with a minimum timestamp for a pending replication of the write operation to each replica tablet.
17. The system of claim 16 , wherein the context object comprises the minimum timestamp associated with each replica tablet.
18. The system of claim 17 , wherein the instructions further cause the processing unit to:
use the context object to probe each replica tablet to determine if data written as part of the write operation has been fully propagated to each replica tablet.
19. A non-transitory computer-readable medium storing code comprising instructions executable to:
request, by a client application, a write operation to one or more tablets associated with a table of a distributed database;
receive at the client application a context object associated with the write operation, wherein the context object is generated by the one or more tablets;
subsequent to a request for the write operation, request, by the client application, a read operation;
determine whether an indexing operation associated with the write operation is completed based on the context object; and
using the context object to reduce a number of tablets of the read operation during verification of indexing operations to an intersection of tablets involved in the read operation and tablets involved in the write operation.
20. The non-transitory computer-readable medium of claim 19 , the instructions executable to:
receive, from the one or more tablets, a statistical estimate of an indexing delay in the one or more tablets; and
query the one or more tablets to determine if the write operation has been indexed, wherein the one or more tablets are queried at a time based on the time delay.