Fast object listing in geographically distributed data storage environment
The described technology is generally directed towards quasi-listing table entries that facilitate the efficient return of an object list in response to a query for relevant objects in a data storage system. A quasi-listing table entry is created when an object is created, and subsequently converted to a normal listing table entry. A normal listing table entry is converted to a quasi-listing table entry when an object is deleted, with the quasi-listing table entry subsequently deleted. When an object list request is received, a normal listing table entry can be used directly to add to the object list; only if a quasi-listing table entry is encountered (before it can be converted or deleted) does the object table need to be accessed to determine if the object actually exists. This eliminates most object table accesses, including those in remote geographic zones, thereby making object listing significantly more efficient.
1. A method, comprising:
receiving, by a system comprising a processor, a request for an object result dataset; and
in response to the request for the object result dataset, generating the object result dataset, the generating comprising,
accessing a listing data structure to obtain a listing data structure entry that references an object,
evaluating the listing data structure entry according to a type of the listing data structure entry, and
for a normal-type listing data structure entry that references the object, adding object information corresponding to the object to the object result dataset, and
for a quasi-type listing data structure entry that references the object, accessing an object data structure to determine whether an object data structure entry corresponding to the quasi-type listing data structure entry exists, and, in response to determining that the object data structure entry exists, adding the object information corresponding to the object to the object result dataset.
2. The method of claim 1 , wherein the quasi-type listing data structure entry is a first quasi-type listing data structure entry, and wherein the object is a first object, and further comprising, receiving an object creation request to create a second object, and, in response to the receiving the object creation request, maintaining the listing data structure, comprising adding a second quasi-type listing data structure entry that references the second object to the listing data structure, writing object data associated with the second object to an object data store, and maintaining the object data structure, comprising adding an object data structure entry that references the second object to the object data structure.
3. The method of claim 2 , wherein the normal-type listing data structure entry is a first normal-type listing data structure entry, and the method further comprising converting the second quasi-type listing data structure entry that references the second object to a second normal-type listing data structure entry that references the second object.
4. The method of claim 3 , further comprising asynchronously converting the second quasi-type listing data structure entry that references the second object to the second normal-type listing data structure entry that references the second object.
5. The method of claim 1 , wherein the normal-type listing data structure entry is a first normal-type listing data structure entry, wherein the quasi-type listing data structure entry is a first quasi-type listing data structure entry, and wherein the object is a first object, and further comprising, receiving an object deletion request to delete a second object, and, in response to the receiving the object deletion request, maintaining the listing data structure, comprising converting a second normal-type listing data structure entry that references the second object to a second quasi-type listing data structure entry, and maintaining the object data structure, comprising deleting an object data structure entry that references the second object from the object data structure.
6. The method of claim 5 , further comprising asynchronously deleting the second quasi-type listing data structure entry that references the second object from the listing data structure.
7. The method of claim 1 , wherein the generating the object result dataset comprises processing at least one other listing data structure entry in the listing data structure until a stopping criterion is met.
8. The method of claim 7 , wherein the processing the at least one other listing data structure entry in the listing data structure until the stopping criterion is met comprises stopping upon detecting a listing data structure entry that references another object that is not a subject of the request.
9. The method of claim 7 , wherein the processing the at least one other listing data structure entry in the listing data structure until the stopping criterion is met comprises stopping upon detecting an end of a time range associated with the request.
10. A non-transitory machine-readable medium, comprising executable instructions that, when executed by a processor, facilitate performance of operations, the operations comprising:
receiving a request for an object list; and
in response to the request for the object list, generating the object list, the generating comprising,
accessing a listing table to obtain a listing table entry that references an object to be potentially added to the object list;
evaluating the listing table entry, and
for a normal listing table entry that references the object, adding object information corresponding to the object to the object list, and
for a quasi-listing table entry that references the object, accessing an object table to determine whether an object table entry corresponding to the quasi-listing table entry exists, and, in response to determining that the object table entry exists, adding the object information corresponding to the object to the object list.
11. The non-transitory machine-readable medium of claim 10 , wherein the object is a first object, and wherein the operations further comprise:
receiving an object creation request to create a second object, and
in response to the receiving the object creation request, maintaining the listing table, the maintaining comprising:
adding another quasi-listing table entry that references the second object to the listing table,
writing object data associated with the second object to an object data store, and maintaining the object table, comprising adding another object table entry that comprises metadata of the second object to the object table.
12. The non-transitory machine-readable storage medium of claim 11 , wherein the normal listing table entry is a first normal listing table entry, and wherein the operations further comprise converting the quasi-listing table entry that references the second object to a second normal listing table entry that references the second object.
13. The non-transitory machine-readable storage medium of claim 11 , wherein the normal listing table entry is a first normal listing table entry, wherein the quasi-listing table entry is a first quasi-listing table entry, and wherein the operations further comprise receiving an object deletion request to delete the second object, and, in response to the receiving the object deletion request, maintaining the listing table, comprising converting a second normal listing table entry that references the second object to a second quasi-listing table entry, and maintaining the object table, comprising deleting the other object table entry that references the second object from the object table.
14. The non-transitory machine-readable medium of claim 13 , wherein the operations further comprise asynchronously deleting the second quasi-listing table entry that references the second object from the listing table.
15. The non-transitory machine-readable medium of claim 10 , wherein the normal listing table entry is a first normal listing table entry, and wherein the operations further comprise accessing the listing table to obtain the quasi-listing table entry, determining whether the quasi-listing table entry has a corresponding object listing table entry, and, in response to the determining that the quasi-listing table entry does not have the corresponding object listing table entry, removing the quasi-listing table entry from the listing table, and, in response to the determining that the quasi-listing table entry does have the corresponding object listing table entry, converting the quasi-listing table entry to a normal-listing table entry in the listing table.
16. A system, comprising:
a processor; and
a memory that stores executable instructions that, when executed by the processor, facilitate performance of operations, the operations comprising:
receiving a request for an object result dataset; and
in response to the request for the object result dataset, generating the object result dataset, the generating comprising,
accessing a listing data structure to obtain a listing data structure entry that references an object,
evaluating the listing data structure entry according to a type of the listing data structure entry, and
for a normal-type listing data structure entry that references the object, adding object information corresponding to the object to the object result dataset, and
for a quasi-type listing data structure entry that references the object, accessing an object data structure to determine whether an object data structure entry corresponding to the quasi-type listing data structure entry exists, and, in response to determining that the object data structure entry exists, adding the object information corresponding to the object to the object result dataset.
17. The system of claim 16 , wherein the quasi-type listing data structure entry is a first quasi-type listing data structure entry, and wherein the object is a first object, and further comprising, receiving an object creation request to create a second object, and, in response to the receiving the object creation request, maintaining the listing data structure, comprising adding a second quasi-type listing data structure entry that references the second object to the listing data structure, writing object data associated with the second object to an object data store, and maintaining the object data structure, comprising adding an object data structure entry that references the second object to the object data structure.
18. The system of claim 17 , wherein the normal-type listing data structure entry is a first normal-type listing data structure entry, and the method further comprising converting the second quasi-type listing data structure entry that references the second object to a second normal-type listing data structure entry that references the second object.
19. The system of claim 18 , further comprising asynchronously converting the second quasi-type listing data structure entry that references the second object to the second normal-type listing data structure entry that references the second object.
20. The system of claim 16 , wherein the normal-type listing data structure entry is a first normal-type listing data structure entry, wherein the quasi-type listing data structure entry is a first quasi-type listing data structure entry, and wherein the object is a first object, and further comprising, receiving an object deletion request to delete a second object, and, in response to the receiving the object deletion request, maintaining the listing data structure, comprising converting a second normal-type listing data structure entry that references the second object to a second quasi-type listing data structure entry, and maintaining the object data structure, comprising deleting an object data structure entry that references the second object from the object data structure.