Method and approach for pagination over data stream with sliding window
An example method includes receiving, at a server, a request for a list of a first group of index records that correspond to stored data, creating a data streaming session with a client, reading an index file and obtaining a list of the first group of index records from the index file, populating a content cache with a signature that corresponds to the first group of index records, creating a sliding window and populating it with the signature and a pointer to a next group of index records, populating an attribute cache with data streaming session information and the sliding window, creating a continuation token which, when received by the server from the client, indicates to the server that the list of the first group of index records has been received by the client.
1. A method, comprising:
receiving at a server, from a client, a request for a list of index records that correspond to stored data, wherein the request includes a header;
when the header is determined to include a valid first continuation token, parsing the first continuation token to obtain information concerning a data streaming session involving the server and the client, and to obtain a pointer to the group of index records;
when the data streaming session is active, looking up an attribute cache for the data streaming session to obtain sliding window information relating to the data streaming session;
when the sliding window information includes a second continuation token, obtaining a signature that corresponds to the index records;
when a content cache includes the signature that corresponds to the index records, obtaining a list of the index records from the content cache; and
sending a response to the client indicating that the index records in the list of index records have been fetched.
2. The method as recited in claim 1 , wherein when it is determined that the data streaming session is inactive, the method further comprises issuing a timeout error by the server.
3. The method as recited in claim 1 , wherein when the first continuation token is determined not to be valid, the method further comprises issuing an error message by the server.
4. The method as recited in claim 1 , wherein when the sliding window information does not include a second continuation token, the method further comprises:
creating a second continuation token that comprises a signature that corresponds to the index records, and also comprises a pointer to a next set of index records; and
transmitting a response to the client, wherein the response includes the second continuation token but does not include a list of the next set of index records.
5. The method as recited in claim 1 , wherein when the content cache does not include the signature that corresponds to the index records, the method further comprises fetching the index records.
6. The method as recited in claim 1 , wherein the first continuation token indicates to the server that a prior list of index records has been received by the client.
7. The method as recited in claim 1 , wherein the first continuation token comprises a tuple that includes a signature that corresponds to data in the content cache, and that also includes the pointer to the group of index records.
8. The method as recited in claim 7 , wherein the signature uniquely identifies the corresponding data in the content cache.
9. The method as recited in claim 7 , wherein content of the sliding window changes as each successive list of records is transmitted to the client.
10. A system, comprising:
one or more hardware processors; and
a non-transitory storage medium storing instructions that are executable by the one or more hardware processors to perform operations comprising:
receiving at a server, from a client, a request for a list of index records that correspond to stored data, wherein the request includes a header;
when the header is determined to include a valid first continuation token, parsing the first continuation token to obtain information concerning a data streaming session involving the server and the client, and to obtain a pointer to the group of index records;
when the data streaming session is active, looking up an attribute cache for the data streaming session to obtain sliding window information relating to the data streaming session;
when the sliding window information includes a second continuation token, obtaining a signature that corresponds to the index records;
when a content cache includes the signature that corresponds to the index records, obtaining a list of the index records from the content cache; and
sending a response to the client indicating that the index records in the list of index records have been fetched.
11. The system as recited in claim 1 , wherein when it is determined that the data streaming session is inactive, the operations further comprise issuing a timeout error by the server.
12. The system as recited in claim 1 , wherein when the first continuation token is determined not to be valid, the operations further comprise issuing an error message by the server.
13. The system as recited in claim 1 , wherein when the sliding window information does not include a second continuation token, the operations further comprise:
creating a second continuation token that comprises a signature that corresponds to the index records, and also comprises a pointer to a next set of index records; and
transmitting a response to the client, wherein the response includes the second continuation token but does not include a list of the next set of index records.
14. The system as recited in claim 1 , wherein when the content cache does not include the signature that corresponds to the index records, the operations further comprise fetching the index records.
15. The system as recited in claim 1 , wherein the first continuation token indicates to the server that a prior list of index records has been received by the client.
16. The system as recited in claim 1 , wherein the first continuation token comprises a tuple that includes a signature that corresponds to data in the content cache, and that also includes the pointer to the group of index records.
17. The system as recited in claim 16 , wherein the signature uniquely identifies the corresponding data in the content cache.
18. The system as recited in claim 16 , wherein content of the sliding window changes as each successive list of records is transmitted to the client.