Data channel between a client and a restartable service
A data communication channel between a client and a service is preserved through a failure of the server by maintaining a request log and an inflight request queue in a protected memory region that preserves the contents of the request log and the inflight request queue even when the service encounters a failure. The method of restarting the data communication channel includes, upon the service being restarted following the failure of the service, determining whether the request log contains requests and, if so, copying the requests from the request log into the in-flight request queue and then removing the copied requests from the request log. The requests in the in-flight request queue, which include any that were in the in-flight request queue at the time of the failure of the service and any that were copied from the request log, are then processed.
1. A method for handling client requests from a client for a process, comprising:
prior to the process encountering a failure:
maintaining a request log and an inflight request queue in a protected memory region that is a shared memory space supported by a hypervisor running in a computing device, the request log and inflight request queue for storing requests that are preserved even when the process encounters the failure;
copying a plurality of requests from a client request queue into the request log and the inflight request queue;
directly in response to completing copying of a request of the plurality of requests into both the inflight request queue and the request log, removing the request from the request log and the client request queue; and
servicing one or more of the plurality of requests out of the inflight request queue; and
upon the process being restarted following the failure of the process:
determining whether the request log contains one or more requests and, if so, copying the one or more requests from the request log into the in-flight request queue and then removing the copied one or more requests from the request log; and
processing the one or more requests in the in-flight request queue, wherein the one or more requests include any that were in the in-flight request queue when the failure of the process was encountered and any that were copied from the request log.
2. The method according to claim 1 , wherein the plurality of requests are also copied into a hash table associated with the inflight request queue, and each request of the plurality of requests is indexed in the hash table according to a hash thereof.
3. The method according to claim 2 , wherein the request is removed from the client request queue and the request log after the request is copied into the inflight request queue and the hash table.
4. The method according to claim 1 , further comprising, upon completing the processing of a (liven request by the process, sending a completion response to the client.
5. The method according to claim 4 , wherein sending the completion response to the client includes:
copying the completion response to the given request into a completion log, which is also maintained in the protected memory region;
removing the given request corresponding to the completion response from the inflight request queue;
copying the completion response into a completion queue; and
after copying the completion response into the completion queue, removing the completion response from the completion log.
6. The method according to claim 5 , wherein the plurality of requests are also copied into a hash table associated with the inflight request queue when the plurality of requests are copied into the inflight request queue, and
prior to removing the request from the inflight request queue, the hash table associated with the inflight request queue is used to find the request.
7. The method according to claim 1 , wherein the client is a virtual machine supported by the hypervisor.
8. The method according to claim 7 , wherein the process is a user space process supported by the hypervisor.
9. The method according to claim 7 , wherein the process is a module within the hypervisor.
10. The method of claim 1 , wherein the protected memory region is a POSIX shared memory object.
11. A non-transitory computer readable storage medium containing instructions that configure a processor of a computing device to carry out a method for handling client requests from a client for a process, the method comprising:
prior to the process encountering a failure:
maintaining a request log and an inflight request queue in a protected memory region that is a shared memory space supported by a hypervisor running in the computing device, the request log and inflight request queue for storing requests that are preserved even when the process encounters the failure;
copying a plurality of requests from a client request queue into the request log and the inflight request queue;
directly in response to completing copying of a request of the plurality of requests into the inflight request queue and the request log, removing the request from the request log and the client request queue; and
servicing one or more of the plurality of requests out of the inflight request queue; and
upon the process being restarted following the failure of the process:
determining whether the request log contains one or more requests and, if so, copying the one or more requests from the request log into the in-flight request queue and then removing the copied one or more requests from the request log; and
processing the one or more requests in the in-flight request queue, wherein the one or more requests include any that were in the in-flight request queue when the failure of the process was encountered and any that were copied from the request log.
12. The non-transitory computer readable storage medium according to claim 11 , wherein the plurality of requests are also copied into a hash table associated with the inflight request queue, and each request of the plurality of requests is indexed in the hash table according to a hash thereof.
13. The non-transitory computer readable storage medium according to claim 12 , wherein the request is removed from the client request queue and the request log after the request is copied into the inflight request queue and the hash table.
14. The non-transitory computer readable storage medium according to claim 10 , wherein the method further comprises upon completing the processing of a given request by the process, sending a completion response to the client.
15. The non-transitory computer readable storage medium according to claim 14 , wherein sending the completion response to the client includes:
copying the completion response to the given request into a completion log, which is also maintained in the protected memory region;
removing the given request corresponding to the completion response from the inflight request queue;
copying the completion response into a completion queue; and
after copying the completion response into the completion queue, removing the completion response from the completion log.
16. The non-transitory computer readable storage medium according to claim 15 ,
wherein the plurality of requests are also copied into a hash table associated with the inflight request queue when the plurality of requests are copied into the inflight request queue, and
prior to removing the request from the inflight request queue, the hash table associated with the inflight request queue is used to find the request.
17. The non-transitory computer readable storage medium of 11 , wherein the protected memory region is a POSIX shared memory object.
18. A system comprising:
a processor; and
a memory coupled to the processor and containing instructions to configure the processor to run a hypervisor that supports one or more virtual machines and a user space process,
the memory including a protected memory region that is a shared memory space supported by the hypervisor in which the hypervisor maintains a request log and an inflight request queue, the request log and the inflight request queue for storing requests that are preserved in the protected memory region even when the user space process encounters a failure;
wherein, prior to the user space process encountering the failure:
the user space process copies a plurality of requests from a client request queue into the request log and the inflight request queue;
directly in response to completing copying of a request of the plurality of requests into the inflight request queue and the request log, the user space process removes the request from the request log and the client request queue; and
the user space process processes one or more of the plurality of requests out of the inflight request queue; and
wherein, when the user space process is restarted following the failure thereof, the user space process:
determines whether the request log contains one or more requests and, if so, copies the one or more requests from the request log into the in-flight request queue and then removes the copied one or more requests from the request log; and
processes the one or more requests in the in-flight request queue, wherein the one or more requests include any that were in the in-flight request queue when the failure of the user space process was encountered and any that were copied from the request log.
19. The system according to claim 18 , wherein the user space process, upon completing the processing of a given request, sends a completion response to a client by:
copying the completion response to the given request into a completion log, which is also maintained in the protected memory region;
removing the given request corresponding to the completion response from the inflight request queue;
copying the completion response into a completion queue; and
after copying the completion response into the completion queue, removing the completion response from the completion log.
20. The system of 18 , wherein the protected memory region is a POSIX shared memory object.