Synchronous message queues
A method comprising receiving a write request; adding the write request to a batch of substantially contiguous disk writes; determining to write the batch of substantially contiguous disk writes to a non-volatile memory; writing the batch of substantially contiguous disk writes to the non-volatile memory; sending a confirmation of writing the batch of substantially contiguous disk writes; receiving a confirmation of the confirmation of writing; and clearing the batch of substantially contiguous disk writes.
1. A method comprising:
receiving a write request;
adding the write request to a batch of disk writes;
determining to write the batch of disk writes to a non-volatile memory;
writing the batch of disk writes to the non-volatile memory;
sending a confirmation of writing the batch of disk writes;
receiving a confirmation of the confirmation of writing; and
clearing the batch of disk writes.
2. The method of claim 1 wherein receiving the write request comprises:
receiving the write request for a memory location that is contiguous to a memory location in the batch of disk writes.
3. The method of claim 1 wherein receiving a write request comprises:
receiving the write request from a processing thread.
4. The method of claim 3 wherein receiving the write request from a processing thread comprises:
receiving the write request from a processing thread located in a multimedia message gateway.
5. The method of claim 1 wherein determining to write the batch of disk writes to a non-volatile memory comprises:
determining a pre-defined period of time has elapsed since a previous write of the batch of disk writes to the non-volatile memory.
6. The method of claim 1 wherein determining to write the batch of disk writes to a non-volatile memory comprises:
determining a pre-defined size of the batch of disk writes has been exceeded.
7. The method of claim 1 wherein determining to write the batch of disk writes to a non-volatile memory comprises:
determining the batch of disk writes contains a plurality of contiguous write requests.
8. The method of claim 1 wherein determining to write the batch of disk writes to a non-volatile memory comprises:
determining a size of the batch of disk writes exceeds a predetermined maximum number of bytes.
9. The method of claim 1 wherein determining to write the batch of disk writes to a non-volatile memory comprises:
determining a number of write requests stored in the batch of disk writes exceeds a predetermined maximum number.
10. The method of claim 1 wherein determining to write the batch of disk writes to a non-volatile memory comprises:
determining a predetermined maximum time to gather the batch of disk writes has been exceeded.
11. The method of claim 1 wherein determining to write the batch of disk writes to a non-volatile memory comprises:
determining a predetermined maximum time since receipt of the last write request has been exceeded.
12. The method of claim 1 wherein determining to write the batch of disk writes to a non-volatile memory comprises:
determining the write request exceeds a maximum allowable page offset in the batch of disk writes.
13. The method of claim 1 wherein determining to write the batch of disk writes to a non-volatile memory comprises:
determining the write request exceeds a maximum allowable byte offset in the batch of disk writes.
14. The method of claim 1 wherein writing the batch of disk writes to the non-volatile memory comprises:
writing the batch of disk writes to the non-volatile memory asynchronously.
15. The method of claim 1 wherein writing the batch of disk writes to the non-volatile memory comprises:
determining whether the batch of disk writes is memory mapped;
memory mapping the batch of disk writes, if the batch of disk writes is not memory mapped; and
writing the batch of disk writes to the non-volatile memory.
16. The method of claim 1 further comprising:
determining whether there is an outstanding unprocessed write request in a temporary storage; and
adding the outstanding unprocessed write request to a new batch of disk writes, if there is an outstanding unprocessed write request in the temporary storage.
17. A machine-readable medium having stored thereon a plurality of executable instructions to perform a method comprising:
receiving a write request;
adding the write request to a batch of disk writes;
determining it is possible to write the batch of substantially contiguous disk writes to a non-volatile memory;
writing the batch of disk writes to the non-volatile memory;
sending a confirmation of writing the batch of disk writes;
receiving a confirmation of the confirmation of writing; and
clearing the batch of disk writes.
18. The machine-readable medium of claim 17 wherein receiving the write request comprises:
receiving the write request for a memory location that is contiguous to a memory location in the batch of disk writes.
19. The machine-readable medium of claim 17 wherein receiving a write request comprises:
receiving the write request from a processing thread.
20. The machine-readable medium of claim 19 wherein receiving the write request from a processing thread comprises:
receiving the write request from a processing thread located in a multimedia message gateway.
21. The machine-readable medium of claim 17 wherein determining to write the batch of disk writes to a non-volatile memory comprises:
determining a pre-defined period of time has elapsed since a previous write of the batch of disk writes to the non-volatile memory.
22. The machine-readable medium of claim 17 wherein determining to write the batch of disk writes to a non-volatile memory comprises:
determining a pre-defined size of the batch of disk writes has been exceeded.
23. The machine-readable medium of claim 17 wherein determining it is possible to write the batch of disk writes to a non-volatile memory comprises:
determining the batch of disk writes contains a plurality of contiguous write requests.
24. The machine-readable medium of claim 17 wherein determining to write the batch of disk writes to a non-volatile memory comprises:
determining a size of the batch of disk writes exceeds a predetermined maximum number of bytes.
25. The machine-readable medium of claim 17 wherein determining to write the batch of disk writes to a non-volatile memory comprises:
determining a number of write requests stored in the batch of disk writes exceeds a predetermined maximum number.
26. The machine-readable medium of claim 17 wherein determining to write the batch of disk writes to a non-volatile memory comprises:
determining a predetermined maximum time to gather the batch of disk writes has been exceeded.
27. The machine-readable medium of claim 17 wherein determining to write the batch of disk writes to a non-volatile memory comprises:
determining a predetermined maximum time since receipt of the last write request has been exceeded.
28. The machine-readable medium of claim 17 wherein determining to write the batch of disk writes to a non-volatile memory comprises:
determining the write request exceeds a maximum allowable page offset in the batch of disk writes.
29. The machine-readable medium of claim 17 wherein determining to write the batch of disk writes to a non-volatile memory comprises:
determining the write request exceeds a maximum allowable byte offset in the batch of disk writes.
30. The machine-readable medium of claim 17 wherein writing the batch of disk writes to the non-volatile memory comprises:
writing the batch of disk writes to the non-volatile memory asynchronously.
31. The machine-readable medium of claim 17 wherein writing the batch of disk writes to the non-volatile memory comprises:
writing the batch of disk writes to the non-volatile memory asynchronously.
32. The machine-readable medium of claim 17 wherein writing the batch of disk writes to the non-volatile memory comprises:
determining whether the batch of disk writes is memory mapped;
memory mapping the batch of disk writes, if the batch of disk writes is not memory mapped; and
writing the batch of disk writes to the non-volatile memory.
33. The machine-readable medium of claim 17 further comprising:
determining whether there is an outstanding unprocessed write request in a temporary storage; and
adding the outstanding unprocessed write request to a new batch of disk writes, if there is an outstanding unprocessed write request in the temporary storage.
34. A method comprising:
acquiring control of a shared mutual exclusion lock by a first thread or process;
sending a first write request from the first thread or process to a synchronization process;
queuing the first write request by the synchronization thread or process;
sending a confirmation of receipt of the first write request to the first thread or process;
releasing control of the shared mutual exclusion lock by the first thread or process upon receipt of the confirmation of the receipt of the first write request;
acquiring control of the shared mutual exclusion lock by a second thread or process;
sending a second write request from the second thread or process to the synchronization process;
queuing the second write request with the first write request by the synchronization process;
sending a confirmation of receipt of the second write request to the second thread or process;
releasing control of the shared mutual exclusion lock by the second thread or process upon receipt of the confirmation of the receipt of the second write request;
writing the queued first and second write requests to a non-volatile memory;
sending a confirmation of writing to the non-volatile memory to each of the first thread or process and the second thread or process;
sending a confirmation of receipt of the confirmation of writing from each of the first thread or process and the second thread or process; and
restarting the method, if the confirmation of receipt of the confirmation of writing is received from each of the first thread or process and the second thread or process.
35. The method of claim 34 wherein writing the queued first and second write requests to a non-volatile memory comprises:
determining whether a predetermined condition has occurred; and
writing the queued first and second write requests to the non-volatile memory, if the predetermined condition has occurred.
36. The method of claim 35 wherein the predetermined condition comprises:
a predetermined period of time is exceeded.
37. The method of claim 35 wherein the predetermined condition comprises:
a size of the queued write requests exceeds a predetermined maximum number of bytes.
38. The method of claim 35 wherein the predetermined condition comprises:
a number of queued write requests exceeds a predetermined maximum number.
39. The method of claim 35 wherein the predetermined condition comprises:
a predetermined maximum time to gather the queued write requests has been exceeded.
40. The method of claim 35 wherein the predetermined condition comprises:
a predetermined maximum time since receipt of a last write request has been exceeded.
41. The method of claim 35 wherein the predetermined condition comprises:
a new write request exceeds a maximum allowable page offset from the queued write requests.
42. The method of claim 35 wherein the predetermined condition comprises:
a new write request exceeds a maximum allowable byte offset from the queued write requests.
43. A machine-readable medium having stored thereon a plurality of executable instructions to perform a method comprising:
acquiring control of a shared mutual exclusion lock by a first thread or process;
sending a first write request from the first thread or process to a synchronization process;
queuing the first write request by the synchronization thread or process;
sending a confirmation of receipt of the first write request to the first thread or process;
releasing control of the shared mutual exclusion lock by the first thread or process upon receipt of the confirmation of the receipt of the first write request;
acquiring control of the shared mutual exclusion lock by a second thread or process;
sending a second write request from the second thread or process to the synchronization process;
queuing the second write request with the first write request by the synchronization process;
sending a confirmation of receipt of the second write request to the second thread or process;
releasing control of the shared mutual exclusion lock by the second thread or process upon receipt of the confirmation of the receipt of the second write request;
writing the queued first and second write requests to a non-volatile memory;
sending a confirmation of writing to the non-volatile memory to each of the first thread or process and the second thread or process;
sending a confirmation of receipt of the confirmation of writing from each of the first thread or process and the second thread or process; and
restarting the method, if the confirmation of receipt of the confirmation of writing is received from each of the first thread or process and the second thread or process.
44. The machine-readable medium of claim 43 wherein queuing the first write request comprises:
storing the first write request in a batch file.
45. The machine-readable medium of claim 44 wherein queuing the second write request comprises:
storing the second write request in the batch file.
46. The machine-readable medium of claim 43 wherein the queued first and second write requests are written upon occurrence of a predetermined condition.
47. The machine-readable medium of claim 46 wherein the predetermined condition comprises:
a predetermined period of time is exceeded.
48. The machine-readable medium of claim 46 wherein the predetermined condition comprises:
a size of the queued write requests exceeds a predetermined maximum number of bytes.
49. The machine-readable medium of claim 46 wherein the predetermined condition comprises:
a number of queued write requests exceeds a predetermined maximum number.
50. The machine-readable medium of claim 46 wherein the predetermined condition comprises:
a predetermined maximum time to gather the queued write requests has been exceeded.
51. The machine-readable medium of claim 46 wherein the predetermined condition comprises:
a predetermined maximum time since receipt of a last write request has been exceeded.
52. The machine-readable medium of claim 46 wherein the predetermined condition comprises:
a new write request exceeds a maximum allowable page offset from the queued write requests.
53. The machine-readable medium of claim 46 wherein the predetermined condition comprises:
a new write request exceeds a maximum allowable byte offset from the queued write requests.
54. An apparatus comprising:
a gateway;
a queue data file coupled to the gateway;
an index file coupled to the gateway; and
a synchronization component coupled to the gateway and the queue data file, the synchronization component to
receive a write request;
add the write request to a batch of disk writes;
determine whether to write the batch of disk writes to a non-volatile memory;
write the batch of disk writes to the non-volatile memory;
send a confirmation of writing the batch of disk writes;
receive a confirmation of the confirmation of writing; and
clear the batch of disk writes.
55. The apparatus of claim 54 wherein the gateway comprises:
a multi-media message gateway.
56. The apparatus of claim 54 wherein the queue data file and the index file are included in a disk queue.
57. The apparatus of claim 54 wherein the queue data file comprises:
a record including information sufficient to rebuild an associated entry for the record in the index file.
58. The apparatus of claim 54 wherein the index file comprises:
an entry associated with a record in the queue data file, the entry including a data structure having an index component and an entry information component.
59. The apparatus of claim 54 wherein the gateway is to read the queue data file and the index file to restore a queue processing state that existed prior to a system crash of the apparatus.
60. The apparatus of claim 54 wherein the queue data file is located on a single disk partition.
61. The apparatus of claim 54 wherein the queue data file comprises a plurality of contiguous disk blocks.
62. A method comprising:
receiving a write request;
determining whether to add the write request to a batch of outstanding requests;
adding the write request to the batch, if it is determined to add the write request to the batch;
determining whether to write the batch to a non-volatile memory;
determining whether the batch is memory-mapped to the non-volatile memory, if it is determined to write the batch to the non-volatile memory;
writing the batch to the non-volatile memory, if it is determined that the batch is memory-mapped to the non-volatile memory;
sending confirmations of writing the batch to a process associated with each write request in the batch;
determining whether all confirmations of writing the batch have been received;
clearing the batch, if all confirmations of writing the batch have been received;
determining whether any write requests were skipped or stored in a temporary storage; and
looping back to receive a next write request, if it is determined there are no skipped or stored write requests.
63. The method of claim 62 further comprising:
adding an unprocessed write request to a new batch of outstanding requests, if it is determined the outstanding unprocessed write request is in the temporary storage.
64. The method of claim 62 further comprising:
storing the write request in the temporary storage, if it is determined to store the write request in the temporary storage; and
looping back to receive a next write request, if it is determined to skip the write request.
65. The method of claim 62 further comprising:
looping back to receive a next write request, if it is determined not to write the batch to the non-volatile memory.
66. The method of claim 62 further comprising:
memory-mapping the batch with the non-volatile memory before writing the batch to the non-volatile memory, if it is determined the batch is not memory-mapped with the non-volatile memory.
67. The method of claim 62 further comprising:
determining whether to write the stored write request to the non-volatile memory, if it is determined that the stored write request is in the temporary storage.
68. A machine-readable medium having stored thereon a plurality of executable instructions to perform a method comprising:
receiving a write request;
determining whether to add the write request to a batch of outstanding requests;
adding the write request to the batch, if it is determined to add the write request to the batch;
determining whether to write the batch to a non-volatile memory;
determining whether the batch is memory-mapped to the non-volatile memory, if it is determined to write the batch to the non-volatile memory;
writing the batch to the non-volatile memory, if it is determined that the batch is memory-mapped to the non-volatile memory;
sending confirmations of writing the batch to a process associated with each write request in the batch;
determining whether all confirmations of writing the batch have been received;
clearing the batch, if all confirmations of writing the batch have been received;
determining whether any write requests were skipped or stored in a temporary storage; and
looping back to receive a next write request, if it is determined there are no skipped or stored write requests.
69. The machine-readable medium of claim 68 wherein the method further comprises:
adding an unprocessed write request to a new batch of outstanding requests, if it is determined the outstanding unprocessed write request is in the temporary storage.
70. The machine-readable medium of claim 68 wherein the method further comprises:
storing the write request in the temporary storage, if it is determined to store the write request in the temporary storage; and
looping back to receive a next write request, if it is determined to skip the write request.
71. The machine-readable medium of claim 68 wherein the method further comprises:
looping back to receive a next write request, if it is determined not to write the batch to the non-volatile memory.
72. The machine-readable medium of claim 68 wherein the method further comprises:
memory-mapping the batch with the non-volatile memory before writing the batch to the non-volatile memory, if it is determined the batch is not memory-mapped with the non-volatile memory.
73. The machine-readable medium of claim 68 wherein the method further comprises:
determining whether to write the stored write request to the non-volatile memory, if it is determined that the stored write request is in the temporary storage.