Managing message queues
A method, and corresponding system and software, is described for writing data to a plurality of queues, each portion of the data being written to a corresponding one of the queues. The method includes, without requiring concurrent locking of more than one queue, determining if a space is available in each queue for writing a corresponding portion of the data, and if available, reserving the spaces in the queues. The method includes writing each portion of the data to a corresponding one of the queues.
1. A method including:
assigning a first transaction to a first data structure that stores messages for a queue and is used exclusively for messages in the first transaction;
assigning a second transaction to a second data structure that stores messages for a queue, is separate from the first data structure, and is shared by a plurality of transactions;
storing a plurality of messages in the first transaction in the first data structure;
storing at least some messages in the second transaction in the second data structure;
committing the first and second transactions after the messages in the first transaction have been stored in the first data structure and the messages in the second transaction have been stored in the second data structure;
adding a message to the second data structure in response to the first transaction being committed, the message locating the messages stored in the first data structure;
reading messages in the second transaction from the second data structure after the second transaction has been committed; and
reading a plurality of the messages in the first transaction from contiguous memory locations after the first transaction has been committed.
2. The method of claim 1 , wherein adding the message to the second data structure includes storing a pointer to the first data structure in the second data structure.
3. The method of claim 2 , wherein reading a plurality of the messages associated with the first transaction from contiguous memory locations includes reading the messages from the first data structure.
4. The method of claim 1 , wherein the first data structure is stored in volatile memory and the second data structure is stored in non-volatile memory.
5. The method of claim 1 , further including:
moving the messages associated with the first transaction from the first data structure to a third data structure; and
wherein adding the message to the second data structure includes storing a pointer to the third data structure in the second data structure.
6. The method of claim 5 , wherein reading a plurality of the messages associated with the first transaction from contiguous memory locations includes reading the messages from the third data structure.
7. The method of claim 1 , wherein reading messages associated with the second transaction from the second data structure includes scanning each of the messages associated with the second transaction to generate a list and passing the list to a reader, and reading the messages associated with the second transaction; and
reading a plurality of the messages associated with the first transaction includes providing a pointer to the first data structure and passing the pointer to a reader that reads each of the messages associated with the first transaction.
8. The method of claim 7 , wherein the list includes pointers to each of the messages associated with the second transaction.
9. Software stored on a computer-readable storage medium including instructions for causing a computer system to:
assign a first transaction to a first data structure that stores messages for a queue and is used exclusively for messages in the first transaction;
assign a second transaction to a second data structure that stores messages for a queue, is separate from the first data structure, and is shared by a plurality of transactions;
store a plurality of messages associated with the first transaction in the first data structure;
store at least some messages in the second transaction in the second data structure;
commit the first and second transactions after the messages in the first transaction have been stored in the first data structure and the messages in the second transaction have been stored in the second data structure;
add a message to the second data structure in response to the first transaction being committed, the message locating the messages stored in the first data structure;
read messages in the second transaction from the second data structure after the second transaction has been committed; and
read a plurality of the messages in the first transaction from contiguous memory locations after the first transaction has been committed.
10. A system including:
means for assigning a first transaction to a first data structure that stores messages for a queue and is used exclusively for messages in the first transaction;
means for assigning a second transaction to a second data structure that stores messages for a queue, is separate from the first data structure, and is shared by a plurality of transactions;
means for storing a plurality of messages in the first transaction in the first data structure;
means for storing at least some messages in the second transaction in the second data structure;
means for committing the first and second transactions after the messages in the first transaction have been stored in the first data structure and the messages in the second transaction have been stored in the second data structure;
means for adding a message to the second data structure in response to the first transaction being committed, the message locating the messages stored in the first data structure;
means for reading messages in the second transaction from the second data structure after the second transaction has been committed; and
means for reading a plurality of the messages in the first transaction from contiguous memory locations after the first transaction has been committed.
11. The software of claim 9 , wherein adding the message to the second data structure includes storing a pointer to the first data structure in the second data structure.
12. The software of claim 11 , wherein reading a plurality of the messages associated with the first transaction from contiguous memory locations includes reading the messages from the first data structure.
13. The software of claim 9 , wherein the first data structure is stored in volatile memory and the second data structure is stored in non-volatile memory.
14. The software of claim 9 , further including instructions for causing the computer system to:
move the messages associated with the first transaction from the first data structure to a third data structure; and
wherein adding the message to the second data structure includes storing a pointer to the third data structure in the second data structure.
15. The software of claim 14 , wherein reading a plurality of the messages associated with the first transaction from contiguous memory locations includes reading the messages from the third data structure.
16. The software of claim 9 , wherein
reading messages associated with the second transaction from the second data structure includes scanning each of the messages associated with the second transaction to generate a list and passing the list to a reader, and reading the messages associated with the second transaction; and
reading a plurality of the messages associated with the first transaction includes providing a pointer to the first data structure and passing the pointer to a reader that reads each of the messages associated with the first transaction.
17. The software of claim 16 , wherein the list includes pointers to each of the messages associated with the second transaction.
18. The system of claim 10 , wherein adding the message to the second data structure includes storing a pointer to the first data structure in the second data structure.
19. The system of claim 18 , wherein reading a plurality of the messages associated with the first transaction from contiguous memory locations includes reading the messages from the first data structure.
20. The system of claim 10 , wherein the first data structure is stored in volatile memory and the second data structure is stored in non-volatile memory.
21. The system of claim 10 , further including:
means for moving the messages associated with the first transaction from the first data structure to a third data structure; and
wherein adding the message to the second data structure includes storing a pointer to the third data structure in the second data structure.
22. The system of claim 21 , wherein reading a plurality of the messages associated with the first transaction from contiguous memory locations includes reading the messages from the third data structure.
23. The system of claim 10 , wherein
reading messages associated with the second transaction from the second data structure includes scanning each of the messages associated with the second transaction to generate a list and passing the list to a reader, and reading the messages associated with the second transaction; and
reading a plurality of the messages associated with the first transaction includes providing a pointer to the first data structure and passing the pointer to a reader that reads each of the messages associated with the first transaction.
24. The system of claim 23 , wherein the list includes pointers to each of the messages associated with the second transaction.