Dynamic subscription and message routing on a topic between publishing nodes and subscribing nodes
A system for dynamic message routing on a topic between publishing nodes and subscribing nodes includes a plurality of message queues, at least one topic/node table, a subscribing module, a publishing module, and other modules to send messages between one or more publisher and one or more subscribers. These modules are coupled together by a bus in a plurality of nodes and provide for the dynamic message routing on a topic between publishing nodes and subscribing nodes. The message queues store messages at each node for delivery to subscribers local to that node. The topic/node table lists which clients subscribe to which topics, and is used by the other modules to ensure proper distribution of messages. The subscribing module is use to establish a subscription to a topic for that node. The publishing module is used to identify subscribers to a topic and transmit messages to subscribers dynamically. The other modules include various devices to optimize message communication in a publish/subscribe architecture operating on a distributed computing system. The present invention also includes a number of novel methods including: a method for publishing a message on a topic, a method for forwarding a message on a topic, a method for subscribing to messages on a topic, a method for automatically removing subscribers, a method for direct publishing of messages, and methods for optimizing message transmission between nodes.
1. A system for dynamically routing a message over a network having a first node and a plurality of other nodes, the system comprising:
a publisher for generating and sending the message on a topic, the publisher having an output;
a plurality of subscribers each having an input for receiving the message, at least one of the plurality of subscribers subscribing to the topic; and
a broker having an input, an output and a topic/node table, the broker associated with the first node, in response to receipt of the message, the broker identifying the topic associated with the message and determining using the topic/node table and the identified topic which of the plurality of other nodes to which to send the message for delivery to the one of the plurality of subscribers subscribing to the topic, the input of the broker coupled to the output of the publisher, and the output of the broker coupled to the inputs of the plurality of subscribers.
2. The system of claim 1 wherein the publisher is a client application program.
3. The system of claim 1 wherein the subscriber is a client application program.
4. The system of claim 1 wherein the broker farther comprises a message queue for storing messages received from the publisher.
5. The system of claim 1 wherein the broker farther comprises a dead message queue for storing messages determined by the broker to be undeliverable.
6. The system of claim 1 wherein the topic/node table is capable of storing a topic and a subtopic for a subscriber or a second broker.
7. The system of claim 1 further comprising:
a second broker associated with one of the plurality of other nodes having an input and an output, the second broker routing the message in response to receipt of the message, the second broker communicatively coupled to the first broker; and
a cluster topic/node table, the cluster topic/node table used by the broker and the second broker to determine which of the plurality of other nodes to which to send the message for delivery.
8. The system of claim 1 wherein the broker further comprises a subscribing module for establishing a subscription to the topic by a new subscriber, the subscribing module coupled for communication with the new subscriber, the subscribing module updating the topic/node table with the new subscriber.
9. The system of claim 1 wherein the broker farther comprises a publishing module, the publishing module coupled to the topic/node table, the publishing module determining the subscribers for the topic and a path to the subscribers, the publishing module sending the message to the subscribers over the determined path.
10. The system of claim 1 wherein the broker further comprises a forwarding module having an input and an output for routing messages in response to receipt of the message from another broker, the forwarding module determining which of the plurality of other nodes to which to send the message for delivery, the input of the forwarding module coupled to receive messages from the other broker, the output of the forwarding module coupled to the inputs of the plurality of subscribers.
11. The system of claim 1 further comprising a path selection module coupled to the broker, the path selection module for identifying at least one path through the plurality of other nodes from the broker to the one of the plurality of subscribers, the path selection module providing the identified path to the broker for routing of the message over the identified path.
12. The system of claim 11 wherein the path selection module provides real-time adjustment of the identified path by monitoring connections between the plurality of other nodes and recalculating the best path.
13. The system of claim 1 further comprising a traffic control module for controlling message traffic over the plurality of other nodes, the traffic control module monitoring connections between the plurality of other nodes for changes in bandwidth and re-routing topics according to a bandwidth preference.
14. The system of claim 1 further comprising a delivery control module for eliminating duplicate messages, the delivery control module coupled to the broker to receive messages.
15. The system of claim 1 further comprising a delivery control module for filtering messages, the deliver control module coupled to the broker to receive and send messages, the deliver control module comparing a selector to the message received from the broker and dropping the message if it does not match the selector.
16. The system of claim 1 further comprising a direct publishing module having an input and an output for publishing a message directly to a cluster, the direct publishing module coupled to the broker, the direct publishing module receiving a cluster identification, determining a path to the cluster and sending the message to the cluster.