Publish-subscribe system with reduced data storage and transmission requirements
A publish-subscribe data messaging system in which wildcard subscriptions are represented in an approximate membership set or filter, such as a Bloom filter, for distribution to brokers together with formatting rules associated with the wildcard subscriptions, where a broker receiving a publication topic of a data publication for distribution to other brokers uses the formatting rules provided by any given broker to create wildcard versions of the publication topic for matching with the wildcard subscriptions that are represented in the approximate membership sets or filters provided by the given brokers, where, if a match exists, the data publication is forwarded to the given broker.
1. A method for operating a publish-subscribe data messaging system, the method comprising:
receiving at a first computer-hosted software application a publication topic associated with a data publication, wherein the publication topic is expressed as a hierarchy of topic levels;
selecting a formatting rule associated with a wildcard subscription that is represented in a collection of subscriptions of a second computer-hosted software application,
wherein the wildcard subscription is expressed as a hierarchy of topic levels, and
wherein the formatting rule identifies at least one topic level of the wildcard subscription that is represented with a wildcard token;
for each topic level of the wildcard subscription that the formatting rule indicates is represented with a wildcard token, and that corresponds to a topic level of the publication topic, replacing the corresponding topic level of the publication topic with the wildcard token from the corresponding topic level of the wildcard subscription, thereby creating a wildcard version of the publication topic;
determining if a match exists in the collection of subscriptions for the wildcard version of the publication topic; and
if the match exists, forwarding the data publication from the first computer-hosted software application to the second computer-hosted software application via a data communications medium.
2. The method according to claim 1 wherein the determining is performed wherein the collection of subscriptions is represented in an approximate membership set or filter.
3. The method according to claim 2 wherein the determining is performed wherein the collection of subscriptions is represented in a Bloom filter.
4. The method according to claim 1 wherein the replacing further comprises:
truncating the publication topic beyond a topic level of the publication topic that corresponds to the lowest topic level of the wildcard subscription that is represented with a wildcard token; and
representing the truncated publication topic in the wildcard version of the publication topic.
5. The method according to claim 1 and further comprising sending, in association with the data publication, either of the formatting rule and an identifier identifying the formatting rule to the second computer-hosted software application.
6. The method according to claim 1 wherein the receiving comprises receiving wherein the computer-hosted software applications are brokers in a publish-subscribe data messaging system.
7. The method of claim 1 wherein the receiving, selecting, replacing, determining, and forwarding are implemented in any of
a) computer hardware, and
b) computer software embodied in a non-transitory, computer-readable medium.
8. A publish-subscribe data messaging system comprising:
a topic formatter configured to
receive at a first computer-hosted software application a publication topic associated with a data publication, wherein the publication topic is expressed as a hierarchy of topic levels,
select a formatting rule associated with a wildcard subscription that is represented in a collection of subscriptions of a second computer-hosted software application,
wherein the wildcard subscription is expressed as a hierarchy of topic levels, and
wherein the formatting rule identifies at least one topic level of the wildcard subscription that is represented with a wildcard token, and
for each topic level of the wildcard subscription that the formatting rule indicates is represented with a wildcard token, and that corresponds to a topic level of the publication topic, replace the corresponding topic level of the publication topic with the wildcard token from the corresponding topic level of the wildcard subscription, thereby creating a wildcard version of the publication topic; and
a topic matcher configured to determine if a match exists in the collection of subscriptions for the wildcard version of the publication topic, and
a publication forwarder configured to forward the data publication from the first computer-hosted software application to the second computer-hosted software application via a data communications medium if the match exists.
9. The system according to claim 1 wherein the collection of subscriptions is represented in an approximate membership set or filter.
10. The system according to claim 9 wherein the collection of subscriptions is represented in a Bloom filter.
11. The system according to claim 1 wherein the topic formatter is configured to
truncate the publication topic beyond a topic level of the publication topic that corresponds to the lowest topic level of the wildcard subscription that is represented with a wildcard token, and
represent the truncated publication topic in the wildcard version of the publication topic.
12. The system according to claim 9 wherein the publication forwarder is configured to send, in association with the data publication, either of the formatting rule and an identifier identifying the formatting rule to the second computer-hosted software application.
13. The system according to claim 9 wherein the computer-hosted software applications are brokers in a publish-subscribe data messaging system.
14. The system of claim 8 wherein the topic formatter, topic matcher, and publication forwarder are implemented in any of
a) computer hardware, and
b) computer software embodied in a non-transitory, computer-readable medium.
15. A computer program product for operating a publish-subscribe data messaging system, the computer program product comprising:
a non-transitory, computer-readable storage medium; and
computer-readable program code embodied in the storage medium, wherein the computer-readable program code is configured to
receive at a first computer-hosted software application a publication topic associated with a data publication, wherein the publication topic is expressed as a hierarchy of topic levels,
select a formatting rule associated with a wildcard subscription that is represented in a collection of subscriptions of a second computer-hosted software application,
wherein the wildcard subscription is expressed as a hierarchy of topic levels, and
wherein the formatting rule identifies at least one topic level of the wildcard subscription that is represented with a wildcard token,
for each topic level of the wildcard subscription that the formatting rule indicates is represented with a wildcard token, and that corresponds to a topic level of the publication topic, replace the corresponding topic level of the publication topic with the wildcard token from the corresponding topic level of the wildcard subscription, thereby creating a wildcard version of the publication topic,
determine if a match exists in the collection of subscriptions for the wildcard version of the publication topic, and
if the match exists, forward the data publication from the first computer-hosted software application to the second computer-hosted software application via a data communications medium.
16. The computer program product according to claim 15 wherein the collection of subscriptions is represented in an approximate membership set or filter.
17. The computer program product according to claim 16 wherein the collection of subscriptions is represented in a Bloom filter.
18. The computer program product according to claim 15 wherein the computer-readable program code is configured to
truncate the publication topic beyond a topic level of the publication topic that corresponds to the lowest topic level of the wildcard subscription that is represented with a wildcard token, and
represent the truncated publication topic in the wildcard version of the publication topic.
19. The computer program product according to claim 15 wherein the computer-readable program code is configured to send, in association with the data publication, either of the formatting rule and an identifier identifying the formatting rule to the second computer-hosted software application.
20. The computer program product according to claim 15 wherein the computer-hosted software applications are brokers in a publish-subscribe data messaging system.