Subscription-based data delivery system
There is provided a computer-implemented method comprising, at an intermediary process acting as an intermediary between a publisher system and a plurality of subscriber processes, the intermediary process having a publisher data store storing one or more subscriptions for which the intermediary process is a publisher and a subscriber data store storing one or more subscriptions for which the intermediary process is a subscriber.
1 . A computer-implemented method comprising, at an intermediary process acting as an intermediary between a publisher system and a plurality of subscriber processes, the intermediary process having a publisher data store storing one or more subscriptions for which the intermediary process is a publisher and a subscriber data store storing one or more subscriptions for which the intermediary process is a subscriber:
receiving, from a first one of the plurality of subscriber processes, a first subscription request comprising a first subscription for the first subscriber process;
storing, in the publisher data store, a record associating the first subscriber process with the first subscription;
determining whether the one or more subscriptions in the subscriber data store collectively cover the first subscription; and
responsive to determining that the one or more subscriptions in the subscriber data store do not collectively cover at least a portion of the first subscription, sending, to the publisher system, a second subscription request comprising a second subscription for the intermediary process, the second subscription comprising at least the at least a portion of the first subscription, and storing, in the subscriber data store, a record of the second subscription.
2 . The method of claim 1 , wherein the second subscription comprises at least one of:
the subscriptions stored in the subscriber data store; or
an additional subscription for content neighbouring the first subscription.
3 . The method of claim 1 , wherein the first subscription comprises a query for selecting data from a database, wherein the method further comprises decomposing the query into a plurality of primitive queries, wherein determining whether the one or more subscriptions in the subscriber data store collectively cover the first subscription comprises determining whether the one or more subscriptions in the subscriber data store collectively cover each of the plurality of primitive queries, wherein the sending of the second subscription request is responsive to determining that the one or more subscriptions in the subscriber data store do not collectively cover at least one of the plurality of primitive queries, and wherein the second subscription comprises at least the at least one of the plurality of primitive queries.
4 . The method of claim 1 , wherein the intermediary process has a state data store storing state data, and the method further comprises: receiving first event data from the publisher system; and updating the state data in the state data store based on the first event data.
5 . The method of claim 4 , further comprising:
generating, from the state data in the state data store, second event data according to a first subscription associated with the first subscriber process in the publisher data store; and sending the second event data to the first subscriber process.
6 . The method of claim 5 , further comprising: receiving, from a second one of the plurality of subscriber processes, a third subscription request comprising a third subscription for the second subscriber process; storing, in the publisher data store, a record associating the second subscriber process with the third subscription; determining that the one or more subscriptions in the subscriber data store collectively cover the third subscription; subsequent to updating the state data, generating, from the state data in the state data store, fourth event data according to the subscription associated with the second subscriber process in the publisher data store; and sending the fourth event data to the second subscriber process.
7 . The method of claim 1 , wherein the subscriptions are for content having one or more particular spatial characteristics.
8 . The method of claim 1 , wherein each subscription comprises a query for selecting data from a database.
9 . The method of claim 1 , wherein the subscriptions are for content or data in a particular region of a space, and wherein determining whether the one or more subscriptions in the subscriber data store collectively cover the first subscription comprises determining whether the particular region corresponding to the first subscription is a subset of the particular regions corresponding to the one or more subscriptions in the subscriber data store.
10 . A computer-implemented method comprising, at an intermediary process acting as an intermediary between a publisher system and a plurality of subscriber processes, the intermediary process having a publisher data store storing one or more subscriptions for which the intermediary process is a publisher, a subscriber data store storing one or more subscriptions for which the intermediary process is a subscriber, and a state data store storing state data:
receiving first event data from the publisher system;
updating the state data in the state data store based on the first event data;
generating, from the state data in the state data store, second event data according to a first subscription associated with a first one of the subscriber processes in the publisher data store; and
sending the second event data to the first subscriber process.
11 . The method of claim 10 , further comprising:
subsequent to updating the state data, receiving third event data from the publisher system; and
updating the state data in the state data store based on the third event data,
wherein the generating is subsequent to updating the state data based on the first event data and updating the state data based on the third event data and the generating is from the state data in the state data store which takes into account the first event data and the third event data.
12 . The method of claim 10 , wherein the intermediary process is a first intermediary process, further comprising, at a second intermediary process acting as an intermediary between the publisher system and a second plurality of subscriber processes, the second intermediary process having a second publisher data store storing one or more subscriptions for which the second intermediary process is a publisher, a second subscriber data store storing one or more subscriptions for which the second intermediary process is a subscriber, and a second state data store storing state data:
receiving third event data from the publisher system;
updating the state data in the second state data store based on the third event data;
generating, with the first intermediary process and from the state data in the second state data store and the state data in the first state data store, fourth event data according to a subscription associated with a given one of the second subscriber processes in the second publisher data store; and
sending the fourth event data to the given one of the second subscriber processes.
13 . The method of claim 10 , wherein the second event data is generated responsive to:
determining that a connection between the first subscriber process and the intermediary process is ready to be written to, having completed previous data sends.
14 . The method of claim 10 , wherein the intermediary process forms part of an intermediary system and wherein the intermediary system is a distributed system.
15 . The method of claim 10 , wherein the second event data is generated responsive to determining that a timer has expired.
16 . The method of claim 10 , wherein the subscriptions are for content having one or more particular spatial characteristics.
17 . The method of claim 10 , wherein each subscription comprises a query for selecting data from a database.
18 . The method of claim 10 , wherein the state data store comprises an active state data store and an inert state data store, the second event data is generated from the state data in the active state data store, and updating the state data comprises:
updating the state data in the inert state data store based on the first event data;
atomically swapping the inert state data store and the active state data store; and
updating the state data in the inert state data store to match the state data in the active state data store.
19 . The method of claim 10 , further comprising receiving, from the first subscriber process, a first subscription request comprising the first subscription for the first subscriber process and storing, in the publisher data store, a record associating the first subscriber process with the first subscription, wherein the first subscription request comprises an indication of computing capabilities of the first subscriber process, and wherein generating the second event data comprises adapting the second event data to the computing capabilities of the first subscriber process.
20 . A non-transitory computer-readable medium comprising instructions which, when executed by one or more computers, cause the one or more computers to perform a method comprising, at an intermediary process acting as an intermediary between a publisher system and a plurality of subscriber processes, the intermediary process having a publisher data store storing one or more subscriptions for which the intermediary process is a publisher, a subscriber data store storing one or more subscriptions for which the intermediary process is a subscriber, and a state data store storing state data:
receiving first event data from the publisher system;
updating the state data in the state data store based on the first event data;
generating, from the state data in the state data store, second event data according to a first subscription associated with a first one of the subscriber processes in the publisher data store; and
sending the second event data to the first subscriber process.