Methods and apparatus for synchronization of data set representations in a bandwidth-adaptive manner
View Patent ↗A bandwidth-adaptive method for synchronizing a consumer node representation of a dynamic data set and the source node representation of the dynamic data includes the step of receiving, from a source node, metadata information identifying a plurality of data packets that represent a state of at least a portion of a changing data set at a point in time. At least one of the identified data packets is received from the source node and at least one of the received data packets is selected responsive to the received meta-data information. The metadata information and the selected at least one data packet are transmitted to a consumer node.
1. In a system having a source node and a plurality of consumer nodes, a bandwidth-adaptive method for synchronizing a consumer node representation of a dynamic data set and the source node representation of the dynamic data set, the method comprising the steps of:
(a) receiving, at a communications service from a source node, a first metadata packet identifying a first plurality of data packets that represent a first state of at least a portion of a changing data set at a first point in time and a second metadata packet identifying a second plurality of data packets that represent a second state of at least a portion of the changing data set at a second point in time, wherein the first and second metadata packets identify at least one identical data packet that has not changed between the first and second points in time;
(b) receiving, at the communication service from the source node, at least one of the identified first plurality of data packets and at least one of the identified second plurality of data packets;
(c) selecting at least one of the received first plurality of data packets responsive to the received first metadata packet that has not yet been transmitted to a first consumer node and at least one of the received second plurality of data packets responsive to the received second metadata packet that has not yet been transmitted to a second consumer node;
(d) transmitting, from the communications service to the first consumer node, the first metadata packet and the selected at least one of the first plurality of data packets; and
(e) transmitting, from the communications service to the second consumer node, the second metadata packet and the selected at least one of the second plurality of data packets;
wherein the first consumer node receives metadata packet and data packet pairs more frequently than the second consumer node.
2. The method of claim 1 further comprising the step of, before step (c), receiving a request from at least one of the first and second consumer nodes for the current state of the changing data set.
3. The method of claim 2 further comprising the step of repeating steps (a) and (b) until the request is received.
4. The method of claim 3 wherein selecting at least one of the received first plurality of data packets in step (c) comprises the steps of:
(c-a) selecting the first metadata packet; and
(c-b) selecting at least one of the received first plurality of data packets identified by the selected first metadata packet.
5. The method of claim 1 wherein selecting at least one of the received first plurality of data packets in step (c) comprises selecting a plurality of the received first plurality of data packets responsive to the received first metadata packet.
6. The method of claim 5 wherein step (d) comprises transmitting to the first consumer node each of the selected first plurality of data packets.
7. The method of claim 1 wherein step (b) comprises receiving from the source node at least one of the identified data packets in encrypted form.
8. The method of claim 1 further comprising the step of storing the received metadata packets in a memory device.
9. The method of claim 1 further comprising the step of storing received data packets in a memory device.
10. The method of claim 9 wherein selecting at least one of the received first plurality of data packets in step (c) comprises:
(c-a) selecting at least one of the received first plurality of data packets responsive to the received first metadata packet; and
(c-b) selecting at least one of the stored data packets responsive to the received first metadata packet.
11. The method of claim 10 where step (d) comprises:
(d-a) transmitting to the first consumer node the selected at least one of the received first plurality of data packets; and
(d-b) transmitting to the consumer node the selected at least one of the stored data packets.
12. The method of claim 1 further comprising the step of storing, in a memory element, a metadata packet identifying a data packet transmitted to one of the first and second consumer nodes.
13. The method of claim 12 wherein selecting a set of the identified data packets in step (c) comprises selecting at least one of the received first plurality of data packets responsive to the received first metadata packet and the stored information identifying the at least one of the first plurality of data packets transmitted to the first consumer node.
14. A bandwidth-adaptive system synchronizing consumer node representations and a source node representation of a changing data set, the system comprising:
a source node transmitting a first metadata packet identifying a first plurality of data packets that represent a first state of a changing data set and transmitting at least one of the identified first plurality of data packets and transmitting a second metadata packet identifying a second plurality of data packets that represent a second state of the changing data set and transmitting at least one of the identified second plurality of data packets, wherein the first and second metadata packets identify at least one identical data packet that has not changed between the first and second states; and
a communications service in communication with the source node, the communications service selecting one pair of the first metadata packet and the at least one of the identified first plurality of data packets that has not yet been received by a first consumer node for transmission to the first consumer node and selecting one pair of the second metadata packet and the at least one of the identified second plurality of data packets that has not yet been received by a second consumer node for transmission to the second consumer node;
wherein the first consumer node receives metadata packet and data packet pairs more frequently than the second consumer node.
15. A communications service synchronizing consumer node representations and a source node representation of a changing data set, the service comprising:
a receiving subsystem receiving a first metadata packet identifying a first plurality of data packets representing a first state of a changing data set and at least one data packet identified by the first metadata packet and receiving a second metadata packet identifying a second plurality of data packets representing a second state of the changing data set and at least one data packet identified by the second metadata packet, wherein the first and second metadata packets identify at least one identical data packet that has not changed between the first and second states;
a synchronization engine selecting a first pair of the first metadata packet and the at least one data packet identified by the first metadata packet that has not yet been received by a first consumer node and selecting a second pair of the second metadata packet and the at least one data packet identified by the second metadata packet that has not yet been received by a second consumer node; and
a transmission subsystem transmitting the first pair to the first consumer node and the second pair to the second consumer node;
wherein the first consumer node receives metadata packet and data packet pairs more frequently than the second consumer node.