Method and system for calculating latency in a data flow
The present invention provides a system for calculating latency in a distributed data flow environment. The system includes a plurality of hop servers and a central registry server. The plurality of hops servers is configured to receive a plurality of data packets across a time window, bucket each of the data packets of the plurality of data packets received across the time window into one or more buckets and transmit count of the plurality of the data packets in the one or more buckets and a window identifier associated with the time window to the central registry server. Each of the data packet of the plurality of data packets includes a header. The central registry server is configured to receive a plurality of counts and a plurality of window identifiers, to receive a latency query and to calculate latency.
1. A system for calculating latency in a distributed data flow environment, the system comprising:
a central registry server; wherein the central registry server is configured to:
a. receive a plurality of counts and a plurality of identifiers of time windows from a plurality of application servers, wherein the plurality of counts indicates plurality of data packets in multiple buckets, and wherein each of the buckets comprises a plurality of data packets grouped together based on a header of each of the data packet from the plurality of the data packets, wherein the header of each data packet of the plurality of data packets includes a timestamp representing a time at which each of the data packets originated and at least two of the buckets correspond to respective time windows representing different periods of time and referenced to a common originating data packet origin time;
b. receive a latency query; and
c. calculate latency based on the buckets and the identifier of the time windows, the referenced origin time, and the time windows for each bucket to facilitate minimizing latency in the distributed data flow environment.
2. The system of claim 1 , wherein the plurality of application servers further comprises one or more data agents, each data agent of the one or more data agents is configured to parse the header of the each data packet of the plurality of data packets.
3. The system of claim 2 , wherein the header of each data packet of the plurality of data packets comprises a timestamp at which the each data packet of the plurality of data packets originated.
4. The system of claim 1 further comprising:
the plurality of application servers coupled to the central registry server, wherein each application server of the plurality of application servers is configured to:
a. receive the plurality of data packets across the time windows, wherein each of the data packets of the plurality of data packets includes a header;
b. bucket each of the data packets of the plurality of data packets received across the time window by the application server into multiple buckets according to the header, wherein at least two of the buckets correspond to time windows representing different periods of time and referenced to a common originating data packet origin time; and
c. transmit at least one of the counts of the plurality of the data packets in the buckets and window identifiers associated with the time windows to the central registry server.
5. The system of claim 1 , wherein to calculate latency based on the plurality of counts received by a first one of the application servers from a second one of the application servers and the plurality of window identifiers comprises to determine a weighted average of maximum latency times for each count based window identifier, the time window, and referenced sending time for each bucket.
6. The system of claim 1 , wherein at least one of the application servers is an advertisement server.
7. A method of calculating latency in a distributed data flow environment, the method comprising:
a. receiving a plurality of counts and a plurality of identifiers of time windows from a plurality of application servers, wherein the plurality of counts indicates plurality of data packets in multiple buckets, and wherein each of the buckets comprises a plurality of data packets grouped together based on a header of each of the data packet from the plurality of the data packets, wherein the header of each data packet of the plurality of data packets includes a timestamp representing a time at which each of the data packets originated and at least two of the buckets correspond to respective time windows representing different periods of time and referenced to a common originating data packet origin time;
b. receiving a latency query; and
c. calculating latency based on the buckets and the identifier of the time windows, the referenced origin time, and the time windows for each bucket to facilitate minimizing latency in the distributed data flow environment.
8. The method of claim 7 , wherein the header of each data packet of the plurality of data packets comprises a timestamp at which the each data packet of the plurality of data packets originated.
9. The method of claim 7 , wherein calculating latency based on the plurality of counts received by a first one of the application servers from a second one of the application servers and the plurality of window identifiers comprises to determine a weighted average of maximum latency times for each count based window identifier, the time window, and referenced sending time for each bucket.
10. The method of claim 7 , wherein at least one of the application servers is an advertisement server.
11. The method of claim 7 , wherein the plurality of application servers further comprises one or more data agents, each data agent of the one or more data agents is configured to parse the header of the each data packet of the plurality of data packets.
12. The method of claim 7 further comprising:
receiving the plurality of data packets across the time windows, wherein each of the data packets of the plurality of data packets includes a header;
bucketing each of the data packets of the plurality of data packets received across the time window by the application server into multiple buckets according to the header, wherein at least two of the buckets correspond to time windows representing different periods of time and referenced to a common originating data packet origin time; and
transmitting at least one of the counts of the plurality of the data packets in the buckets and window identifiers associated with the time windows to the central registry server.
13. One or more non-transitory, computer program products, wherein at least one of the non-transitory, computer program products comprises a first program stored therein and executable by a processor to:
receive a plurality of counts and a plurality of identifiers of time windows from a plurality of application servers, wherein the plurality of counts indicates plurality of data packets in multiple buckets, and wherein each of the buckets comprises a plurality of data packets grouped together based on a header of each of the data packet from the plurality of the data packets, wherein the header of each data packet of the plurality of data packets includes a timestamp representing a time at which each of the data packets originated and at least two of the buckets correspond to respective time windows representing different periods of time and referenced to a common originating data packet origin time;
receive a latency query; and
calculate latency based on the buckets and the identifier of the time windows, the referenced origin time, and the time windows for each bucket to facilitate minimizing latency in the distributed data flow environment.
14. The one or more non-transitory, computer program products of claim 13 , wherein at least one of the non-transitory, computer program products comprises a second program stored therein and executable by a processor to:
receive the plurality of data packets across the time windows, wherein each of the data packets of the plurality of data packets includes a header;
bucket each of the data packets of the plurality of data packets received across the time window by the application server into multiple buckets according to the header, wherein at least two of the buckets correspond to time windows representing different periods of time and referenced to a common originating data packet origin time; and
transmit at least one of the counts of the plurality of the data packets in the buckets and window identifiers associated with the time windows to a central registry server.
15. The one or more non-transitory, computer program products of claim 13 , wherein the header of each data packet of the plurality of data packets comprises a timestamp at which the each data packet of the plurality of data packets originated.
16. The one or more non-transitory, computer program products of claim 13 , wherein to calculate latency based on the plurality of counts received by a first one of the application servers from a second one of the application servers and the plurality of window identifiers comprises to determine a weighted average of maximum latency times for each count based window identifier, the time window, and referenced sending time for each bucket.
17. The one or more non-transitory, computer program products of claim 13 , are installed in a data flow environment, and the data flow environment comprises multiple application servers coupled together, and at least one of the application servers is an advertisement server.
18. The one or more non-transitory, computer program products of claim 17 , wherein the plurality of application servers further comprises one or more data agents, each data agent of the one or more data agents is configured to parse the header of the each data packet of the plurality of data packets.