Real-time adaptive processing of network data packets for analysis
View Patent ↗A network monitoring system that summarizes a plurality of data packets of a session into a compact session record for storage and processing. Each session record may be produced in real-time and made available during the session and/or after the termination of the session. Depending on protocols, a network monitoring system extracts different sets of information, removes redundant information from the plurality of data packets, and adds performance information to produce the session record. The network monitoring system may retrieve and process a single session record or multiple session records for the same or different protocols to determine cause of events, resolve issues in a network or evaluate network performance or conditions. The session record enables analysis in the units of session instead of individual packets. Hence, the network monitoring system can analyze events, issues or performance of the network more efficiently and effectively.
1. A method for processing network data for analysis, comprising:
receiving data packets by a network monitoring device at a location in a network;
determining a session associated with the received data packets;
generating a common header summarizing information in headers of a plurality of data packets in the session by the network monitoring device;
generating a single unit of payload metadata summarizing information in payloads of the plurality of the data packets in the session by the network monitoring device;
determining a protocol associated with the session;
generating performance parameters representing performance of the network for transmitting the plurality of the data packets in the session based on the protocol associated with the session;
generating, for each session, a session record including the common header, the payload metadata, and the performance parameters by the network monitoring device; and
storing the session record.
2. The method of claim 1 , wherein information included in the common header or the payload metadata of each session is determined based on a protocol associated with the session.
3. The method of claim 1 , wherein the common header and the payload metadata are generated in real-time responsive to receiving data packets for the session from the network.
4. The method of claim 1 , wherein the performance parameters comprise at least one of a packet loss rate, a response time, a number of packets or bytes transmitted, parameters indicative of transmission latency, codec parameters, total transfer time for the session, R-Factor (rating factor), MOS (Mean Opinion Score), a file or page size, a file name, sender/receive information, a total transfer time, a user ID, jitter, and phone number.
5. The method of claim 1 , further comprising generating mapping information representing at least one other session associated with the session, the mapping information included in the session record.
6. The method of claim 5 , wherein the session and the other session are associated with planes in a same protocol.
7. The method of claim 5 , wherein the session is associated with a first protocol and the other session is associated with a second protocol different than the first protocol.
8. The method of claim 1 , wherein the session record further stores links to the plurality of the data packets in the session.
9. The method of claim 1 , further comprising:
receiving a condition for a set of data packets;
identifying one or more session records matching the condition; and
determining a set of data packets associated with the one or more identified session records.
10. The method of claim 1 , wherein the common header comprises fields of the headers that remain unchanged throughout each session.
11. The method of claim 1 , wherein the session record is generated in a primary memory device of the network monitoring device without reading the received data packets from a secondary memory device of the network monitoring device, the secondary memory device having a slower access speed compared to the primary memory device.
12. A network monitoring device for processing network data for analysis, comprising:
a network interface configured to receive data packets at a location in a network;
a session tracing engine configured to:
determine a session associated with the received data packets;
generate a common header summarizing information in headers of a plurality of data packets in a session;
generate a single unit of payload metadata summarizing information in payloads of the plurality of the data packets in the session; and
determine a protocol associated with the session;
generate performance parameters representing performance of the network for transmitting the plurality of the data packets in the session based on the protocol associated with the session
generate, for each session, a session record including the common header, the payload metadata and the performance parameters; and
a storage device configured to store the generated session record.
13. The network monitoring device of claim 12 , wherein at least one of the common header and the payload metadata generated based on the protocol.
14. The network monitoring device of claim 13 , wherein the session tracing engine is configured to add links to the plurality of the data packets in the session record.
15. The network monitoring device of claim 13 , wherein the common header comprises fields of the headers that remain unchanged throughout each session.
16. The network monitoring device of claim 12 , wherein the session tracing engine is configured to generate the common header and the payload metadata in real-time responsive to receiving data packets for the session from the network.
17. The network monitoring device of claim 12 , wherein the performance parameters comprise at least one of a packet loss rate, a response time, a number of packets or bytes transmitted, parameters indicative of transmission latency, codec parameters, total transfer time for the session, R-Factor (rating factor), MOS (Mean Opinion Score), a file or page size, a file name, sender/receive information, a total transfer time, a user ID, jitter, and phone number.
18. The network monitoring device of claim 12 , wherein the session tracing engine is further configured to generate mapping information representing at least one other session associated with the session, the mapping information included in the session record.
19. The network monitoring device of claim 18 , wherein the session and the other session are associated with different planes in a same protocol.
20. The network monitoring device of claim 18 wherein the session is associated with a first protocol and the other session is associated with a second protocol different from the first protocol.
21. A non-transitory computer readable storage medium structured to store instructions, the instructions when executed, cause a processor in a network monitoring device to:
receive data packets at a location in a network;
determine a session associated with the received data packets;
generate a common header summarizing information in headers of a plurality of data packets in a session;
generate a single unit of payload metadata summarizing information of payloads of the plurality of the data packets in the session;
determine a protocol associated with the session;
generate performance parameters representing performance of the network for transmitting the plurality of the data packets in the session based on the protocol associated with the session;
generate, for each session, a session record including the common header, the payload metadata and the performance parameters; and
store the session record.