Distributed stream analysis using general purpose processors
View Patent ↗Methods and apparatuses for directing data transferred in a network to multiple analysis processors for network analysis processing. A network processor is configured to receive network data transferred in a network link. The network processor is configured to transmit the network data and a path control signal to a distribution module coupled to the network processor. The distribution module is configured to receive the path control signal and receive the network data from the network processor. The distribution module is further configured to route the network data to multiple outputs based on the path control signal received. A plurality of analysis processors are coupled to the distribution module for receiving the network data from a different output of the distribution module. Each analysis processor analyzes the network data received by the analysis processor for errors, purposes of detecting or measuring errors, performance, security, compliance, statistics, or patterns.
1. A network analyzer for analyzing data transferred in a network, the network analyzer comprising:
a network processor configured to receive network data representing at least a portion of a data stream transferred in a network link, wherein the network processor prepares the network data for analysis and associates an identifier with one or more portions of the network data, wherein the network processor is an integrated unit within which the following are contained:
a parser for extracting network data identification information from the network data;
a transaction look-up-table (LUT) storing identifiers including transaction identification information;
a transaction LUT manager for querying the transaction LUT and assigning a transaction identification (TID) to the network data;
a path LUT for storing path data associating a path with the TID; and
a path manager configured to receive the TID, query the path LUT, and transmit a path control signal to the distribution module based on the results of the path LUT query;
a distribution module that is separate from the network processor and is coupled to the network processor and configured to receive the network data from the network processor, the distribution module configured to route the network data to at least one of a plurality of distribution module outputs based on the identifier of the one or more portions of the network data or based on said path control signal generated by the network processor; and
a plurality of analysis processors coupled to the distribution module, each analysis processor coupled to a different output of the distribution module for receiving the network data from a different output of the distribution module, each analysis processor performing analysis specific portions of the network data, wherein:
the plurality of analysis processors analyze the network data for errors located in at least one of a physical layer, a packet layer, a command layer, an apparatus layer, and a network layer;
the same transactions are distributed to more than one of the plurality of analysis processors, and wherein analysis information is used to distribute the analysis processing burden between the analysis processors analyzing the same portions of the network data, and
each analysis processor receiving the same portions of the network data perform different network analysis tests on the same portions of the network data.
2. The network analyzer of claim 1 , wherein the distribution module is a router chip compliant with SPI4.2.
3. The network analyzer of claim 1 , wherein the distribution module is one of a demultiplexer or a field programmable gate array.
4. The network analyzer of claim 1 , wherein the network processor interleaves the identifier or other control data including a timestamp with the one or more portions of the network data.
5. The network analyzer of claim 1 , further comprising a front-end field programmable gate array connected between the network link and the network processor, the front-end field programmable gate array configured to prepare the network data for the network processor by interleaving timestamp data with the network data.
6. The network analyzer of claim 1 , wherein the one or more portions of the network data are analyzed at real-time speed.
7. The network analyzer of claim 1 , further comprising a plurality of memory buffers connected between each output of the distribution module and the plurality of analysis processors, wherein the network processor receives a memory buffer status signal from each memory buffer and provides the path control signal based on the status signal received from each memory buffer.
8. The network analyzer of claim 1 , wherein at least one of the plurality of analysis processors is coupled to another analysis processor to share information.
9. The network analyzer of claim 1 , wherein each analysis processor conducts at least one of tests to verify the network data header structure, tests to verify header content, tests to verify protocol payload, tests to verify protocol structure, tests to verify primitive handshake data, and tests to verify initialization sequence.
10. The network analyzer of claim 1 , wherein the path control signal includes instructions to the distribution module for routing portions of the network data belonging to the same transaction to specific analysis processors in the plurality of analysis processors.
11. The network analyzer of claim 1 , wherein the network analyzer further comprises:
a graphical display for presenting a result of the analysis to a user; and/or
memory for storing a result of the analysis.
12. A method for analyzing network data in a network analysis system, the method comprising the following acts:
receiving network data representing at least a portion of a data stream transmitted in a network link at a network processor;
extracting identification information from the network data with the network processor;
assigning the network data to a plurality of analysis processors by including an identifier in one or more portions of the network data;
routing by a distribution module the network data to the assigned analysis processor for analysis such that each portion of the network data with the same identifier is routed to the same analysis processor, wherein the same network data is routed to multiple analysis processors for different analysis tests between the plurality of analysis processors, wherein analysis information is used to distribute the analysis processing burden between the analysis processors analyzing the same portions of the network data, wherein the different analysis tests comprise at least one of tests to verify the network data header structure, tests to verify header content, tests to verify protocol payload, tests to verify protocol structure, tests to verify primitive handshake data, and tests to verify initialization sequence; and
at each analysis processor, analyzing the same network data for the different analysis tests, wherein the network processor is an integrated unit within which the following are contained:
a parser for extracting network data identification information from the network data;
a transaction look-up-table (LUT) storing identifiers including transaction identification information;
a transaction LUT manager for querying the transaction LUT and assigning a transaction identification (TID) to the network data;
a path LUT for storing path data associating a path with the TID; and
a path manager configured to receive the TID, query the path LUT, and transmit a path control signal to the distribution module based on the results of the path LUT query; and
wherein the network processor and the distribution module are separate; and
wherein the plurality of analysis processors are coupled to the distribution module, each analysis processor coupled to a different output of the distribution module for receiving the network data from a different output of the distribution module, each analysis processor performing analysis specific portions of the network data,
wherein: the plurality of analysis processors analyze the network data for errors located in at least one of a physical layer, a packet layer, a command layer, an apparatus layer, and a network layer.
13. The method of claim 12 , wherein the analysis of the network data is conducted at real-time speed.
14. The method of claim 12 , further comprising the act of determining that a channel is not functioning properly and routing the network data to a different analysis processor based on this determination.
15. The method of claim 12 , further comprising receiving at least one memory buffer status signal from one or more memory buffers, wherein the network data is routed to the plurality of analysis processors based at least in part on the at least one memory buffer status signal received from the one or more memory buffers.
16. The method of claim 12 , further comprising distributing the processing burden of analyzing the network data between at least two of the plurality of analysis processors.
17. The method of claim 12 , further comprising providing communication between at least two of the plurality of analysis processors so as to coordinate different analysis tests conducted between at the least two analysis processors on the same network data.