High speed packet capture
View Patent ↗Packets can be read from a network interface into an application using a single kernel copy. In one embodiment, the invention includes a receiver packet memory to store captured packets, and a network interface driver operating in a kernel of a device to read packets captured by network interface hardware into the kernel by storing captured packets in the receiver packet memory. Then, an application interface can expose the receiver packet memory to an application executing on the device by representing the receiver packet memory as a virtual file.
1. A method for reading data into an application, the method comprising:
reading a packet of data captured by a network interface coupled to a network interface driver that provides a communicative interface between the network interface and a kernel of a capture system;
storing the packet in a receiver packet memory in a wraparound fashion such that the packet is placed contiguously next to a last new packet received, and wherein the packet and the last new packet are stored as read only data in the receiver packet memory such that multiple applications can access the packets, wherein the network interface and the network interface driver are configured on a kernel side of an operating system provisioned within the capture system, and wherein the applications reside on an application side of the capture system that is different from the kernel side;
updating a head indicator, the head indicator identifying the last new packet in the receiver packet memory; and
representing the receiver packet memory as a virtual file, wherein a first application of the multiple applications is configured to migrate packets from the receiver packet memory by memory mapping the virtual file, the first application employing file system commands and handles to reference and to manipulate selected data in the receiver packet memory such that the virtual file is mapped in a read only mode to allow a second application of the multiple applications to access the selected data in the receiver packet memory, and wherein the capture system includes a control memory that provides a plurality of pointers to the receiver packet memory in order to allow access to the control memory by both the network interface driver and the first and second applications through an application interface.
2. The method of claim 1 , further comprising reading new packets from the memory-mapped virtual file by the first application.
3. The method of claim 2 , wherein reading the new packets comprises reading the new packets from a memory location indicated by a tail indicator, the tail indicator identifying the last packet read by the first application.
4. The method of claim 3 , wherein reading the new packets comprises reading the new packets from memory locations between the tail indicator and the head indicator.
5. The method of claim 3 , further comprising updating the tail indicator to account for the new packets read by the first application.
6. The method of claim 3 , wherein the head indicator comprises a head pointer into the receiver packet memory and the tail indicator comprises a tail pointer into the receiver packet memory.
7. A capture device comprising:
a receiver packet memory to store captured packets that include data;
a network interface driver operating in a kernel of the capture device to read packets captured by a network interface module into the kernel by storing captured packets in the receiver packet memory in a wraparound fashion such that the packet is placed contiguously next to a last new packet received, and wherein the packet and the last new packet are stored as read only data in the receiver packet memory such that multiple applications can access the packets; and
an application interface configured to expose the receiver packet memory to a first application of the multiple applications executing on the capture device by representing the receiver packet memory as a virtual file, wherein the network interface module and the network interface driver are configured on a kernel side of an operating system provisioned within the capture device, and wherein the applications reside on an application side of the capture device that is different from the kernel side, and wherein the first application is configured to migrate packets from the receiver packet memory by memory mapping the virtual file, the first application employing file system commands and handles to reference and to manipulate selected data in the receiver packet memory such that the virtual file is mapped in a read only mode to allow a second application of the multiple applications to access the selected data in the receiver packet memory, and wherein the capture device includes a control memory that provides a plurality of pointers to the receiver packet memory in order to allow access to the control memory by both the network interface driver and the first and second applications through the application interface.
8. The capture device of claim 7 , further comprising a control memory containing a head indicator and a tail indicator.
9. The capture device of claim 8 , wherein the head indicator comprises a head pointer pointing to the last packet written to the receiver packet memory by the network interface driver, and the tail indicator comprises a tail pointer pointing to the last packet processed by the first application.
10. The capture device of claim 9 , wherein the first application migrates packets from the kernel to the first application by performing memory read operations on the memory-mapped virtual file.
11. The capture device of claim 10 , wherein the first application performs read operations by reading packets between the head pointer and the tail pointer.
12. The capture device of claim 11 , wherein the first application updates the tail pointer after each packet migration.
13. The capture device of claim 7 , wherein the network interface module comprises a network interface controller (NIC).
14. A non-transitory machine-readable medium having stored thereon data representing instructions that, when executed by a processor running an operating system kernel and at least one application, cause the processor to perform operations comprising:
reading a packet of data captured by a network interface by coupled to a network interface driver that provides a communicative interface between the network interface and a kernel of a capture system,
the packet of data being part of a document that includes a plurality of objects that identify characteristics of the document,
wherein the document is captured based on a capture rule that specifies the objects, and
wherein the capture rule designates whether to discard or to store the objects of the document, and
wherein the capture rule is part of a default rule sot for a capture system configured to monitor network traffic and capture the document;
storing the packet in a receiver packet memory in a wraparound fashion such that the packet is placed contiguously next to a last new packet received, and
wherein the packet and the last new packet are stored as read only data in the receiver packet memory such that multiple applications can access the packets,
wherein the network interface and the network interface driver are configured on a kernel side of an operating system provisioned within the capture system, and
wherein the applications reside on an application side of the capture system that is different from the kernel side;
updating a head indicator, the head indicator identifying the last new packet in the receiver packet memory; and
exposing the receiver packet memory to the application by representing the receiver packet memory as a virtual file
wherein a first application of the multiple applications is configured to migrate packets from the receiver packet memory by memory mapping the virtual file,
the first application employing file system commands and handles to reference and to manipulate selected data in the receiver packet memory such that the virtual file is mapped in a read only mode to allow a second application of the multiple applications to access the selected data in the receiver packet memory, and
wherein the capture system includes a control memory that provides a plurality of pointers to the receiver packet memory in order to allow access to the control memory by both the network interface driver and the first and second applications through an application interface.
15. The machine-readable medium of claim 14 , wherein the instructions further cause the processor have the first application to read new packets from the memory-mapped virtual file.
16. The machine-readable medium of claim 15 , wherein reading the new packets comprises reading the new packets from a memory location indicated by a tail indicator, the tail indicator identifying the last packet read by the first application.
17. The machine-readable medium of claim 16 , wherein reading the new packets comprises reading the new packets from memory locations between the tail indicator and the head indicator.
18. The machine-readable medium of claim 16 , wherein the instructions further cause the processor to have the first application update the tail indicator to account for the new packets read by the first application.