Target-based SMB and DCE/RPC processing for an intrusion detection system or intrusion prevention system

1. A method performed in a processor of an intrusion detection/prevention system (IDS/IPS), for checking for valid packets in a server message block (SMB) named pipe in a communication network, comprising:

receiving, in a processor configured as an IDS/IPS, a packet in a transmission and determining a kind of application of a target of the packet in response to receiving the packet;

including, in the IDS/IPS, data in the packet as part of the SMB named pipe data inspected by the IDS/IPS as part of the SMB named pipe on only one of a condition that:

(a) a file ID (FID) in an SMB command header of the packet is valid (i) for segments/fragments in the SMB named pipe and (ii) for the determined kind of application of the target of the packet, as indicated by a reassembly table, and

(b) the determined kind of application of the target of the packet does not check the FID, as indicated by the reassembly table;

receiving, in the processor configured as an IDS/IPS, a fragment/segment, and determining a kind of application of a target of the fragment/segment in response to receiving the fragment/segment;

separating, in the IDS/IPS, fragments/segments with a same multiplex ID (MID) as part of a same SMB transaction command from fragments/segments with a different MID, the MID being in the SMB frame header, all for fragments/segments in the same SMB named pipe, when a reassembly table indicates that the kind of application of the target separates based on MID;

processing, in the IDS/IPS, the same SMB transaction command with the same MID as being in a separate SMB transaction command instead of with the fragments/segments with the different MID when the kind of application of the target separates based on MID;

receiving, in the processor configured as an IDS/IPS, plural request fragments belonging to a single distributed computing environment/remote procedure call (DCE/RPC) request;

determining, in the IDS/IPS, the kind of application of a target of the DCE/RPC request; selecting, in the IDS/IPS, one of the request fragments as a source of a context ID depending on the target kind of application as indicated in a reassembly table;

selecting, in the IDS/IPS, one of the request fragments as a source of an operation number as indicated in a reassembly table depending on the target kind of application;

reassembling, in the IDS/IPS, the plural request fragments into a reassembled request; and

inserting, in the IDS/IPS, the context ID from the selected request fragment into the context ID of a DCE/RPC header of the reassembled request; and

inserting, in the IDS/IPS, the operation number from a DCE/RPC header of the selected request fragment into the operation number of a DCE/RPC header of the reassembled request.