Traffic deduplication in a visibility network

1. An apparatus, comprising:

a traffic discriminator module;

a session management module;

a data traffic rule lookup module;

a data deduplication module;

at least one processor; and

a memory stored with program code stored thereon, wherein, when the program code is executed, the at least one processor is configured to:

determine, by the traffic discriminator module, whether a packet is a control packet or a data packet;

receive, at the session management module from the traffic discriminator module, the packet in response to determining the packet to be a control packet;

receive, at the data traffic rule lookup module from the traffic discriminator module, the packet in response to determining the packet to be a data packet;

extract, at the session management module, control information from the control packet, wherein the control information is used to identify a session associated with the control packet;

determine, at the data traffic rule lookup module, a forwarding rule based on the control information, wherein the forwarding rule comprises egress port information;

receive, at the data deduplication module from the session management module, the packet;

determine, at the data deduplication module, whether the packet received from the session management module is a duplicate of a previously-received packet;

in response to the packet being a non-duplicate, determine whether the packet corresponds to the identified session based on the control information; and

in response to determining that the packet corresponds to the identified session, transmit the packet over an egress port based on the egress port information.

2. The apparatus of claim 1 , wherein, when the program code is executed, the at least one processor is further configured to:

identify a traffic flow associated with the control information; and

store the control information and the identified session in the memory.

3. The apparatus of claim 1 , wherein, when the program code is executed, the at least one processor is further configured to:

determine the egress port based on a mapping between the egress port and the control information; and

generate the forwarding rule to forward data packets associated with the identified session to the egress port.

4. The apparatus of claim 1 , wherein, when the program code is executed, the at least one processor is further configured to:

calculate a first cyclic redundancy check (CRC) value of the data packet;

calculate a second CRC value of a new data packet;

compare the first and the second CRC values;

in response to the first and the second CRC values being the same, compare a header parameter of the data packet with a header parameter of the new data packet; and

in response to the header parameter of the data packet matching the header parameter of the new data packet, drop the data packet or the new data packet.

5. A method, comprising:

determining, by a traffic discriminator module of a packet broker, whether a packet is a control packet or a data packet;

receiving, at a session management module of the packet broker from the traffic discriminator module, the packet in response to determining the packet to be a control packet;

receiving, at a data traffic rule lookup module of the packet broker from the traffic discriminator module, the packet in response to determining the packet to be a data packet;

extracting, at the session management module, control information from the control packet, wherein the control information is used to identify a session associated with the control packet;

determining, at the data traffic rule lookup module, a forwarding rule based on the control information in the control packet;

receiving, at a data deduplication module of the packet broker from the session management module, the packet;

determining, at the data deduplication module, whether the packet is a duplicate of a previously-received packet;

in response to the packet being a non-duplicate, determining, at the data deduplication module, whether the packet corresponds to the identified session based on the control information; and

in response to determining that the packet corresponds to the identified session, tagging, at the packet broker, the packet with egress port information corresponding to the forwarding rule to transmit over an egress port based on the egress port information.

6. The method of claim 5 , further comprising:

identifying a traffic flow associated with the control information, wherein the control information each comprises one or more of an international mobile subscriber identity (IMSI), an access point name (APN), a tunnel endpoint identifier (TEID), and a user equipment (UE) Internet Protocol (IP) address.

7. The method of claim 5 , further comprising:

determining the egress port based on a mapping between the egress port and the control information; and

generating the forwarding rule to forward information associated with the identified session to the egress port.

8. The method of claim 5 , further comprising:

calculating a first cyclic redundancy check (CRC) value of the data packet;

calculating a second CRC value of a new data packet;

comparing the first CRC value to the second CRC value;

in response to the first CRC value and the second CRC value being the same, comparing a header parameter of the data packet to a header parameter of the new data packet; and

in response to the header parameter of the data packet matching the header parameter of the new data packet, dropping the data packet or the new data packet.

9. A system, comprising:

a packet broker configured to transmit a data packet;

a session management device configured to extract control information from the data packet;

a data traffic rule lookup device configured to determine a forwarding rule based on the extracted control information;

a traffic discriminator device configured to:

receive the data packet from the packet broker;

identify information in the data packet;

in response to the identified information being control information, forward the data packet to the session management device; and

in response to the identified information being data information, forward the data packet to the data traffic rule lookup device;

a data deduplication device configured to:

receive the data packet from the session management device;

determine whether the data packet is a duplicate of a previously-received data packet; and

in response to the data packet being a non-duplicate, forward the data packet; and

a transmitter device comprising an egress port, where the transmitter device is configured to:

receive the data packet from the data deduplication device;

tag the data packet with egress port information; and

transmit the data packet over the egress port.

10. The system of claim 9 , wherein the session management device is further configured to:

identify a traffic flow associated with the extracted control information; and

store the extracted control information in a memory.

11. The system of claim 9 , wherein the extracted control information comprises one or more of an international mobile subscriber identity (IMSI), an access point name (APN), a tunnel endpoint identifier (TED), and a user equipment (UE) Internet Protocol (IP) address.

12. The system of claim 9 , wherein the session management device is further configured to:

determine the egress port based on a mapping between the egress port and the extracted control information; and

generate the forwarding rule to forward information associated with the data packet to the egress port.

13. The system of claim 9 , wherein the data traffic rule lookup device is further configured to:

receive a new data packet from the traffic discriminator device;

compare control information in the new data packet to the extracted control information to determine the forwarding rule; and

forward the new data packet to the egress port.

14. The system of claim 9 , further comprising:

a configuration management device configured to accept configuration information comprising:

inputs to generate a load balance table; and

a policy to filter or forward the data packet.

15. The system of claim 9 , further comprising a traffic filtering device configured to:

determine at least one policy to filter or forward the data packet based on a criterion comprising one of an international mobile subscriber identity (IMSI), an access point name (APN), a tunnel endpoint identifier (TEID), location of a user equipment (UE), radio access technology (RAT), and a type of the UE.

16. The system of claim 9 , wherein the data deduplication device is further configured to:

calculate a first cyclic redundancy check (CRC) value of the data packet;

calculate a second CRC value of a new data packet;

compare the first CRC value to the second CRC value;

in response to the first CRC value and the second CRC value being the same, compare a header parameter of the data packet to a header parameter of the new data packet; and

in response to the header parameter of the data packet matching the header parameter of the new data packet, drop the data packet or the new data packet.

17. The system of claim 16 , wherein the deduplication device further is configured to:

in response to determining the header parameter of the data packet is different from the header parameter of the new data packet, forward the data packet and the new data packet to the egress port.