System and method for autonomously fingerprinting and enumerating internet of thing (IoT) devices based on nated IPFIX and DNS traffic
This document describes a system and method for detecting the presence of Internet of Things (IoTs) from network traffic that has undergone a Network Address Translation (NAT) process, i.e., NATed network traffic, regardless of whether the network traffic comprises IP Flow Information Export (IPFIX) type of traffic or Domain Name System (DNS) type of traffic. Such a capability is crucial as the adoption rate of IoTs have increased exponentially over the past few years. In order to protect IoTs from cyber-attacks, one would first have to understand what type of IoTs are being used, and how many/how widely used these IoTs are. Once the IoT landscape has been defined, cyber defenders may then dedicate resources to identify and subsequently address vulnerabilities that may be in these IoTs.
1. A system for autonomously fingerprinting and enumerating Internet of Thing (IoT) devices based on Network Address Translated (NAT-ed) network traffic of the IoT devices that has been collected over a time period, the system comprising:
memory storing instructions; and
one or more processors coupled to the memory and configured to process the stored instructions to:
retrieve Internet protocol (IP) Flow Information Export (IPFIX) records from the NAT-ed network traffic;
group the IPFIX records into groups according to collection time, source IP address, and destination IP address;
order the IPFIX records in each of the groups chronologically according to collection times of the IPFIX records;
generate a plurality of IPFIX data samples, whereby each IPFIX data sample is generated from one of the groups of the chronologically ordered IPFIX records, and whereby each IPFIX data sample comprises a plurality of discrete time series comprising a protocol, an octet, a reverse octet, a source port, a destination port, a packet and a reverse packet associated with IPFIX records having the collection times that fall within a particular time bin of the plurality of discrete time series;
determine, using a first head of a trained two-headed neural network, identities of the IoT devices that generated the NAT-ed network traffic by carrying out per device tempered binary classification on the plurality of IPFIX data samples; and
enumerate, using a second head of the trained two-headed neural network, each of the IoT devices identified by the first head of the trained two-headed neural network,
whereby the first head of the trained two-headed neural network is trained using a loss model that treats labels from a training dataset as noisy labels and the second head of the trained two-headed neural network is trained using a mean squared error (MSE) loss function, and
whereby the training dataset used to train the trained two-headed neural network comprises a plurality of IPFIX training data samples generated based on unNAT-ed network traffic and comprises a plurality of the IPFIX training data samples that have undergone a data augmentation process.
2. The system according to claim 1 , whereby the loss model that treats the labels from the training dataset as the noisy labels comprises a bi-tempered cross entropy loss function that is parameterised by a temperature term, t 1 , wherein the bi-tempered cross entropy loss function is defined as:
∑
-
(
i
-
1
)
⋀
k
≡
(
log_
(
t_
1
)
-
(
log_
(
t_
1
)
)
-
1
/
(
2
-
t_
1
)
(
y_i
⋀
(
2
-
t_
1
)
-
y
⋀
_i
⋀
(
2
-
t_
1
)
)
)
.
3. The system according to claim 1 , whereby the loss model that treats the labels from the training dataset as the noisy labels comprises a bi-tempered softmax function that is parameterised by a temperature term, t 2 , wherein the bi-tempered softmax function is defined as:
ŷ i =exp t 2 ( â i −λ t 2 ( â )), where λ t 2 ( â )∈ is s.t. Σ j=1 k exp t 2 ( â j −λ t 2 ( â ))=1.
4. The system according to claim 1 , wherein the one or more processors are further configured to process the stored instructions to generate a protocol for the particular time bin of an IPFIX data sample by:
randomly selecting a protocol of an IPFIX record that has a collection time that falls within the particular time bin.
5. The system according to claim 1 wherein the one or more processors are further configured to process the stored instructions to generate the reverse octet for the particular time bin of an IPFIX data sample by:
summing all octets of the IPFIX records that have the collection times that fall within the particular time bin.
6. The system according to claim 1 wherein the one or more processors are further configured to process the stored instructions to generate the reverse octet for the particular time bin of an IPFIX data sample by:
summing all reverse octets of the IPFIX records that have the collection times that fall within the particular time bin.
7. The system according to claim 1 wherein the one or more processors are further configured to process the stored instructions to generate the source port for the particular time bin of an IPFIX data sample by:
randomly selecting a source port of an IPFIX record that has a collection time that falls within the particular time bin.
8. The system according to claim 1 wherein the one or more processors are further configured to process the stored instructions to generate the destination port for the particular time bin of an IPFIX data sample by:
randomly selecting a destination port of an IPFIX record that has a collection time that falls within the particular time bin.
9. The system according to claim 1 wherein the one or more processors are further configured to process the stored instructions to generate the packet for the particular time bin of an IPFIX data sample by:
summing all packets of the IPFIX records that have the collection times that fall within the particular time bin.
10. The system according to claim 1 wherein the one or more processors are further configured to process the stored instructions to generate the reverse packet for the particular time bin of an IPFIX data sample by:
summing all reverse packets of the IPFIX records that have the collection times that fall within the time particular bin.
11. The system according to claim 1 , wherein the data augmentation process comprises:
for each destination IP address associated with an IPFIX training data sample:
selecting IPFIX training data samples from the plurality of IPFIX training data samples that have similar destination IP addresses;
applying random circular time shifts to the time bins of the selected IPFIX training data samples;
merging and reordering the time-shifted time bins to form time-shifted IPFIX training data samples;
merging all the time-shifted IPFIX training data samples into a consolidated data sample whereby the consolidated data sample forms part of the training dataset.
12. The system according to claim 11 wherein the one or more processors are further configured to process the stored instructions to:
randomly select the time bins from the consolidated data sample; and
randomly perturb values in the selected time bins.
13. The system according to claim 11 wherein the one or more processors are further configured to process the stored instructions to:
randomly select the time bins from the consolidated data sample; and
randomly introduce random noise to values in the selected time bins.
14. The system according to claim 1 , wherein the labels of the training dataset comprise identification labels and enumeration labels generated during the data augmentation process.