Systems, methods and devices for device fingerprinting and automatic deployment of software in a computing network using a peer-to-peer approach
Disclosed herein are embodiments of methods, devices and systems for device fingerprinting and automatic and dynamic software deployment to one or more endpoints on a computer network. The device fingerprinting systems and devices herein are configured to operate with limited data without sitting between network devices and the internet, without monitoring all network traffic, and without limited or no active scanning. The embodiments herein may passively collect information as distributed peers and may perform very limited active scans. In some embodiments, the information is used as an input to a custom hierarchical learning model to fingerprint devices on a network by identifying attributes of the devices such as the operating system family, operating system version, and device role. In some embodiments, a dynamic deployer selection process may be utilized to simply and efficiently deploy software. Some embodiments herein involve end-to-end encryption of credentials in a deployment process.
1. A system for device fingerprinting, the system comprising:
one or more computer readable storage devices configured to store a plurality of computer executable instructions; and
one or more hardware computer processors in communication with the one or more computer readable storage devices and configured to execute the plurality of computer executable instructions in order to cause the system to:
collect, via one or more distributed software agents on one or more endpoint devices of a computer network, endpoint device data;
analyze, by the one or more distributed software agents or a central server, the endpoint device data to determine at least one of an operating system family and an operating system type of at least one of the one or more endpoint devices, wherein analyzing the endpoint device data comprises:
inputting the endpoint device data into a hierarchical machine learning model comprising:
a first layer, the first layer configured to classify each of the one or more endpoint devices into an operating system family classification; and
a second layer, the second layer configured to classify each of the one or more endpoint devices of a classified operating system family into an operating system type classification;
aggregate, by the one or more distributed software agents or the central server, the operating system family classification and the operating system type classification for each of the one or more endpoint devices to generate a device fingerprint for the respective endpoint device.
2. The system of claim 1 , wherein the endpoint device data comprises both passive broadcast data and active unicast data.
3. The system of claim 2 , wherein the passive broadcast data comprises broadcast protocol data including one or more of address resolution protocol (ARP), dynamic host configuration protocol (DHCP), server message block (SMB), simple service discovery protocol (SSDP), multicast domain Name system (mDNS), or simple network management protocol (SNMP).
4. The system of claim 2 , wherein the active unicast data comprises data obtained by scanning, by the one or more distributed software agents, open ports including one or more of secure shell (SSH), Telnet, hypertext transfer protocol (HTTP), hypertext transfer protocol secure (HTTPS), and remote desktop protocol (RDP).
5. The system of claim 1 , wherein the one or more hardware computer processors are further configured to execute the plurality of computer executable instructions in order to cause the system to apply, by the one or more distributed software agents, one or more security protocols to the one or more endpoint devices based at least in part on the determined operating system family or operating system type.
6. The system of claim 1 , wherein the one or more hardware computer processors are further configured to execute the plurality of computer executable instructions in order to cause the system to apply, by the one or more distributed software agents or the central server, one or more labels to the endpoint device data based on one or more manual or automatically generated labeling rules prior to analyzing the endpoint device data.
7. The system of claim 1 , wherein the one or more distributed software agents continuously collect the endpoint data or the network traffic data from the one or more endpoint devices.
8. The system of claim 1 , wherein the one or more hardware computer processors are further configured to execute the plurality of computer executable instructions in order to cause the system to access, by the one or more distributed software agents or the central server, one or more databases to collect additional endpoint device data comprising one or more of device media access control (MAC) addresses, operating system versions, device manufacturers, and granular physical device identifications.
9. The system of claim 1 , wherein the first and second layers of the hierarchical machine learning model each comprise one or more machine learning sub-models that comprise XGBoost or another decision tree ensemble.
10. The system of claim 1 , wherein the first and second layers of the hierarchical machine learning model are part of a fully connected neural network.
11. The system of claim 10 , wherein the output of the neural network comprises a numerical result indicating the probability of an endpoint device having a specific OS family for the first layer, and the probability of an endpoint device having a specific OS type for the second layer, and wherein if the probability is above a predetermined threshold, the layer classification is generated.
12. A computer implemented method for device fingerprinting, the method comprising:
collecting, via one or more distributed software agents on one or more endpoint devices of a computer network, endpoint device data;
analyzing, by the one or more distributed software agents or a central server, the endpoint device data to determine at least one of an operating system family and an operating system type of at least one of the one or more endpoint devices, wherein analyzing the endpoint device data comprises:
inputting the endpoint device data into a hierarchical machine learning model comprising:
a first layer, the first layer configured to classify each of the one or more endpoint devices into an operating system family classification; and
a second layer, the second layer configured to classify each of the one or more endpoint devices of a classified operating system family into an operating system type classification;
aggregating, by the one or more distributed software agents or the central server, the operating system family classification and the operating system type classification for each of the one or more endpoint devices to generate a device fingerprint for the respective endpoint device.
13. The method of claim 12 , wherein the endpoint device data comprises both passive broadcast data and active unicast data.
14. The method of claim 13 , wherein the passive broadcast data comprises broadcast protocol data including one or more of address resolution protocol (ARP), dynamic host configuration protocol (DHCP), server message block (SMB), simple service discovery protocol (SSDP), multicast domain Name system (mDNS), or simple network management protocol (SNMP).
15. The method of claim 13 , wherein the active unicast data comprises data obtained by scanning, by the one or more distributed software agents, open ports including one or more of secure shell (SSH), Telnet, hypertext transfer protocol (HTTP), hypertext transfer protocol secure (HTTPS), and remote desktop protocol (RDP).
16. The method of claim 12 , wherein the one or more hardware computer processors are further configured to execute the plurality of computer executable instructions in order to cause the system to apply, by the one or more distributed software agents, one or more security protocols to the one or more endpoint devices based at least in part on the determined operating system family or operating system type.
17. The method of claim 12 , wherein the one or more hardware computer processors are further configured to execute the plurality of computer executable instructions in order to cause the system to apply, by the one or more distributed software agents or the central server, one or more labels to the endpoint device data based on one or more manual or automatically generated labeling rules prior to analyzing the endpoint device data.
18. The method of claim 12 , wherein the one or more distributed software agents continuously collect the endpoint data or the network traffic data from the one or more endpoint devices.
19. The method of claim 12 , wherein the one or more hardware computer processors are further configured to execute the plurality of computer executable instructions in order to cause the system to access, by the one or more distributed software agents or the central server, one or more databases to collect additional endpoint device data comprising one or more of device media access control (MAC) addresses, operating system versions, device manufacturers, and granular physical device identifications.
20. The method of claim 12 , wherein the first and second layers of the hierarchical machine learning model each comprise one or more machine learning sub-models that comprise XGBoost or another decision tree ensemble.
21. The method of claim 12 , wherein the first and second layers of the hierarchical machine learning model are part of a fully connected neural network.
22. The method of claim 21 , wherein the output of the neural network comprises a numerical result indicating the probability of an endpoint device having a specific OS family for the first layer, and the probability of an endpoint device having a specific OS type for the second layer, and wherein if the probability is above a predetermined threshold, the layer classification is generated.