Scanning server ports to infer service protocols
Systems and methods are disclosed to infer, using a machine learned model, a service protocol of a server based on the banner data produced by the server. In embodiments, the machine learned model is implemented by a network scanner configured to receive banner data from open ports on servers. A received banner is parsed into a set of features, such as the counts or presence of particular characters or strings in the banner. In embodiments, certain types of banner content such as network addresses, hostnames, dates, and times, are replaced with special characters. The machine learned model is applied to the features to infer a most likely protocol of the server port that produced the banner. Advantageously, the model can be trained to perform the inference task with high accuracy and without using human-specified rules, which can be brittle for unconventional banner data and carry undesired biases.
1. A system, comprising:
one or more hardware processors with associated memory that implement a network scanning system, configured to:
send one or more scan requests to one or more ports of the server;
receive a response from a port of the server in response to a scan request, wherein the response includes banner data associated with the port;
use a machine learned model to infer a service protocol supported by the port based on a set of features extracted from the banner data; and
log metadata about the server indicating the service protocol inferred for the port.
2. The system of claim 1 , wherein the network scanning system is configured to scan ports of computers in client networks of a plurality of clients.
3. The system of claim 1 , wherein the network scanning system is configured to scan computers ports that are publicly accessible via the Internet.
4. The system of claim 1 , wherein the network scanning system is configured to:
send one or more subsequent service requests to the port according to the service protocol;
receive one or more service response from the server according to the service protocol; and
include in the metadata additional information about the server determined based on the one or more service responses.
5. The system of claim 4 , wherein
the metadata is used to construct a machine representation of the server, and
the machine representation is used by a vulnerability assessment service to identify security vulnerabilities of the server.
6. The system of claim 1 , wherein the machine learned model is trained to infer a plurality of different service protocols, comprising two or more of:
Post Office Protocol version 3 (POP3),
File Transfer Protocol (FTP),
Telnet protocol,
Secure Shell protocol (SSH),
Internet Message Access Protocol (IMAP),
Network News Transfer Protocol (NNTP),
Simple Mail Transfer Protocol (SMTP),
Hypertext Transfer Protocol (HTTP), and
Hypertext Transfer Protocol Secure (HTTPS).
7. The system of claim 6 , wherein the machine learned model is trained to output an indeterminate result when no service protocol can be inferred from a particular banner data with a sufficient level of confidence.
8. The system of claim 1 , wherein the network scanning system is configured to:
parse the banner into individual text characters; and
extract the set of features from the banner data, including one or more of:
a count of a particular character in the text sequence,
a presence or absence of a particular character in the text sequence,
a count of alphabetical characters in the text sequence, and
a count of numerical characters in the text sequence.
9. The system of claim 8 , wherein the network scanning system is configured to:
determine one or more types of content in the banner data, including one or more of:
a network address,
a hostname,
a date or time, and
a time zone;
replace the one or more types of content with one or more tokens or special characters; and
include in the set of features a count or a presence or absence the one or more tokens or special characters in the banner data.
10. The system of claim 1 , wherein the network scanning system is configured to:
determine that the port has a port number that is different from a standard port number associated with the service protocol; and
include in the metadata an indication that the service protocol is detected on a non-standard port number.
11. The system of claim 1 , wherein the network scanning system is configured to:
determine that a particular service protocol inferred from a particular banner data is incorrect or inferred with low confidence; and
store the particular banner data in a training data store, wherein contents of the training data store is used to compile training datasets for further training of the machine learned model.
12. The system of claim 11 , wherein the machine learning model is periodically retrained using the training datasets store and redeployed.
13. The system of claim 11 , wherein the network scanning system is configured to display the metadata about the server on a graphical user interface, including all open ports detected on the server and service protocols inferred on the open ports.
14. A method comprising:
performing, by one or more hardware processors with associated memory that implement a network scanning system:
sending one or more scan requests to one or more ports of the server;
receiving a response from a port of the server in response to a scan request, wherein the response includes banner data associated with the port;
using a machine learned model to infer a service protocol supported by the port based on a set of features extracted from the banner data; and
logging metadata about the server indicating the service protocol inferred for the port.
15. The method of claim 14 , further comprising the network scanning system:
sending one or more subsequent service requests to the port according to the service protocol;
receiving one or more service response from the server according to the service protocol; and
including in the metadata additional information about the server determined based on the one or more service responses.
16. The method of claim 14 , further comprising the network scanning system:
determining that the port has a port number that is different from a standard port number associated with the service protocol; and
including in the metadata an indication that the service protocol is detected on a non-standard port number.
17. The method of claim 14 , further comprising the network scanning system:
determining that a particular service protocol inferred from a particular banner data is incorrect or inferred with low confidence; and
storing the particular banner data in a training data store, wherein contents of the training data store is used to compile training datasets for further training of the machine learned model.
18. The method of claim 17 , wherein the machine learning model is periodically retrained using the training datasets store and redeployed.
19. The method of claim 14 , further comprising the network scanning system:
displaying the metadata about the server on a graphical user interface, including all open ports detected on the server and service protocols inferred on the open ports.
20. The method of claim 14 , further comprising assessing the metadata to identify one or more security vulnerabilities of the server.