Detecting NSTP Attacks

Submission Deadline:

The goals of this assignment are to:

  1. Implement a simple signature-based network intrusion detection system
  2. Gain experience in the principles of signature-based intrusion detection
  3. Understand the engineering trade-offs involved in performing intrusion detection

NSTP Intrusion Detection

In this assignment, you will be implementing an inline intrusion detection system (IDS) for an NSTPv2 server implementation. This NSTP server has been found to be vulnerable to several attacks, for which you will need to develop appropriate signatures.

Your IDS will be provided with a UNIX socket that will serve as an event bus. Events include network-level events such as connection establishment or termination. They also include every NSTP message issued or received by the server along with metadata identifying the remote client and session information.

Version 2 of NSTP has not changed in terms of semantics from version 1. However, the wire format has been migrated to protocol buffers, or protobufs. NSTPv2 protobufs are available here. Note that we are only using protobuf messages, not gRPC!

IDS messages are also protobufs, which are defined along with the main protocol messages. In the following, we describe these messages in more detail. However, note that each NSTP and IDS message is a length-delimited protobuf message. That is, each protocol and IDS message is implicitly prefixed by a u16 integer that holds the serialized length of the following message.


// NSTP IDS event message
message IDSEvent {
    // Event identifier
    uint64 event_id = 1;
    // Seconds since UNIX epoch
    uint64 timestamp = 2;
    // Address family: AF_INET or AF_INET6
    uint32 address_family = 3;
    // Server IPv4 or IPv6 address
    bytes server_address = 4;
    // Server port
    uint32 server_port = 5;
    // Client IPv4 or IPv6 address
    bytes remote_address = 6;
    // Remote port
    uint32 remote_port = 7;

    // Actual event, either a network-level event (e.g., connection establishment or
    // termination) or an NSTP message.
    oneof event {
        ConnectionEstablished connection_established = 8;
        ConnectionTerminated connection_terminated = 9;
        ClientHello client_hello = 10;
        ServerHello server_hello = 11;
        ErrorMessage error_message = 12;
        PingRequest ping_request = 13;
        PingResponse ping_response = 14;
        LoadRequest load_request = 15;
        LoadResponse load_response = 16;
        StoreRequest store_request = 17;
        StoreResponse store_response = 18;

    // Direction flag
    bool client_to_server = 19;

IDS events are expressed in terms of the above protobuf. Each event carries metadata such as an event identifier, timestamp, address family (AF_INET or AF_INET6), and addresses and ports that uniquely describe – at any single point in time – a valid NSTP session. This metadata is followed by a single network- or protocol-level message.


// IDS decision message
message IDSDecision {
    // Event identifier
    uint64 event_id = 1;
    // Decision
    bool allow = 2;

As an inline IDS, your detector must respond to each event that is received from the server, explicitly allowing or denying it according to your detection logic. This decision is expressed as an IDSDecision protobuf, shown above. This message simply consists of the event identifier you are responding to along with a boolean value corresponding to whether the event is allowed or not.


// IDS connection termination message
message IDSTerminateConnection {
    // Address family: AF_INET or AF_INET6
    uint32 address_family = 1;
    // Server IPv4 or IPv6 address
    bytes server_address = 2;
    // Server port
    uint32 server_port = 3;
    // Client IPv4 or IPv6 address
    bytes remote_address = 4;
    // Remote port
    uint32 remote_port = 5;

The IDS can also issue asynchronous connection termination messages to the server via the above protobuf. Here, the IDS must explicitly identify the connection tuple representing the connection to terminate.

Security Advisories

The following describes the three specific security vulnerabilities that your IDS will need to detect.


The NSTPv2 server is vulnerable to a path traversal vulnerability that allows an attacker to access arbitrary paths on the server filesystem. The reference implementation uses the host filesystem as a backing store for the key-value database, where keys are simply filesystem paths intended to be members of a subtree rooted at a configurable server document root. However, due to insufficient sanitization, keys can include ‘/’ and ‘..’ character sequences that allow an attacker to reference paths outside of the document root.


The NSTPv2 server is vulnerable to a buffer overflow in its handling of store requests. In particular, keys that are longer than 512 bytes in length will overflow a heap-allocated buffer. This overflow allows an attacker to control heap metadata for neighboring memory chunks, leading to potential control-flow hijacking exploitation.


The NSTPv2 server is vulnerable to a denial-of-service attack. In particular, it is by default limited to a concurrent maximum of 1024 open file descriptors at any single point in time. This limit includes open sockets. An attacker can deny service to legitimate clients by creating too many open connections, preventing new connections from being established to the server.

Evaluation Criteria

Your IDS must consist of a standalone program that will take as an argument a path to the IDS event bus socket. It will be evaluated on both its true and false positive rates on a labeled test set of benign and malicious NSTP traffic.

Malicious traffic is defined as events that violate the NSTPv2 specification as well as attacks as described in the above security advisories.

Clients that are deemed malicious must not be allowed future interaction with the NSTP server.

An image containing a partial test suite can be pulled from Running this image with a socket path as an argument will execute the test suite, which will forward IDS messages on that socket and expect decision messages in response. The test suite will report whether the decisions issued by your IDS were correct or not according to a ground truth configuration.

Since the socket will need to be shared between the host and the test suite, you will need to mount a host directory in the running container. For instance, to share an existing socket located at /tmp/nstp_ids.socket you could run the test suite like so:

$ docker run -it --rm -v /tmp:/tmp /tmp/nstp_ids.socket

Note that the socket will be owned by the user that the test suite runs as, which is root by default.

Your submission must contain a Dockerfile in the repository root that builds a container containing your IDS when the following sequence of commands is performed:

$ git clone ${your_repo_url} submission
$ cd submission
$ docker build -t submission .

Your IDS should be invoked by default when running the resulting container image (use the ENTRYPOINT Dockerfile directive). Grading of your solution will take place using an automated test suite that will expect your IDS to use a provided event socket like so:

$ docker run -d -v ${socket_dir}:${socket_dir} submission ${socket_dir}/${socket_name}

Submission Instructions

Push the source code for your server to a git repository on the project server at ~/assignments/01-intrusion_detection.

# On the project server
$ mkdir -p ~/assignments/01-intrusion_detection
$ cd ~/assignments/01-intrusion_detection
$ git init --bare

# On your development machine
$ cd ${repo_path}
$ git remote add submission ${user}
$ git push -u submission --all
$ git push -u submission --tags

You may tag the commit you wish to be considered for grading with the tag solution; otherwise, the latest commit on master will be treated as such.