As part of a broader organisational restructure, data networking research at Swinburne University of Technology has moved from the Centre for Advanced Internet Architecture (CAIA) to the Internet For Things (I4T) Research Lab.

Although CAIA no longer exists, this website reflects CAIA's activities and outputs between March 2002 and February 2017, and is being maintained as a service to the broader data networking research community.


Code Structure - Overview

Netsniff is based on an object-oriented design and implemented using C++. The underlying PCAP library is used to capture all the network traffic which is then passed on to a class hierarchy for parsing and logging of relevant information.

The hierarchical layout allows a packet header to be processed for information by a single class which then decides whether to pass the enclosed payload to another class for processing. Netsniff works slightly differently with stream based protocols such as TCP Streams. Netsniff uses a TCPStream instance to reconstruct an entire TCP Stream - gathering TCP level statistics in the process - and passing the TCP bit-stream to an application level parser for further processing. Captured packets are processed both at the packet level and at the stream level where appropriate.

Current implementation of packet level processing is indicated in Figure 1, where an arrow indicates what packet types are currently checked for within a particular packet type.The PCAPDev class is not a packet in it its own right, but contains the function called by the pcap library and creates the first instance of the packet class used to process the captured data.

Figure 1. Collaboration diagram of packet classes

Example of parsing an ICMP Packet

An ICMP packet is captured on an Ethernet device:

  1. The PCAPDev class constructs an EtherPacket instance.
  2. The EtherPacket instance processes the Ethernet headers and constructs an IPPacket instance.
  3. The IPPacket instance processes the IP header and constructs an ICMPPacket instance.
  4. The ICMPPacket instance processes the payload of the ICMP Packet.
This class-based design allows for parsing of ICMP packets over a variety of different underlaying protocols as long as their parsers are complete.

Application streams are currently processed using the approach shown in Figure 2, indicating which applications running over a TCP session are currently processed and logged. Any packets that are not processed as part of a supported application are automatically shortened to 68 bytes and written to a log file in tcpdump format.

Figure 2. Stream level protocols

Physical and link-layer protocol implementation

For each physical and link-layer protocol, a class instance is created to handle the protocol under consideration. The instance is created by the class type processing the immediately outlying packet type. Each packet is constructed with a pointer to its encapsulating packet, allowing classes furhter down the protocol tree to backtrack and request information from parent classes (protocols). All information within the packet headers and payload must be processed within the constructor. Once the class is constructed, the memory allocated to store the packet contents is discarded by the pcap library. If the packet and/or header contents are required for other purposes they must be copied or extracted during class construction. The destructor frees any resources allocated in the constructor. For TCPPacket, a ParseStream() method constructs and maintains the TCP streams. The Output() method is called to log all collected information to the provided file handle.

The run-time procedure is for the packet headers and payload to be parsed in the main Packet class and subsequent constructors which store information in their member variables. Once the captured information has been parsed, the Output() method on the main Packet class is automatically called with the correct output stream to output all stored information, the Output() method on any encapsulated packets is also called, so that all processed packet information is output.

Packet-based application layer protocol implementation

Applications that fall into this category perform all their communications at a Packet level rather than at a Stream level. To parse a new application type, first subclass the Packet class and:.
  • All information contained within the packet payload must be processed or copied by the class constructor as the memory allocated to the payload will be freed once the constructor terminates.
  • The DumpFileName() method should be overloaded to return the file name to dump packet information to. An output grammar specifies the data to be
  • The Output() method should be overloaded to output the stored payload information. All data logged by encapsulating packets (eg. IP Layer) will be automatically output prior to the application layer Output() method being called.
  • If necessary, the boolean OmmitOutputFromParent() method allows exclusion of the encapsulating packets output.

Stream-based application layer protocol implementation

Currently only TCP-based stream applications are processed by netsniff. Netsniff will automatically reconstruct the TCP stream and extract TCP level information for logging, passing the reconstructed TCP bitstream to a registered application parser. This allows us to construct a parser that is concerned only with the application layer information. Netsniff currently supports the HTTP, FTP, HTTPS(TLS) POP3, SMTP and IMAP4 application protocols.

Each parser is implemented in a class subclassed from the APPParser class. Port numbers for the application are registered in APPParserFactory. Data from the TCP bitstream is collected a portion at a time, the parser must be able to process this data in blocks as it is provided. The proceedure is:

  • The underlying TCPStream class reconstructs the bitstream in the correct order and passes portions to the parser via the ParseClient() and ParseServer() methods. The parser processes these data blocks for any relevant information and stores it for later output.
  • The parser must honour the global anonymisation flag to anonymise data if required.
  • The boolean Parsed() method should return whether the TCP Stream is parsed or not (allowing non-parsed streams to be logged in the notparsed.dump file.
  • The DumpFileName() method should return the file name to dump all logged information to.
  • The Output() method is overloaded to output the stored information. Prepended to this output (for each individual TCP stream) is the TCP Stream statistics logged while the TCP bitstream was reconstructed.
  • The static Create() method is called by the APPParserFactory to dynamically create an instance of the required parser.

Anonymisation tools implementation

Netsniff can anonymise potentially sensitive information. To allow for correlation between applications, sessions, usernames and hosts, we would like to be able to determine if email addresses, IP addresses and other identifying information has been repeated. To help perform this task, netsniff includes a serie of classes to perform these functions.
  • IP Address Anonymisation - The IPAddressMap class anonymises addresses using a similar algorithm to that used by tcpdcriv running with the -A 50 flag. The class maintains a map of seen IP addresses to anonymised IP adresses so that the same IP address will be mapped to the same anonymised IP address. This algorithm also maintains network locality information between two IP addresses in their anonymised forms.
  • String Hashing - Strings are anonymised using a secure hash with a random key (which is regenerated for each run of netsniff). The output of the string anonymisation is a hex string which uniquely represents the original input string. Comparison of similar input strings is not possible since the hash function has the required property of equally dispersing strings in the input-space into the output-space.
For more details on the anonymisation features of netsniff, please see here.

More Information

For further information on developing modules for the netsniff architecture, please see the netsniff documentation page.
Last Updated: Tuesday 26-Jun-2007 16:06:17 AEST | Maintained by: Jason But ( | Authorised by: Grenville Armitage (