Live Traffic (nDAG)

Overview

Definition

In addition to post-processing archived pcap files, STARDUST users can also perform real-time analysis against a live feed of the traffic observed at the UCSD network telescope. This feed is known as the “nDAG stream”. DAG refers to the DAG hardware capture card that is used to capture the telescope traffic and the “n” stands for “network” (to reflect that we are taking a DAG capture and exporting it directly to multiple users via a network).

The nDAG methodology and protocol were invented specifically for the STARDUST project.

There are two streams of packets: raw packets and tagged packets. Raw packets are the original data and are formatted exactly like pcap data, while tagged packets contain more information through processing.

Tagged Packets

Tagged packets differ from raw packets because they are processed by corsarotagger which adds additional metadata called tags. Part of this processing includes prepending the results to the captured packets as “tags.”

There are four categories of tags:

Standard

Standard tags are applied to all captured packets and include the following entries:

• Source port (or ICMP type for ICMP packets)
• Destination port (or ICMP code for ICMP packets)
• Transport protocol
• Flow hash value: a 32 bit hash of the fields in the packet that define which flowtuple the packet belongs to.
• A bitmask showing which built-in filters were matched by the packet

To determine if a tagged packet matches one of the built-in filters:

uint64_t fbits = bswap_be_to_host64(tags->filterbits);
if (fbits & (1 << CORSARO_FILTERID_SPOOFED)) {
   // this packet is spoofed
}

and replace SPOOFED with the ID of the filter that you are interested in (e.g. ERRATIC, ROUTED, etc.).

Prefix2ASN

Prefix2ASN maps the source IP address to its autonomous system number (ASN).

Maxmind

Maxmind looks up the source IP address in the Maxmind geo-location dataset and tags the packet with the country and continent that the address belongs to.

NetAcq-Edge

NetAcq-Edge looks up the source IP address in the NetAcuity Edge geo-location dataset and tags the packet with the country, continent, and polygon for that IP address. Note that not all users will receive access to the Netacuity tags due to licensing limitations.

User Guide

Libtrace has built-in support for receiving packets via nDAG (as of version 4.0.2), so any libtrace programs or tools can use nDAG as an input source. To access the data, libtrace needs a pointer (like a URI) to the data source.

Accessing the UCSD Telescope Stream

Limbo VMs

To access the UCSD telescope stream on a limbo VM, use the URI: ndag:ens4,225.44.0.1,44000.

You may need to replace ens4 with the name of the interface that is on the 10.224.0.0/16 network.

Find the Interface

If you do not know the name of the interface that is on the 10.224.0.0/16 network, you can find this interface name by running:

user@vm001:~$ ip a | grep 10.224 | awk '{print $(NF)}'
ens3

STARDUST containers

To access the UCSD telescope stream within a container, use the URI: ndag:eth1,225.44.0.1,44000.

Testing

tracepktdump is a good tool for testing whether your nDAG URI is correct.

Example

user@vm001:~$ tracepktdump -c 5 ndag:eth1,225.44.0.1,44000

should result in 5 packets immediately being decoded and dumped to your terminal.

Example

user@vm001:~$ tracepktdump -c 5 ndag:ens3,225.44.0.1,44000
Added new stream 225.44.0.1:32599 to thread 0
Added new stream 225.44.0.1:32601 to thread 0
Added new stream 225.44.0.1:32603 to thread 0
Added new stream 225.44.0.1:32605 to thread 0
Added new stream 225.44.0.1:32607 to thread 0
Added new stream 225.44.0.1:32609 to thread 0
Added new stream 225.44.0.1:32611 to thread 0
Added new stream 225.44.0.1:32613 to thread 0

Tue Nov  3 18:28:19 2020
 Capture: Packet Length: 64/64 Direction Value: 0
 Ethernet: Dest: 3c:fd:fe:19:d8:00 Source: 00:de:fb:ba:06:c7 Ethertype: 0x0800
 IP: Header Len 20 Ver 4 DSCP 00 ECN 0 Total Length 40
 IP: Id 9721 Fragoff 0
 IP: TTL 238 Proto 6 (tcp) Checksum 64197
 IP: Source 193.27.229.86 Destination 44.174.216.240
 TCP: Source 46359 Dest 8001
 TCP: Seq 2092491336
 TCP: Ack 0
 TCP: DOFF 5 Flags: SYN Window 1024
 TCP: Checksum 52343 Urgent 0
unknown protocol tcp/8001
 Unknown Protocol: 8001
  00 00 48 f7 8d ad 7a 1d 4b 34                      ..H...z.K4                  

...

Parallelism

The UCSD telescope nDAG stream is delivered as eight parallel streams. Packets within each stream are guaranteed to be in chronological order, but we can make no guarantees about the relative ordering of packets between different streams.

For that reason, we strongly recommend that any libtrace analysis programs that are used with the nDAG feed are pre-configured to run eight processing threads of their own (one for each of the nDAG streams). Less threads will result in multiple streams being processed by the same thread and therefore the chronological order of packets cannot be guaranteed for each thread.

If you are using a pre-built libtrace tool that supports parallelism (e.g. tracertstats), then there will be a -t command line option which you can use to set the number of processing threads.

If you are writing your own parallel libtrace program, you can call the function trace_set_perpkt_threads() to specify the number of processing threads to use – make sure you call this before you call trace_pstart().