Networking Tools & Wireshark
Preface
Networking is one of those undergraduate courses that requires practice alongside the theory. Fortunately, grokking the practice can be straightforward through playing with tools, protocols, individual bits on the wire.
All the tools I mention in this text are available on Linux. As with any other Linux utility, the full manual is available in the man pages;
man dig
man ip # etc.
Man pages are the definitive documentation of the programs, they are in depth and wordy. If you feel lost or are looking for something terse, check out cht.sh for cheat sheets.
δ ~ curl cheat.sh/curl # available through the command line, or navigate to http://cht.sh/curl
# curl
# Command-line tool for transferring data with URL syntax
# Process a single GET request, and show its output on stdout.
curl http://path.to.the/file
# -snip-
curl
The utility we used to print a website's output to the terminal above is curl.
It can be thought as a command line web browser.
It is especially useful to test or use APIs;
δ curl 'https://cat-fact.herokuapp.com/facts/random?amount=1'
{"status":{"verified":true,"sentCount":1},"_id":"591f98703b90f7150a19c126","__v":0,"text":"Cats eat grass to aid their digestion and to help them get rid of any fur in their stomachs.","source":"api","updatedAt":"2020-08-23T20:20:01.611Z","type":"cat","createdAt":"2018-01-04T01:10:54.673Z","deleted":false,"used":false,"user":"5a9ac18c7478810ea6c06381"}
APIs often use lengthy JSON, we can use jq, the JSON processor to get what we are interested in;
δ curl 'https://cat-fact.herokuapp.com/facts/random?amount=1' | jq '.text'
"Cats are among only a few animals that walk by moving their two right legs one after another and then their two left legs, rather than moving diagonal limbs simultaneously. Giraffes and camels also have this quality."
If you want to download a file, use the -O flag.
δ curl -O https://raw.githubusercontent.com/johanhelsing/matchbox/main/README.md
curl is definitely a standard.
For instance, you can make a request with your browser then copy that request as a curl command to get the headers required.
curl 'https://github.com/eyal0/Chicken-story/blob/main/README.md' \
-H 'User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:94.0) Gecko/20100101 Firefox/94.0' \
-H 'Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,*/*;q=0.8' \
-H 'Accept-Language: en-US,en;q=0.5' --compressed -H 'DNT: 1' \
-H 'Connection: keep-alive' -H 'Upgrade-Insecure-Requests: 1' \
-H 'Sec-Fetch-Dest: document' -H 'Sec-Fetch-Mode: navigate' \
-H 'Sec-Fetch-Site: same-origin' -H 'Sec-GPC: 1' \
-H 'Cache-Control: max-age=0' -H 'TE: trailers'
There is also httpie. It offers easier syntax for HTML query strings or headers. Here's a comparison;
δ curl -L -G -d 'q=httpie' -d 'per_page=1' https://api.github.com/search/repositories | jq
δ http https://api.github.com/search/repositories q==httpie per_page==1 # json formatting built in
ip (ipconfig/ifconfig)
ip is most useful for checking your computer's networking configuration.
ip address will list addresses and ip link will list the network devices (which will be the loopback device and the ethernet/Wi-Fi card for most of us).
ip route gives the route table.
Unless you have configured the route table, your computer is currently sending everything to your home modem by default.
The address, link, route etc. are called "objects" and we can abbreviate them, ip will get what we mean;
δ ~ ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: eno1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
link/ether eb:2a:bc:e9:7b:91 brd ff:ff:ff:ff:ff:ff
altname enp0s25
inet 192.168.1.48/24 brd 192.168.1.255 scope global dynamic noprefixroute eno1
valid_lft 2201sec preferred_lft 2201sec
inet6 fe80::fcfc:fab6:d3b3:2e1b/64 scope link noprefixroute
valid_lft forever preferred_lft forever
For advanced use cases, ip can also manipulate the network configuration.
You can define a NAT or create network namespaces (which is what container runtimes like Docker use).
dig
dig is used to query DNS servers.
δ dig ceng.metu.edu.tr
; <<>> DiG 9.16.22 <<>> ceng.metu.edu.tr
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 948
;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1
;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 1232
;; QUESTION SECTION:
;ceng.metu.edu.tr. IN A
;; ANSWER SECTION:
ceng.metu.edu.tr. 36000 IN A 144.122.145.146
;; Query time: 19 msec
;; SERVER: 1.1.1.1#53(1.1.1.1)
;; WHEN: Sat Nov 13 00:10:07 +03 2021
;; MSG SIZE rcvd: 61
You can specify the DNS record type as well, no dash (-) needed;
δ ~ dig ceng.metu.edu.tr mx # for mail servers
# -snip-
;; ANSWER SECTION:
ceng.metu.edu.tr. 34285 IN MX 10 mailhost.ceng.metu.edu.tr.
Or use any to retrieve every available record for that URL.
Finally, you can use +trace flag to perform iterative queries and see the resolution/answer process step by step.
There is also dog, a DNS client like dig written in rust.
It comes with nicer, human readable output.
You can also check out this comic about how DNS works.
traceroute
You can use traceroute to print the trace of the network packets you send to a server.
δ ~/desk/personal_site traceroute metu.edu.tr
traceroute to metu.edu.tr (144.122.145.153), 30 hops max, 60 byte packets
1 _gateway (192.168.1.1) 0.518 ms 0.738 ms 0.724 ms
2 212.156.201.33.static.turktelekom.com.tr (212.156.201.33) 10.203 ms 10.189 ms 10.180 ms
3 81.212.2.93.static.turktelekom.com.tr (81.212.2.93) 4.006 ms 3.990 ms 4.945 ms
4 06-umitkoy-sr12e-t2-2---06-umitkoy-sr12e-t3-1.statik.turktelekom.com.tr (81.212.24.79) 5.192 ms 5.177 ms 6.106 ms
5 06-ulus-xrs-t2-2---06-umitkoy-sr12e-t2-2.statik.turktelekom.com.tr (81.212.209.33) 6.455 ms 6.442 ms 6.709 ms
6 06-ulus-sr12e-t3-1---06-ulus-xrs-t2-2.statik.turktelekom.com.tr (81.212.222.208) 6.697 ms 3.902 ms 4.175 ms
7 212.156.64.46.static.turktelekom.com.tr (212.156.64.46) 4.130 ms 4.641 ms 4.629 ms
8 70.96.154.212.static.turk.net (212.154.96.70) 5.395 ms 6.261 ms 6.745 ms
9 * * *
10 144.122.1.18 (144.122.1.18) 5.725 ms 6.964 ms 6.955 ms
11 * * *
12 * * *
13 * * *
14 * * *
15 * * *
-snip-
telnet
telnet is an application (and also the name of the protocol) that lets us connect to a server's socket and talk directly to the server.
Without a browser, curl or anything that helps us deal with the protocols around, we are left to imitate the client by typing what the client would say, or whatever we want really!
telnet is mostly obsolete nowadays, as evident by the fact that we cannot even use man telnet to reach to full documentation but instead directed to info telnet.
Whenever that happens, you can be sure that you are about to use something truly ancient.
Yet, I find using telnet once in your (network engineering) life to be demystifying so it's included here.
Anything showcased here can be accomplished better and easier using dedicated tools like http.
Without further ado, here's some handmade requests;
Handmade HTTP
The highlighted lines are the user input and the rest are from the server.
δ ~ telnet example.com 80
Trying 93.184.216.34...
Connected to example.com.
Escape character is '^]'.
GET / HTTP/1.1
Host: example.com
HTTP/1.1 200 OK
Age: 504840
Cache-Control: max-age=604800
Content-Type: text/html; charset=UTF-8
Date: Fri, 12 Nov 2021 23:14:15 GMT
Etag: "3147526947+ident"
Expires: Fri, 19 Nov 2021 23:14:15 GMT
Last-Modified: Thu, 17 Oct 2019 07:18:26 GMT
Server: ECS (dcb/7EEF)
Vary: Accept-Encoding
X-Cache: HIT
Content-Length: 1256
<!doctype html>
<html>
-snip-
Handmade SMTP (Sending Mail)
Again, the highlighted lines are the user input and the rest are from the server.
δ ~ telnet mailhost.ceng.metu.edu.tr 25
Trying 144.122.171.43...
Connected to mailhost.ceng.metu.edu.tr.
Escape character is '^]'.
220 guvercin.ceng.metu.edu.tr ESMTP
HELO guvercin.ceng.metu.edu.tr
250 guvercin.ceng.metu.edu.tr
MAIL FROM: batman@ceng.metu.edu.tr
250 2.1.0 Ok
RCPT TO: yigit@ceng.metu.edu.tr
250 2.1.5 Ok
DATA
354 End data with <CR><LF>.<CR><LF>
FROM: batman <batman@ceng.metu.edu.tr>
TO: yigit <yigit@ceng.metu.edu.tr>
Subject: handmade mail
The Batman Province contains the strategic Tigris river with fertile lands by its sides, as well as...
.
250 2.0.0 Ok: queued as 9B9462C17C
QUIT
221 2.0.0 Bye
Connection closed by foreign host.
You might think that this is a security issue.
We just sent a mail from someone that does not exist without entering a password!
Good news is, this is only enabled on our faculty staff network :).
If you connect to guvercin from anywhere else (even from the campus), you will get rejected.
Wireshark
After installing Wireshark using your distro's package manager, you will need to add your user to the wireshark user group to get capturing privileges.
# either of these will work, you can just enter your username instead of whoami, run as root
$ gpasswd -a $(whoami) wireshark
$ usermod -aG wireshark $(whoami)
Usually this change takes effect after you log back in (restart your PC).
If you are lazy, run wireshark from terminal as yourself, with a little trick;
$ sudo su $(whoami)
[sudo] password for yigit:
$ wireshark # new group membership applied!
Wireshark greets you with a standard GUI.
If you cannot see the interfaces listed under ip link, then you do not have capturing privileges.
You can start a new capture here or open an existing capture through File -> Open.
Usually, Wireshark is busy, your computer is always talking to other machines! We can quiet this traffic down by closing down applications (like your browser, Spotify, Steam, Discord etc.). This is also a good way to tell what your computer is doing behind your back.
We can use filters to get the information we are interested in.
Here is the documentation of filters (they are called DisplayFilters formally).
tcpdump
tcpdump is a packet sniffer like Wireshark, minus the GUI.
This makes it very convenient when you want to capture traffic on machines without displays, such as those you ssh into.
Use;
δ tcpdump -D
to see the available interfaces you can capture from - similar to the Wireshark greeting screen.
Usually you would like to write the packets to a file to import them back to Wireshark and inspect them visually.
The following will capture from the <interface> and write to out.pcap.
You can then use something like scp to get the packet capture on your own machine for inspection.
δ tcpdump -i <interface> -w out.pcap
A note about IPv4 addresses
Here is the table of major IP address blocks for Turkey.
We can look through the list to get some interesting tidbits.
I'm using visidata -- the Swiss Army Knife for .csv files for this section.
The highest number of IP addresses are owned by Turk Telekom which is around 7 million, lower than the 9.4 million subscribers they have reported in 2017. IPv6 adoption in Turkey is about 0.14%, NAT is here to stay.
I've cleaned up the file a bit and added headers, it's here if you want to play with it.
Closing
If you have any questions or comments, I'm available through my contact page or faculty mail
Image courtesy of Maartje Eyskens