Re: Diagnosing virtual machine network issues

From: Alex Arslan <ararslan_at_comcast.net>
Date: Fri, 19 Jul 2024 16:08:54 UTC
> I would start a pcap inside and outside of the VM for all udp port 53 traffic as a start to see if its a network issue going out of the box.  If it happens frequently and you think it might be the network, perhaps try with the Intel em driver instead of the virtio network driver ?

Thanks so much for your help!

The way I implemented your pcap suggestion was to use tcpdump, hopefully
that's correct. I ran tcpdump simultaneously on the host and VM then ran
the code where libcurl gives a timeout rather than the expected domain
resolution failure. The output is below. I'm pretty well outside of my
depth here; what is it I'm looking for that would be indicative of a
network issue going out of the VM?

Linux host:
$ sudo /usr/sbin/tcpdump -v -i any 'host 192.168.122.35 and port 53'
tcpdump: listening on any, link-type LINUX_SLL (Linux cooked v1), capture size 262144 bytes
21:06:03.320754 IP (tos 0x0, ttl 64, id 29048, offset 0, flags [none], proto UDP (17), length 60)
    192.168.122.35.24119 > amdci6.domain: 23532+ A? domain.invalid. (32)
21:06:03.320754 IP (tos 0x0, ttl 64, id 29048, offset 0, flags [none], proto UDP (17), length 60)
    192.168.122.35.24119 > amdci6.domain: 23532+ A? domain.invalid. (32)
21:06:03.321633 IP (tos 0x0, ttl 64, id 27798, offset 0, flags [none], proto UDP (17), length 73)
    192.168.122.35.18137 > amdci6.domain: 61699+ PTR? 35.122.168.192.in-addr.arpa. (45)
21:06:03.321633 IP (tos 0x0, ttl 64, id 27798, offset 0, flags [none], proto UDP (17), length 73)
    192.168.122.35.18137 > amdci6.domain: 61699+ PTR? 35.122.168.192.in-addr.arpa. (45)
21:06:03.321701 IP (tos 0x0, ttl 64, id 44762, offset 0, flags [DF], proto UDP (17), length 113)
    amdci6.domain > 192.168.122.35.18137: 61699* 1/0/0 35.122.168.192.in-addr.arpa. PTR freebsd-debugging-amdci6-0. (85)
21:06:03.321707 IP (tos 0x0, ttl 64, id 44762, offset 0, flags [DF], proto UDP (17), length 113)
    amdci6.domain > 192.168.122.35.18137: 61699* 1/0/0 35.122.168.192.in-addr.arpa. PTR freebsd-debugging-amdci6-0. (85)
21:06:03.322188 IP (tos 0x0, ttl 64, id 27799, offset 0, flags [none], proto UDP (17), length 72)
    192.168.122.35.37631 > amdci6.domain: 23871+ PTR? 1.122.168.192.in-addr.arpa. (44)
21:06:03.322188 IP (tos 0x0, ttl 64, id 27799, offset 0, flags [none], proto UDP (17), length 72)
    192.168.122.35.37631 > amdci6.domain: 23871+ PTR? 1.122.168.192.in-addr.arpa. (44)
21:06:08.446737 IP (tos 0x0, ttl 64, id 29049, offset 0, flags [none], proto UDP (17), length 60)
    192.168.122.35.24119 > amdci6.domain: 23532+ A? domain.invalid. (32)
21:06:08.446737 IP (tos 0x0, ttl 64, id 29049, offset 0, flags [none], proto UDP (17), length 60)
    192.168.122.35.24119 > amdci6.domain: 23532+ A? domain.invalid. (32)
21:06:18.567376 IP (tos 0x0, ttl 64, id 29050, offset 0, flags [none], proto UDP (17), length 60)
    192.168.122.35.37009 > amdci6.domain: 36459+ AAAA? domain.invalid. (32)
21:06:18.567376 IP (tos 0x0, ttl 64, id 29050, offset 0, flags [none], proto UDP (17), length 60)
    192.168.122.35.37009 > amdci6.domain: 36459+ AAAA? domain.invalid. (32)
21:06:23.671046 IP (tos 0x0, ttl 64, id 29051, offset 0, flags [none], proto UDP (17), length 60)
    192.168.122.35.37009 > amdci6.domain: 36459+ AAAA? domain.invalid. (32)
21:06:23.671046 IP (tos 0x0, ttl 64, id 29051, offset 0, flags [none], proto UDP (17), length 60)
    192.168.122.35.37009 > amdci6.domain: 36459+ AAAA? domain.invalid. (32)
^C
14 packets captured
20 packets received by filter
2 packets dropped by kernel

FreeBSD VM:
$ sudo tcpdump -v port 53
tcpdump: listening on vtnet0, link-type EN10MB (Ethernet), capture size 262144 bytes
21:06:06.179751 IP (tos 0x0, ttl 64, id 29048, offset 0, flags [none], proto UDP (17), length 60)
    freebsd-debugging-amdci6-0.24119 > amdci6.domain: 23532+ A? domain.invalid. (32)
21:06:06.180634 IP (tos 0x0, ttl 64, id 27798, offset 0, flags [none], proto UDP (17), length 73)
    freebsd-debugging-amdci6-0.18137 > amdci6.domain: 61699+ PTR? 35.122.168.192.in-addr.arpa. (45)
21:06:06.180826 IP (tos 0x0, ttl 64, id 44762, offset 0, flags [DF], proto UDP (17), length 113)
    amdci6.domain > freebsd-debugging-amdci6-0.18137: 61699* 1/0/0 35.122.168.192.in-addr.arpa. PTR freebsd-debugging-amdci6-0. (85)
21:06:06.181193 IP (tos 0x0, ttl 64, id 27799, offset 0, flags [none], proto UDP (17), length 72)
    freebsd-debugging-amdci6-0.37631 > amdci6.domain: 23871+ PTR? 1.122.168.192.in-addr.arpa. (44)
21:06:06.194107 IP (tos 0x0, ttl 64, id 44764, offset 0, flags [DF], proto UDP (17), length 118)
    amdci6.domain > freebsd-debugging-amdci6-0.37631: 23871 2/0/0 1.122.168.192.in-addr.arpa. PTR amdci6., 1.122.168.192.in-addr.arpa. PTR amdci6.local. (90)
21:06:11.305743 IP (tos 0x0, ttl 64, id 29049, offset 0, flags [none], proto UDP (17), length 60)
    freebsd-debugging-amdci6-0.24119 > amdci6.domain: 23532+ A? domain.invalid. (32)
21:06:21.426439 IP (tos 0x0, ttl 64, id 29050, offset 0, flags [none], proto UDP (17), length 60)
    freebsd-debugging-amdci6-0.37009 > amdci6.domain: 36459+ AAAA? domain.invalid. (32)
21:06:26.530138 IP (tos 0x0, ttl 64, id 29051, offset 0, flags [none], proto UDP (17), length 60)
    freebsd-debugging-amdci6-0.37009 > amdci6.domain: 36459+ AAAA? domain.invalid. (32)
^C
8 packets captured
427 packets received by filter
0 packets dropped by kernel