Re: Diagnosing virtual machine network issues

> On Aug 13, 2024, at 9:15 AM, Bakul Shah <bakul@iitbombay.org> wrote:
> 
> This weird 127. address seems like a systemd feature/bug thing: https://unix.stackexchange.com/questions/612416/why-does-etc-resolv-conf-point-at-127-0-0-53
> 
> This behavior seems like some strange interaction between systemd assumptions and freebsd’s, or something not being set up quite right on the linux side when the vm is running freebsd. 

Could libvirt be a factor here, do you think? For example, perhaps the
network should be configured differently than the default when the host
is using systemd-resolved and/or when the guest is FreeBSD. In the network
XML format for libvirt (https://libvirt.org/formatnetwork.html), there is
a `domain` element with a `localOnly` attribute that I have seen set by
some virtualization projects. As far as I can tell, our setup isn't using
the `domain` element at all.

> 
>> On Aug 13, 2024, at 8:46 AM, Alex Arslan <ararslan@comcast.net> wrote:
>> 
>> 
>> Hi Rodney,
>> 
>>> On Aug 10, 2024, at 9:11 AM, Rodney W. Grimes <freebsd-rwg@gndrsh.dnsmgr.net> wrote:
>>> 
>>>> 
>>>> 
>>>>> On Aug 2, 2024, at 5:58?PM, Bakul Shah <bakul@iitbombay.org> wrote:
>>>>> 
>>>>> On Aug 2, 2024, at 3:52?PM, Alex Arslan <ararslan@comcast.net> wrote:
>>>>>> 
>>>>>>> Just a comment and a name server line:
>>>>>>> 
>>>>>>> $ cat /etc/resolv.conf
>>>>>>> # Generated by resolvconf
>>>>>>> nameserver 192.168.122.1
>>>>>> 
>>>>>> I believe that is the host IP, so I guess the VM is using the host for DNS
>>>>>> resolution? Interestingly, if I add `nameserver 8.8.8.8` below the line
>>>>>> with the host IP, it takes 10 seconds rather than 30 to reach the expected
>>>>>> domain resolution failure. If I put 8.8.8.8 above the host IP, the domain
>>>>>> resolution failure is instantaneous.
>>>>> 
>>>>> What does your host use as a namesever?
>>>> 
>>>> The nameserver is 127.0.0.53. It sets options edns0 and trust-ad, and
>>>> includes a search entry as well.
>>> 
>>> First, is that a typo and you mean 127.0.0.1:53?
>> 
>> No, the host's /etc/resolv.conf has `nameserver 127.0.0.53`, I just went
>> back and rechecked to be sure.
>> 
>>> Second, is that name server locked to 127.0.0.1, or is it
>>> actually listinging on *:53?  If it is LOCKED you have no name server
>>> running on 192.168.122.1 to be reached by the VM, if it is NOT locked
>>> can the guest ping 192.168.122.1, and can it reach dns at that IP on
>>> port 53?   Can the host send a packet BACK to the guest?
>> 
>> I apologize but I don't really know enough about these things to know how
>> to answer your question. I did post the output of tcpdump on the VM and
>> the host a while back but that was for the invalid request, so that
>> probably doesn't capture what you're describing.
>> 
>>> Third you can "fix" the "nameserver 192.168.122.1" entry in /etc/resolv.conf
>>> by configuring the DHCP server that handed out the lease to the VM to send
>>> a namserver entry of 8.8.8.8.
>> 
>> If I understand correctly, that is indeed what we've done as a Band-Aid fix
>> for the time being: I added the line `prepend_nameservers=8.8.8.8` to
>> /etc/resolvconf.conf.
>> 
>>>> 
>>>>> 
>>>>>> Not a particularly satisfying conclusion to this saga as I don't understand
>>>>>> why it's happening but at least I have a workaround that should hopefully
>>>>>> do the job. I really appreciate everyone's help and input thus far!
>>>>>> 
>>>>>> What's the best way to add `nameserver 8.8.8.8` to /etc/resolv.conf as
>>>>>> part of the VM's configuration?
>>>>> 
>>>>> You should diagnose the problem of the nameserver at 192.168.122.1
>>>>> and fix it to act properly. I don't use vm (just bhyve) so can't help
>>>>> you with its config.
>>>> 
>>>> I do still plan to try to figure out what the actual issue is, but I also
>>>> now have a path forward in the meantime. :)
>>>> 
>>>> 
>>> 
>>> -- 
>>> Rod Grimes                                                 rgrimes@freebsd.org <mailto:rgrimes@freebsd.org>