Re: Suddenly unable to access VMs

From: Rodney W. Grimes <freebsd-rwg_at_gndrsh.dnsmgr.net>
Date: Fri, 12 Jul 2024 16:38:40 UTC
> On Thu, Jul 11, 2024 at 5:49?PM Rodney W. Grimes <
> freebsd-rwg@gndrsh.dnsmgr.net> wrote:
> 
> > > My bhyve VMs have been all fine until now.
> > > I can't ping them and can't SSH into them. However, I can connect to them
> > > with VNCViewer from a remote host (my PC from my house) :-(
> > >
> > > I haven't done any changes on the host at all.
> > > dnsmasq is running, but seems like the VMs aren't getting the IPs for
> > some
> > > reason.
> > >
> > > ```
> > > cloned_interfaces="bridge0 tap0 tap1 tap2 tap3 tap4 tap5"
> > > ifconfig_bridge0_name="vmbridge"
> > > ifconfig_vmbridge="addm em1 addm tap0 addm tap1 addm tap2 addm tap3 addm
> > > tap4 addm tap5 up"
> > > ifconfig_vmbridge_alias0="inet 172.16.0.1 netmask 255.255.255.0"
> > > ```
> > > What might have happened?
> > >
> > >
> > > root@gw:/home/wash # ifconfig vmbridge
> > > vmbridge: flags=1008843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST,LOWER_UP>
> > > metric 0 mtu 1500
> > >         options=0
> > >         ether 58:9c:fc:10:df:1d
> > >         inet 172.16.0.1 netmask 0xffffff00 broadcast 172.16.0.255
> > >         id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15
> > >         maxage 20 holdcnt 6 proto rstp maxaddr 2000 timeout 1200
> > >         root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0
> > >         member: tap5 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
> > >                 ifmaxaddr 0 port 10 priority 128 path cost 2000000
> > >         member: tap4 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
> > >                 ifmaxaddr 0 port 9 priority 128 path cost 2000000
> > >         member: tap3 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
> > >                 ifmaxaddr 0 port 8 priority 128 path cost 2000000
> > >         member: tap2 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
> > >                 ifmaxaddr 0 port 7 priority 128 path cost 2000000
> > >         member: tap1 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
> > >                 ifmaxaddr 0 port 6 priority 128 path cost 2000000
> > >         member: tap0 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
> > >                 ifmaxaddr 0 port 5 priority 128 path cost 2000000
> > >         member: em1 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
> > >                 ifmaxaddr 0 port 2 priority 128 path cost 55
> > >         groups: bridge
> > >         nd6 options=9<PERFORMNUD,IFDISABLED>
> > > root@gw:/home/wash # ssh 172.16.0.99
> > > ssh: connect to host 172.16.0.99 port 22: Permission denied
> > > root@gw:/home/wash # ssh 172.16.0.100
> > > ssh: connect to host 172.16.0.100 port 22: Permission denied
> > > root@gw:/home/wash # ping 172.16.0.100
> > > PING 172.16.0.100 (172.16.0.100): 56 data bytes
> > > ping: sendto: Permission denied
> > > ping: sendto: Permission denied
> > > ping: sendto: Permission denied
> > > ping: sendto: Permission denied
> > > ^C
> > > --- 172.16.0.100 ping statistics ---
> > > 4 packets transmitted, 0 packets received, 100.0% packet loss
> > > root@gw:/home/wash # ping 172.16.0.99
> > > PING 172.16.0.99 (172.16.0.99): 56 data bytes
> > > ping: sendto: Permission denied
> > > ping: sendto: Permission denied
> > > ping: sendto: Permission denied
> > > ^C
> > > --- 172.16.0.99 ping statistics ---
> > > 3 packets transmitted, 0 packets received, 100.0% packet loss
> > > root@gw:/home/wash # service dnsmasq status
> > > dnsmasq is running as pid 4190.
> > > root@gw:/home/wash #
> >
> > Permission denied is almost certainly coming from firewall,
> > either ipfw or pf.
> >
> 
> I haven't changed anything in my pf.conf either.
> What also baffles me is that the VMs are not obtaining IP addresses from
> dnsmasq.

You may of not changed anything, but I would take a very close
look at pf and what rule in PF is denying your packets, cause
the error you show is more likely than not to be caused by
a pf rule.

> -- 
> Best regards,
> Odhiambo WASHINGTON,
> Nairobi,KE
> +254 7 3200 0004/+254 7 2274 3223
>  In an Internet failure case, the #1 suspect is a constant: DNS.
> "Oh, the cruft.", egrep -v '^$|^.*#' ?\_(?)_/? :-)
> [How to ask smart questions:
> http://www.catb.org/~esr/faqs/smart-questions.html]

-- 
Rod Grimes                                                 rgrimes@freebsd.org