From nobody Fri Jul 12 16:38:40 2024 X-Original-To: freebsd-virtualization@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4WLHNW10rfz5QmN2 for ; Fri, 12 Jul 2024 16:38:51 +0000 (UTC) (envelope-from freebsd-rwg@gndrsh.dnsmgr.net) Received: from gndrsh.dnsmgr.net (pdx.rh.CN85.dnsmgr.net [65.75.216.6]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4WLHNV0trHz4Shs for ; Fri, 12 Jul 2024 16:38:49 +0000 (UTC) (envelope-from freebsd-rwg@gndrsh.dnsmgr.net) Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=pass (policy=none) header.from=gndrsh.dnsmgr.net; spf=pass (mx1.freebsd.org: domain of freebsd-rwg@gndrsh.dnsmgr.net designates 65.75.216.6 as permitted sender) smtp.mailfrom=freebsd-rwg@gndrsh.dnsmgr.net Received: from gndrsh.dnsmgr.net (localhost [127.0.0.1]) by gndrsh.dnsmgr.net (8.13.3/8.13.3) with ESMTP id 46CGceRW055351; Fri, 12 Jul 2024 09:38:40 -0700 (PDT) (envelope-from freebsd-rwg@gndrsh.dnsmgr.net) Received: (from freebsd-rwg@localhost) by gndrsh.dnsmgr.net (8.13.3/8.13.3/Submit) id 46CGce8U055350; Fri, 12 Jul 2024 09:38:40 -0700 (PDT) (envelope-from freebsd-rwg) From: "Rodney W. Grimes" Message-Id: <202407121638.46CGce8U055350@gndrsh.dnsmgr.net> Subject: Re: Suddenly unable to access VMs In-Reply-To: To: Odhiambo Washington Date: Fri, 12 Jul 2024 09:38:40 -0700 (PDT) CC: FreeBSD virtualization X-Mailer: ELM [version 2.4ME+ PL121h (25)] List-Id: Discussion List-Archive: https://lists.freebsd.org/archives/freebsd-virtualization List-Help: List-Post: List-Subscribe: List-Unsubscribe: X-BeenThere: freebsd-virtualization@freebsd.org Sender: owner-freebsd-virtualization@FreeBSD.org MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=US-ASCII X-Spamd-Bar: --- X-Spamd-Result: default: False [-3.69 / 15.00]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_SHORT(-0.89)[-0.887]; DMARC_POLICY_ALLOW(-0.50)[gndrsh.dnsmgr.net,none]; R_SPF_ALLOW(-0.20)[+ip4:65.75.216.0/23]; MIME_GOOD(-0.10)[text/plain]; FREEMAIL_TO(0.00)[gmail.com]; RCPT_COUNT_TWO(0.00)[2]; ASN(0.00)[asn:10494, ipnet:65.75.216.0/23, country:US]; ARC_NA(0.00)[]; MID_RHS_MATCH_FROM(0.00)[]; MLMMJ_DEST(0.00)[freebsd-virtualization@FreeBSD.org]; RCVD_TLS_LAST(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; FROM_HAS_DN(0.00)[]; R_DKIM_NA(0.00)[]; RCVD_COUNT_TWO(0.00)[2]; TO_DN_ALL(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; MIME_TRACE(0.00)[0:+] X-Rspamd-Queue-Id: 4WLHNV0trHz4Shs > On Thu, Jul 11, 2024 at 5:49?PM Rodney W. Grimes < > freebsd-rwg@gndrsh.dnsmgr.net> wrote: > > > > My bhyve VMs have been all fine until now. > > > I can't ping them and can't SSH into them. However, I can connect to them > > > with VNCViewer from a remote host (my PC from my house) :-( > > > > > > I haven't done any changes on the host at all. > > > dnsmasq is running, but seems like the VMs aren't getting the IPs for > > some > > > reason. > > > > > > ``` > > > cloned_interfaces="bridge0 tap0 tap1 tap2 tap3 tap4 tap5" > > > ifconfig_bridge0_name="vmbridge" > > > ifconfig_vmbridge="addm em1 addm tap0 addm tap1 addm tap2 addm tap3 addm > > > tap4 addm tap5 up" > > > ifconfig_vmbridge_alias0="inet 172.16.0.1 netmask 255.255.255.0" > > > ``` > > > What might have happened? > > > > > > > > > root@gw:/home/wash # ifconfig vmbridge > > > vmbridge: flags=1008843 > > > metric 0 mtu 1500 > > > options=0 > > > ether 58:9c:fc:10:df:1d > > > inet 172.16.0.1 netmask 0xffffff00 broadcast 172.16.0.255 > > > id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15 > > > maxage 20 holdcnt 6 proto rstp maxaddr 2000 timeout 1200 > > > root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0 > > > member: tap5 flags=143 > > > ifmaxaddr 0 port 10 priority 128 path cost 2000000 > > > member: tap4 flags=143 > > > ifmaxaddr 0 port 9 priority 128 path cost 2000000 > > > member: tap3 flags=143 > > > ifmaxaddr 0 port 8 priority 128 path cost 2000000 > > > member: tap2 flags=143 > > > ifmaxaddr 0 port 7 priority 128 path cost 2000000 > > > member: tap1 flags=143 > > > ifmaxaddr 0 port 6 priority 128 path cost 2000000 > > > member: tap0 flags=143 > > > ifmaxaddr 0 port 5 priority 128 path cost 2000000 > > > member: em1 flags=143 > > > ifmaxaddr 0 port 2 priority 128 path cost 55 > > > groups: bridge > > > nd6 options=9 > > > root@gw:/home/wash # ssh 172.16.0.99 > > > ssh: connect to host 172.16.0.99 port 22: Permission denied > > > root@gw:/home/wash # ssh 172.16.0.100 > > > ssh: connect to host 172.16.0.100 port 22: Permission denied > > > root@gw:/home/wash # ping 172.16.0.100 > > > PING 172.16.0.100 (172.16.0.100): 56 data bytes > > > ping: sendto: Permission denied > > > ping: sendto: Permission denied > > > ping: sendto: Permission denied > > > ping: sendto: Permission denied > > > ^C > > > --- 172.16.0.100 ping statistics --- > > > 4 packets transmitted, 0 packets received, 100.0% packet loss > > > root@gw:/home/wash # ping 172.16.0.99 > > > PING 172.16.0.99 (172.16.0.99): 56 data bytes > > > ping: sendto: Permission denied > > > ping: sendto: Permission denied > > > ping: sendto: Permission denied > > > ^C > > > --- 172.16.0.99 ping statistics --- > > > 3 packets transmitted, 0 packets received, 100.0% packet loss > > > root@gw:/home/wash # service dnsmasq status > > > dnsmasq is running as pid 4190. > > > root@gw:/home/wash # > > > > Permission denied is almost certainly coming from firewall, > > either ipfw or pf. > > > > I haven't changed anything in my pf.conf either. > What also baffles me is that the VMs are not obtaining IP addresses from > dnsmasq. You may of not changed anything, but I would take a very close look at pf and what rule in PF is denying your packets, cause the error you show is more likely than not to be caused by a pf rule. > -- > Best regards, > Odhiambo WASHINGTON, > Nairobi,KE > +254 7 3200 0004/+254 7 2274 3223 > In an Internet failure case, the #1 suspect is a constant: DNS. > "Oh, the cruft.", egrep -v '^$|^.*#' ?\_(?)_/? :-) > [How to ask smart questions: > http://www.catb.org/~esr/faqs/smart-questions.html] -- Rod Grimes rgrimes@freebsd.org