From nobody Tue Aug 17 13:55:23 2021 X-Original-To: freebsd-virtualization@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4B83717692BC; Tue, 17 Aug 2021 13:56:02 +0000 (UTC) (envelope-from marietto2008@gmail.com) Received: from mail-lf1-x12d.google.com (mail-lf1-x12d.google.com [IPv6:2a00:1450:4864:20::12d]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Gpsxs2NLMz4phV; Tue, 17 Aug 2021 13:56:01 +0000 (UTC) (envelope-from marietto2008@gmail.com) Received: by mail-lf1-x12d.google.com with SMTP id p38so41725279lfa.0; Tue, 17 Aug 2021 06:56:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to; bh=yCqfVM+Wq+w2AECoz1MYUMaTpByE9hVZ8A8XKYHCYkk=; b=RjM36RW6DIZVv649yNFoBvEG5F84nhy5hA14IokbYq4mgpv1hEuvvFPJvZIEghTWtX 6C6wrl86mER5Uf4MX4jfSurxC0hfHej1L3HbD3ep0iqaRfD2/x6fuP7/2u3SYGsgpi2t w+Ot4lH4pUaNKNohVdHXuLwuiK0Z1ch1jZRGnTUsv+dauNFksvwyD0zYw2QaiMPZIkd0 7nMRa4badks3HiARI3Hq7TNsK1W3hrAyFjp8/N/3xRmuWTmJjguoZxaI+Ep+ZBDKsxn4 iL4wcwz4o1QGmL3febvxq49dmeqB6fHow+u9vvlTwbpqxlKER5YdECgwxjaMJfbYCYdj Gstw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to; bh=yCqfVM+Wq+w2AECoz1MYUMaTpByE9hVZ8A8XKYHCYkk=; b=rli6nKLlmJ/uU9evuOXDrj2koUj9GSbzCKnF5e6p+ahEzyUMCHoK7xaNm0ycQvcKZl oCYhCpVHfIgruraR+7yCohbITwqUR2hIPUiEIOnZLjMWgzTmC3xZGfNyJ2Zh38AfHoRE PMxWWfi4EwCdbRb61kEv4as7k1czTHyqIcG5yBpVhz430bYQcR+VziOeLZVHB9hEyd6E FlMxLaaBGI1a4KM/EnsFx0kKhe2IHaHz9OHlHhTkC0K1pSj3RNNKOtfk0B9nIMj0Guhs hYG5kFgOIZtectfLKExGAMg4xNi1W7sAreen0XIzSI5rGWu3Lop3eDydWb451iDpuFAh KxBQ== X-Gm-Message-State: AOAM5332rywz1KosHL5ryFEXK9brP1W36DFecFJUwpHDFyDuuYRx0zRb ZPbvL8Xbr5XMCXV705dwYAu+2ZN6w1f17xUKX9SzL/qRmpAzbw== X-Google-Smtp-Source: ABdhPJydfwZfJOcmB94SLiNvi3zeWgBhAVgmKHYE9ppTda/c70NIA4tivob8wS2F7wqzVyVLPVhehl1+gf4RsFY2GXg= X-Received: by 2002:a19:c392:: with SMTP id t140mr2513034lff.387.1629208559508; Tue, 17 Aug 2021 06:55:59 -0700 (PDT) List-Id: Discussion List-Archive: https://lists.freebsd.org/archives/freebsd-virtualization List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-virtualization@freebsd.org X-BeenThere: freebsd-virtualization@freebsd.org MIME-Version: 1.0 References: <1629141950-13176-mlmmj-4abd3c89@FreeBSD.org> In-Reply-To: From: Mario Marietto Date: Tue, 17 Aug 2021 15:55:23 +0200 Message-ID: Subject: Re: Confirm subscription to freebsd-net@FreeBSD.org To: freebsd-net@freebsd.org, freebsd-virtualization@freebsd.org Content-Type: multipart/alternative; boundary="00000000000074620f05c9c1ad86" X-Rspamd-Queue-Id: 4Gpsxs2NLMz4phV X-Spamd-Bar: - Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20161025 header.b=RjM36RW6; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (mx1.freebsd.org: domain of marietto2008@gmail.com designates 2a00:1450:4864:20::12d as permitted sender) smtp.mailfrom=marietto2008@gmail.com X-Spamd-Result: default: False [-1.99 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20161025]; FROM_HAS_DN(0.00)[]; R_SPF_ALLOW(-0.20)[+ip6:2a00:1450:4000::/36:c]; FREEMAIL_FROM(0.00)[gmail.com]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; TO_DN_NONE(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; URI_COUNT_ODD(1.00)[19]; MID_RHS_MATCH_FROMTLD(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; DKIM_TRACE(0.00)[gmail.com:+]; RCPT_COUNT_TWO(0.00)[2]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; HTTP_TO_IP(1.00)[]; RCVD_IN_DNSWL_NONE(0.00)[2a00:1450:4864:20::12d:from]; NEURAL_HAM_SHORT(-0.99)[-0.991]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:~]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US]; RCVD_COUNT_TWO(0.00)[2]; RCVD_TLS_ALL(0.00)[]; DWL_DNSWL_NONE(0.00)[gmail.com:dkim] X-Spam: Yes X-ThisMailContainsUnwantedMimeParts: N --00000000000074620f05c9c1ad86 Content-Type: text/plain; charset="UTF-8" Hello. I'm trying to configure the bridging and the tap interface on freebsd 13 because I want to boot and use a bhyve VM based on Ubuntu 21.04 and Windows 10,but unfortunately,after several readings and help requests on : 0. https://www.reddit.com/r/Ubuntu/comments/p38ru1/ubuntu_2104_cant_connect_to_internet_when_it_is/ 1. https://forums.freebsd.org/threads/how-to-duplicate-the-tap0-interface-in-a-new-tap1-network-interface-that-it-is-not-used.81671/page-2#post-527355 2. https://forums.freebsd.org/threads/ubuntu-budgie-emulated-with-bhyve-doesnt-connect-to-internet-because-the-error-network-unreachable-n-2.81340/#post-524303 3. https://forums.freebsd.org/threads/im-tryng-to-emulate-ubuntu-budgie-with-bhyve-but-it-doesnt-connect-to-internet-because-the-error-network-unreachable.80916/#post-517845 4. https://askubuntu.com/questions/1357670/ubuntu-21-04-cant-connect-to-internet-when-it-is-emulated-with-bhyve-on-my-free 5. https://unix.stackexchange.com/questions/664466/ubuntu-21-04-cant-connect-to-internet-when-it-is-emulated-with-bhyve-on-my-free?noredirect=1#comment1250440_664466 6. https://serverfault.com/questions/1074745/my-bridging-configuration-on-freebsd-13-inst-properly-configured-because-any-bh 7. https://www.linuxquestions.org/questions/linux-virtualization-and-cloud-90/my-bridging-configuration-on-freebsd-13-ins%27t-properly-configured-because-any-bhyve-vm-based-on-ubuntu-21-04-and-windows-10-can%27t-connect-4175699240/ I haven't been able to understand why Ubuntu and Windows 10 VM can't connect to the internet,while if I use vm-bhyve,it configures the network in some different way and the vm can connect to the internet. Since I'm not a coder,I'm not able to look inside the vm-bhyve code and I'm not able to understand what's the secret key. Below I want to give you some useful information for understanding how I have configured the bridge and the tap interface on FreeBSD and on Ubuntu. /etc/resolv.conf search homenet.telecomitalia.it nameserver 127.0.0.1 options edns0 my /etc/rc.conf is like this : hostname="marietto" keymap="it.kbd" ifconfig_em0="DHCP" local_unbound_enable="YES" sshd_enable="YES" powerd_enable="YES" ntpdate_enable="YES" ntpd_enable="YES" ntpd_sync_on_start="YES" dumpdev="NO" dbus_enable="YES" slim_enable="NO" libvirt_enable="YES" linux_enable="YES" linux_mounts_enable="YES" gateway_enable="YES" kld_list="fusefs" lightdm_enable="NO" sendmail_enable="NO" microcode_update_enable="YES" fsck_y_enable="YES" tcp_drop_synfin="NO" telnet_enable="NO" cupsd_enable="NO" portmap_enable="NO" rlogin_enable="NO" inetd_enable="NO" webcamd_enable="NO" samba_enable="NO" lpd_enable="NO" winbindd_enable="NO" nfs_server_enable="NO" nfs_client_enable="NO" vm_enable="YES" vm_dir="/mnt/da1p1/vms/os/" my /boot/loader.conf is like this : vmm_load="YES" nmdm_load="YES" if_tap_load="YES" if_bridge_load="YES" fusefs_load="YES" linprocfs_load="YES" linsysfs_load="YES" tmpfs_load="YES" verbose_loading="YES" vboxdrv_load="YES" console=vga,com1 boot_multicons="YES" boot_serial="YES" console="comconsole,vidconsole" my file /etc/sysctl.conf is like this : net.link.tap.up_on_open=1 net.inet.ip.forwarding=1 I've read here to get some orientation about what to do : https://wiki.freebsd.org/bhyve/UEFI specially in this section : kldload vmm ifconfig tap0 create up ifconfig bridge0 create up ifconfig bridge0 addm em0 As a first attempt,with the suggestions of Vasily Postnicov,I tried to configure the network interfaces of freebsd and of linux in a different way,to be able to give the internet connection to ubuntu emulated with bhyve. This is what I did : root@marietto:/home/marietto # ifconfig em0 delete root@marietto:/home/marietto # dhclient bridge0 DHCPDISCOVER on bridge0 to 255.255.255.255 port 67 interval 4 DHCPOFFER from 192.168.1.1 DHCPREQUEST on bridge0 to 255.255.255.255 port 67 DHCPACK from 192.168.1.1 bound to 192.168.1.2 -- renewal in 10800 seconds. root@marietto:/home/marietto # ifconfig tap0 create up root@marietto:/home/marietto # ifconfig bridge0 addm tap0 root@marietto:/usr/home/marietto # ifconfig em0: flags=8963 metric 0 mtu 1500 options=4810099 ether e0:d5:5e:e2:1f:22 inet 0.0.0.0 netmask 0xff000000 broadcast 255.255.255.255 media: Ethernet autoselect (1000baseT ) status: active nd6 options=29 lo0: flags=8049 metric 0 mtu 16384 options=680003 inet6 ::1 prefixlen 128 inet6 fe80::1%lo0 prefixlen 64 scopeid 0x2 inet 127.0.0.1 netmask 0xff000000 groups: lo nd6 options=21 bridge0: flags=8843 metric 0 mtu 1500 ether 58:9c:fc:10:ff:b7 inet 192.168.1.2 netmask 0xffffff00 broadcast 192.168.1.255 id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15 maxage 20 holdcnt 6 proto rstp maxaddr 2000 timeout 1200 root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0 member: tap0 flags=143 ifmaxaddr 0 port 5 priority 128 path cost 2000000 member: em0 flags=143 ifmaxaddr 0 port 1 priority 128 path cost 2000000 groups: bridge nd6 options=9 vm-public: flags=8843 metric 0 mtu 1500 ether b6:74:ff:a8:4d:4e id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15 maxage 20 holdcnt 6 proto rstp maxaddr 2000 timeout 1200 root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0 groups: bridge vm-switch viid-4c918@ nd6 options=9 tap0: flags=8903 metric 0 mtu 1500 options=80000 ether 58:9c:fc:10:ff:8e groups: tap media: Ethernet autoselect status: no carrier nd6 options=29 Unfortunately as soon as the vm boots,the em0 interface on freebsd gets again the old IP number : 192.168.1.6 and I lose the connection to the net on FreeBSD. As a further try this is what I did : (take in consideration that I'm trying to make a parallel network configuration on Freebsd,since I want to keep vm-bhyve installed. Anyway,I've removed vm-bhyve several times and I tried to configure the bridge network for bhyve,but even in this case it didn't work. In short terms,it worked only the first time,but I haven't been able to repeat succesfully the steps anymore. It's working only if I let vm-bhyve configure the network by itself. This is the initial state of the network / bridge configuration on my FreeBSD with vm-bhyve installed but without starting a virtual machine. marietto@marietto:~ $ ifconfig em0: flags=8963 metric 0 mtu 1500 options=481009b ether e0:d5:5e:e2:1f:22 inet 192.168.1.6 netmask 0xffffff00 broadcast 192.168.1.255 media: Ethernet autoselect (1000baseT ) status: active nd6 options=29 lo0: flags=8049 metric 0 mtu 16384 options=680003 inet6 ::1 prefixlen 128 inet6 fe80::1%lo0 prefixlen 64 scopeid 0x2 inet 127.0.0.1 netmask 0xff000000 groups: lo nd6 options=21 vm-public: flags=8843 metric 0 mtu 1500 ether 0a:77:93:94:d5:77 id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15 maxage 20 holdcnt 6 proto rstp maxaddr 2000 timeout 1200 root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0 member: em0 flags=143 ifmaxaddr 0 port 1 priority 128 path cost 20000 groups: bridge vm-switch viid-4c918@ nd6 options=9 Since I already have a bridged interface (vm-public) it does not make any sense to create another one (bridge0),so on the terminal I do : root@marietto:/home/marietto # kldload vmm kldload: can't load vmm: module already loaded or in kernel root@marietto:/home/marietto # ifconfig tap0 create up root@marietto:/home/marietto # ifconfig vm-public addm tap0 so,this is how appear the network configuration on FreeBSD after having configured it like above : root@marietto:/home/marietto # ifconfig em0: flags=8963 metric 0 mtu 1500 options=4810099 ether e0:d5:5e:e2:1f:22 inet 192.168.1.6 netmask 0xffffff00 broadcast 192.168.1.255 media: Ethernet autoselect status: no carrier nd6 options=29 lo0: flags=8049 metric 0 mtu 16384 options=680003 inet6 ::1 prefixlen 128 inet6 fe80::1%lo0 prefixlen 64 scopeid 0x2 inet 127.0.0.1 netmask 0xff000000 groups: lo nd6 options=21 vm-public: flags=8843 metric 0 mtu 1500 ether 0a:77:93:94:d5:77 id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15 maxage 20 holdcnt 6 proto rstp maxaddr 2000 timeout 1200 root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0 member: tap0 flags=143 ifmaxaddr 0 port 4 priority 128 path cost 2000000 member: em0 flags=143 ifmaxaddr 0 port 1 priority 128 path cost 20000 groups: bridge vm-switch viid-4c918@ nd6 options=9 tap0: flags=8903 metric 0 mtu 1500 options=80000 ether 58:9c:fc:10:ff:8e groups: tap media: Ethernet autoselect status: no carrier nd6 options=29 Then,I launch the bhyve VM with this commands : bhyve -c 4 -m 8G -w -H \ -s 0,hostbridge \ -s 4,virtio-blk,/mnt/da1p1/vms/os/ubuntu-budgie-gpu/ubuntu-2104-gpu.img \ -s 5,virtio-net,tap0 \ -s 29,fbuf,tcp=0.0.0.0:5900,w=1440,h=900,wait \ -s 30,xhci,tablet \ -s 31,lpc -l com1,stdio \ -l bootrom,/usr/local/share/uefi-firmware/BHYVE_UEFI.fd \ -s 20,hda,play=/dev/dsp8,rec=/dev/dsp8 \ vm0 As I said Ubuntu VM can't connect to the internet. Below I want to show you how I have configured Ubuntu,instead. On Ubuntu my /etc/resolv.conf file is like this : nameserver 127.0.0.53 options edns0 trust-ad search homenet.telecomitalia.it In addition I've created a netplan file inside the folder /etc/netplan,with this content inside : network version: 2 renderer: NetworkManager ethernets: enp0s5: addresses: - 192.168.1.7/24 gateway4: 192.168.1.1 nameservers: search : [homenet.telecomitalia.it] addresses: 192.168.1.1 netplan apply and this is what happens : the network interface enp0s5 gets the IP number 192.168.1.7 ping 192.168.1.7 : OK ping 192.168.1.6 (FreeBSD IP) = destination host unreachable ping 192.168.1.1 : destination host unreachable PING google.it (172.217.21.67) : host unreachable tcpdump : listening on enp0s5,link-type EN10MB (Ethernet),capture size 262144 bytes 20:58:04:688913 ARP, request who-has 192.168.1.1 tell 192.168.1.7,length 28 route -n destination gateway genmask flags metric ref use iface 0.0.0.0 192.168.1.1 0.0.0.0 UG 100 0 0 enp0s5 169.254.0.0 0.0.0.0 255.255.0.0 U 1000 0 0 enp0s5 192.168.1.0 0.0.0.0 255.255.255.0 U 100 0 0 enp0s5 ip route list default via 192.168.1.1 dev enp0s5 proto static metric 100169.254.0.0/16 dev enp0s5 scope link metric 100192.168.1.0/24 dev enp0s5 proto kernel scope link src 192.168.1.7 metric 100 ping google.it = temporary error in name resolution firewall on ubuntu = not installed firewall on router = down instead,from freebsd to ubuntu : ping 192.168.1.7 : host is down netstar -r routing tables internet: destination gateway flags Netif Expire default modemtim UGS em0 localhost link#2 UH lo0192.168.1.0/24 link#1 U em0 192.168.1.6 link#1 UHS lo0 I forgot. This is what happens if I remove the netplan conf file and I let ubuntu choose the IP via DHCP : /etc/netplan.conf network: version: 2 renderer: networkd ethernets: enp3s0: dhcp4: true ping PING google.it (172.217.21.67) : network unreachable. What else ? PS : Since I'm not a real system administrator,but I do it as a hobby,don't give me generic suggestions because probably I will not understant what to do. If you want to be a kind person,try to be exhaustive and precise. You will help me a lot to understand how to fix the problem. thanks. PS : more failed attempt to fix the problem are listed here : https://forums.freebsd.org/threads/how-to-duplicate-the-tap0-interface-in-a-new-tap1-network-interface-that-it-is-not-used.81671/page-3 --00000000000074620f05c9c1ad86--