From nobody Fri Jul 23 21:06:41 2021 X-Original-To: freebsd-jail@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id E5C2012AFD8B for ; Fri, 23 Jul 2021 21:06:49 +0000 (UTC) (envelope-from infoomatic@gmx.at) Received: from mout.gmx.net (mout.gmx.net [212.227.17.22]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange ECDHE (P-256) server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "mout.gmx.net", Issuer "TeleSec ServerPass Class 2 CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4GWhhT4Vgzz3FKb for ; Fri, 23 Jul 2021 21:06:48 +0000 (UTC) (envelope-from infoomatic@gmx.at) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=gmx.net; s=badeba3b8450; t=1627074401; bh=JGJcrwvYzeRDi1EbESORSUOduG6KnbuYxq1ZBvLDlDY=; h=X-UI-Sender-Class:Subject:To:References:From:Date:In-Reply-To; b=HCUvJ9/qX4fjcd5UrrVFvuy9IJCzfXhc/IlmPKOSlU7ykHHErTjKrWv1eK3JrsMQ3 LU5H2rvqm7OvVnwlzGBlIm3fPclyWXN48uoS/vlMBQZqFjBA/N+LuxGOiWWJY5gPZe yToDNqrlwhSP1jky19zTFGvDEQnJC1xncMPSRHIo= X-UI-Sender-Class: 01bb95c1-4bf8-414a-932a-4f6e2808ef9c Received: from [10.0.1.209] ([178.114.235.19]) by mail.gmx.net (mrgmx105 [212.227.17.168]) with ESMTPSA (Nemesis) id 1MVvLB-1lhUbU2RjZ-00Ru2k for ; Fri, 23 Jul 2021 23:06:41 +0200 Subject: Re: iocage, vnet jail does not go outside To: freebsd-jail@freebsd.org References: From: infoomatic Message-ID: <40b7782d-9d5c-099a-ed58-4476b3523d7a@gmx.at> Date: Fri, 23 Jul 2021 23:06:41 +0200 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:78.0) Gecko/20100101 Thunderbird/78.12.0 List-Id: Discussion about FreeBSD jail(8) List-Archive: https://lists.freebsd.org/archives/freebsd-jail List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-jail@freebsd.org MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: quoted-printable Content-Language: en-US X-Provags-ID: V03:K1:NY+bEGTFnIf6y05ZxJVFJEp1SdYzujdNyCanCa34/OjcjItnQe7 VpgwrDQ7qRXtY2RdGVB9lRZTwPSM5bZB1ZTLbXlXv73XFPVbXNjl2X84FSCxn4BbY5/FSnI zNZzo/T0k9AzpNJol1V5QXEZXZMsrZ9//EdK7aLttzdRBEQhQxT7fMCLZ5g5IU3boWpO9zG VCzq6qT/a/jDP6ThWzjLQ== X-Spam-Flag: NO X-UI-Out-Filterresults: notjunk:1;V03:K0:uHo6IFMOqJ0=:tLt8yNZ5Bd5zKmwV9FEV7O 2ZBO0A01vy3agZedNxGKc/6yQuVGcaPUDTqEyxFMO5WsWk8jMWbOPqFQcST1by72WgXZDWKg1 pu+oZuX57Su+8NxrZxBQS2lbJSnuudbu8rHEPV4JL2VBFdB6DZzQD09m5rAsBHQLD8jU+Ium0 qEfJL7sco35fQMkFtZ3geo+obimj6MBd1pPz1ALOfJKqESSUZAawEHylP6v72sELpnXztmLGW N6fQEWqzV0fLSQjL3NNFI1kW73S7c8yU0eNj8gKHvBJbKxa1Hmtba6NDGemVQrTk48KWJ8Vej IT/6zvayzbdYnO6fS2aB5PKxa4XHuhkN1ydO2ea6Tysdgu4QVbSEd/OBg8I5OQqStzG3I+Ek9 ZaklNR96fC+++/vK+CqHYG537vGL4XXCWQ1X6E5Vd0fFD+LA+fOAh0Lvbki4JdaKHpTtaGJkr BSC5SUPAYiHFqvQ++JQPx7YGzcZHxEymX4tvM97QmhA5IKEYzYfznS4X2KsahH+hamkFonkAj pPnGScX9ySwNGJKkNRyAJsza8GkyAf/vC3pi5Blj3LoTqe7wIR6KptJrcJkuYMrHsXQJHs7SF pLWFMEU/HY9TOq19VHEDVOgAfn7nLfAuwxSluP3RJzfYG3ubuByoGbP2cCHNEL+cxrsKW+6eT HmOdb/bSWO5IJAgefuZG4emDQdV/c2CkgNJJSbVMR8EC2LuWj67Dn4r+Gm91afD1lBFAzV2D/ 8yOeCMV2LSTtVB4di7V47uZKs/OZhyXhAYaXqrMrWgcNu6KYG1PSL7I4+iSZ9/u2D/KKVrEFS UJR7hQU8YLsq5Gk4O9ul9fZew4VejYBHh3iBhLNcwvsCsHccVvpouyHmCU/cfsFRZwfjbheGM YbOpyq2MLoH1eK6M91l8RR+gKW3CGjgY5EkxSfPFnw2I0HefC5PRL0HPTLGnVj48tvsAB4AGr i4E1OQa4hhOq5qEM+83kTXtI8H7xOdv8hoSD/nZF9KRhFRaijGPmmfWcFwES4Cu7sYDNBUAii gP46zUqz+qTnybFIHz/cnXLXXeiiRQU4Sy9YPVWRfXQV8ElrnB1VRc97Fzxb+ByGpU7zmmxMo n9B8b/kW4kfuzTPZzwpd4TazdGkUEwb5oDm X-Rspamd-Queue-Id: 4GWhhT4Vgzz3FKb X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[] X-ThisMailContainsUnwantedMimeParts: N iocage autoatically creates a bridge with your physical interface and the vnet interface. Imho this is wrong behaviour so I quit using iocage, however, there is a workaround, for more info see [1] Regards, Robert [1] https://github.com/iocage/iocage/issues/521 On 23.07.21 18:36, Jacques Foucry wrote: > Hello friends, > > I'm turing crazy. > > I made a new jail ,on my hosted system using iocage. > > Here is the config.json file: > > more config.json > { > "allow_mount": 1, > "allow_mount_devfs": 1, > "allow_mount_nullfs": 1, > "allow_mount_procfs": 1, > "allow_mount_tmpfs": 1, > "allow_mount_zfs": 1, > "allow_raw_sockets": 1, > "allow_socket_af": 1, > "allow_sysvipc": 1, > "bpf": 1, > "cloned_release": "13.0-RELEASE", > "defaultrouter": "10.0.10.1", > "defaultrouter6": "auto", > "dhcp": 0, > "host_hostname": "examplejail", > "host_hostuuid": "examplejail", > "ip4_addr": "vnet0|10.0.10.23/24", > "ip6_addr": "vnet0|2a01:4f9:4a:1fd8::23", > "jail_zfs_dataset": "iocage/jails/examplejail/data", > "last_started": "2021-07-23 15:11:28", > "nat": 0, > "release": "13.0-RELEASE-p3", > "vnet": 1, > "vnet0_mac": "b42e999c5bca b42e999c5bcb", > "vnet_default_interface": "auto" > } > > The jail's ifconfig: > > ifconfig > lo0: flags=3D8049 metric 0 mtu 16384 > options=3D680003 > inet6 ::1 prefixlen 128 > inet6 fe80::1%lo0 prefixlen 64 scopeid 0x1 > inet 127.0.0.1 netmask 0xff000000 > groups: lo > nd6 options=3D21 > pflog0: flags=3D0<> metric 0 mtu 33160 > groups: pflog > epair0b: flags=3D8843 metric 0 m= tu 1500 > options=3D8 > ether b4:2e:99:9c:5b:cb > hwaddr 02:ae:46:07:62:0b > inet 10.0.10.23 netmask 0xffffff00 broadcast 10.0.10.255 > inet6 2a01:4f9:4a:1fd8::23 prefixlen 64 > inet6 fe80::b62e:99ff:fe9c:5bcb%epair0b prefixlen 64 scopeid 0x3 > groups: epair > media: Ethernet 10Gbase-T (10Gbase-T ) > status: active > nd6 options=3D21 > > The jail's netstat: > > netstat -rn > Routing tables > > Internet: > Destination Gateway Flags Netif Expire > default 10.0.10.1 UGS epair0b > 10.0.10.0/24 link#3 U epair0b > 10.0.10.23 link#3 UHS lo0 > 127.0.0.1 link#1 UH lo0 > > Internet6: > Destination Gateway Flags = Netif Expire > ::/96 ::1 UGRS = lo0 > default fe80::1%epair0b UGS = epair0b > ::1 link#1 UHS = lo0 > ::ffff:0.0.0.0/96 ::1 UGRS = lo0 > 2a01:4f9:4a:1fd8::/64 link#3 U = epair0b > 2a01:4f9:4a:1fd8::23 link#3 UHS = lo0 > fe80::/10 ::1 UGRS = lo0 > fe80::%lo0/64 link#1 U = lo0 > fe80::1%lo0 link#1 UHS = lo0 > fe80::%epair0b/64 link#3 U = epair0b > fe80::b62e:99ff:fe9c:5bcb%epair0b link#3 UHS = lo0 > ff02::/16 > > On the host, the ifconfig (note thereis a lot of old fashion jails): > > ifconfig > em0: flags=3D8963 metric= 0 mtu 1500 > options=3D4810099 > ether b4:2e:99:6a:80:9d > inet6 2a01:4f9:4a:1fd8::2 prefixlen 64 > inet6 fe80::b62e:99ff:fe6a:809d%em0 prefixlen 64 scopeid 0x1 > inet6 2a01:4f9:4a:1fd8::5 prefixlen 64 > inet6 2a01:4f9:4a:1fd8::11 prefixlen 64 > inet6 2a01:4f9:4a:1fd8::12 prefixlen 64 > inet6 2a01:4f9:4a:1fd8::15 prefixlen 64 > inet6 2a01:4f9:4a:1fd8::16 prefixlen 64 > inet6 2a01:4f9:4a:1fd8::18 prefixlen 64 > inet6 2a01:4f9:4a:1fd8::19 prefixlen 64 > inet6 2a01:4f9:4a:1fd8::21 prefixlen 64 > inet6 2a01:4f9:4a:1fd8::22 prefixlen 64 > inet6 2a01:4f9:4a:1fd8::25 prefixlen 64 > inet6 2a01:4f9:4a:1fd8::14 prefixlen 64 > inet6 2a01:4f9:4a:1fd8::29 prefixlen 64 > inet6 2a01:4f9:4a:1fd8::17 prefixlen 64 > inet 95.217.83.231 netmask 0xffffffc0 broadcast 95.217.83.255 > media: Ethernet autoselect (1000baseT ) > status: active > nd6 options=3D21 > lo0: flags=3D8049 metric 0 mtu 16384 > options=3D680003 > inet6 ::1 prefixlen 128 > inet6 fe80::1%lo0 prefixlen 64 scopeid 0x2 > inet 127.0.0.1 netmask 0xff000000 > inet 127.0.12.1 netmask 0xff000000 > inet 127.0.1.5 netmask 0xffffffff > inet 127.0.1.11 netmask 0xffffffff > inet 127.0.1.12 netmask 0xffffffff > inet 127.0.1.15 netmask 0xffffffff > inet 127.0.1.16 netmask 0xffffffff > inet 127.0.1.18 netmask 0xffffffff > inet 127.0.1.19 netmask 0xffffffff > inet 127.0.1.21 netmask 0xffffffff > inet 127.0.1.22 netmask 0xffffffff > inet 127.0.1.25 netmask 0xffffffff > inet 127.0.1.14 netmask 0xffffffff > inet 127.0.1.29 netmask 0xffffffff > inet 127.0.1.17 netmask 0xffffffff > groups: lo > nd6 options=3D21 > lo1: flags=3D8049 metric 0 mtu 16384 > options=3D680003 > inet 192.168.12.1 netmask 0xffffff00 > inet 192.168.12.5 netmask 0xffffffff > inet 192.168.12.11 netmask 0xffffff00 > inet 192.168.12.12 netmask 0xffffff00 > inet 192.168.12.15 netmask 0xffffff00 > inet 192.168.12.16 netmask 0xffffff00 > inet 192.168.12.18 netmask 0xffffff00 > inet 192.168.12.19 netmask 0xffffff00 > inet 192.168.12.21 netmask 0xffffff00 > inet 192.168.12.22 netmask 0xffffff00 > inet 192.168.12.25 netmask 0xffffff00 > inet 192.168.12.14 netmask 0xffffff00 > inet 192.168.12.29 netmask 0xffffff00 > inet 192.168.12.17 netmask 0xffffff00 > groups: lo > nd6 options=3D29 > pflog0: flags=3D100 metric 0 mtu 33160 > groups: pflog > bridge0: flags=3D8843 metric 0 m= tu 1500 > description: jails-bridge > ether 58:9c:fc:10:ed:66 > inet 10.0.10.1 netmask 0xffffff00 broadcast 10.0.10.255 > id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15 > maxage 20 holdcnt 6 proto rstp maxaddr 2000 timeout 1200 > root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0 > member: vnet0.655 flags=3D143 > ifmaxaddr 0 port 6 priority 128 path cost 2000 > member: em0 flags=3D143 > ifmaxaddr 0 port 1 priority 128 path cost 20000 > groups: bridge > nd6 options=3D9 > vnet0.655: flags=3D8943 = metric 0 mtu 1500 > description: associated with jail: examplejail as nic: epair0b > options=3D8 > ether b4:2e:99:9c:5b:ca > hwaddr 02:ae:46:07:62:0a > groups: epair > media: Ethernet 10Gbase-T (10Gbase-T ) > status: active > nd6 options=3D29 > > And host's netstat (again with many old fashion jail): > > netstat -rn > Routing tables > > Internet: > Destination Gateway Flags Netif Expire > default 95.217.83.193 UGS em0 > 10.0.10.0/24 link#5 U bridge0 > 10.0.10.1 link#5 UHS lo0 > 95.217.83.192/26 link#1 U em0 > 95.217.83.231 link#1 UHS lo0 > 127.0.0.1 link#2 UH lo0 > 127.0.1.5 link#2 UH lo0 > 127.0.1.11 link#2 UH lo0 > 127.0.1.12 link#2 UH lo0 > 127.0.1.14 link#2 UH lo0 > 127.0.1.15 link#2 UH lo0 > 127.0.1.16 link#2 UH lo0 > 127.0.1.17 link#2 UH lo0 > 127.0.1.18 link#2 UH lo0 > 127.0.1.19 link#2 UH lo0 > 127.0.1.21 link#2 UH lo0 > 127.0.1.22 link#2 UH lo0 > 127.0.1.25 link#2 UH lo0 > 127.0.1.29 link#2 UH lo0 > 127.0.12.1 link#2 UH lo0 > 192.168.12.1 link#3 UH lo1 > 192.168.12.5 link#3 UH lo1 > 192.168.12.11 link#3 UH lo1 > 192.168.12.12 link#3 UH lo1 > 192.168.12.14 link#3 UH lo1 > 192.168.12.15 link#3 UH lo1 > 192.168.12.16 link#3 UH lo1 > 192.168.12.17 link#3 UH lo1 > 192.168.12.18 link#3 UH lo1 > 192.168.12.19 link#3 UH lo1 > 192.168.12.21 link#3 UH lo1 > 192.168.12.22 link#3 UH lo1 > 192.168.12.25 link#3 UH lo1 > 192.168.12.29 link#3 UH lo1 > > Internet6: > Destination Gateway Flags = Netif Expire > ::/96 ::1 UGRS = lo0 > default fe80::1%em0 UGS = em0 > ::1 link#2 UHS = lo0 > ::ffff:0.0.0.0/96 ::1 UGRS = lo0 > 2a01:4f9:4a:1fd8::/64 link#1 U = em0 > 2a01:4f9:4a:1fd8::2 link#1 UHS = lo0 > 2a01:4f9:4a:1fd8::5 link#1 UHS = lo0 > 2a01:4f9:4a:1fd8::11 link#1 UHS = lo0 > 2a01:4f9:4a:1fd8::12 link#1 UHS = lo0 > 2a01:4f9:4a:1fd8::14 link#1 UHS = lo0 > 2a01:4f9:4a:1fd8::15 link#1 UHS = lo0 > 2a01:4f9:4a:1fd8::16 link#1 UHS = lo0 > 2a01:4f9:4a:1fd8::17 link#1 UHS = lo0 > 2a01:4f9:4a:1fd8::18 link#1 UHS = lo0 > 2a01:4f9:4a:1fd8::19 link#1 UHS = lo0 > 2a01:4f9:4a:1fd8::21 link#1 UHS = lo0 > 2a01:4f9:4a:1fd8::22 link#1 UHS = lo0 > 2a01:4f9:4a:1fd8::25 link#1 UHS = lo0 > 2a01:4f9:4a:1fd8::29 link#1 UHS = lo0 > fe80::/10 ::1 UGRS = lo0 > fe80::%em0/64 link#1 U = em0 > fe80::b62e:99ff:fe6a:809d%em0 link#1 UHS = lo0 > fe80::%lo0/64 link#2 U = lo0 > fe80::1%lo0 link#2 UHS = lo0 > ff02::/16 ::1 UGRS = lo0 > > The bridge0 had the em0 and vnet0:655 interfaces. > > From the jail in can ping oustside world: > > ping google.ca > PING6(56=3D40+8+8 bytes) 2a01:4f9:4a:1fd8::23 --> 2a00:1450:400f:803::20= 03 > 16 bytes from 2a00:1450:400f:803::2003, icmp_seq=3D0 hlim=3D118 time=3D7= .927 ms > 16 bytes from 2a00:1450:400f:803::2003, icmp_seq=3D1 hlim=3D118 time=3D7= .800 ms > 16 bytes from 2a00:1450:400f:803::2003, icmp_seq=3D2 hlim=3D118 time=3D7= .798 ms > ^C > --- google.ca ping6 statistics --- > 3 packets transmitted, 3 packets received, 0.0% packet loss > round-trip min/avg/max/std-dev =3D 7.798/7.842/7.927/0.061 ms > > The problem is, I cannot ssh to an external computer (for example, my > nextcloud hosted at home): > > ssh -vvv nextcloud.foucry.net -p2250 > OpenSSH_7.9p1, OpenSSL 1.1.1k-freebsd 25 Mar 2021 > debug1: Reading configuration data /etc/ssh/ssh_config > debug2: resolving "nextcloud.foucry.net" port 2250 > debug2: ssh_connect_direct > debug1: Connecting to nextcloud.foucry.net [2a01:e0a:434:44e0:ff:60ff:fe= ba:b582] port 2250. > debug1: connect to address 2a01:e0a:434:44e0:ff:60ff:feba:b582 port 2250= : Operation timed out > debug1: Connecting to nextcloud.foucry.net [82.65.174.130] port 2250. > debug1: connect to address 82.65.174.130 port 2250: Operation timed out > ssh: connect to host nextcloud.foucry.net port 2250: Operation timed out > > What's look strange (for me) is the traceroute (using ipv4): > > traceroute nextcloud.foucry.net > traceroute to nextcloud.foucry.net (82.65.174.130), 64 hops max, 40 byte= packets > 1 10.0.10.1 (10.0.10.1) 0.086 ms 0.051 ms 0.037 ms > 2 static.193.83.217.95.clients.your-server.de (95.217.83.193) 0.451 m= s 0.571 ms 0.392 ms > 3 core32.hel1.hetzner.com (213.239.252.97) 11.621 ms > core31.hel1.hetzner.com (213.239.252.93) 1.812 ms > core32.hel1.hetzner.com (213.239.252.97) 2.793 ms > 4 core9.fra.hetzner.com (213.239.224.166) 21.295 ms > core8.fra.hetzner.com (213.239.224.149) 20.730 ms > core9.fra.hetzner.com (213.239.224.170) 20.333 ms > 5 core4.fra.hetzner.com (213.239.245.85) 28.499 ms > core4.fra.hetzner.com (213.239.224.177) 20.507 ms 22.850 ms > 6 * * * > 7 * * * > 8 * * * > 9 * * * > 10 * * * > 11 * * * > 12 * * * > 13 *^C > > > Look's like something wrong on the way, but I could connect on the same = host > form any other jails. > > > There is for me a mysterious behaviiors that I can't understand. > > Any help will be appreciate. > > Thanks for reading me, and the time your spend on my problem.