From nobody Fri Nov 08 09:25:49 2024 X-Original-To: bugs@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4XlD7y1zFWz5cSyQ for ; Fri, 08 Nov 2024 09:25:50 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R10" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4XlD7x6z3tz43Ms for ; Fri, 8 Nov 2024 09:25:49 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1731057950; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=0AyGzNY1la83gHEqeICO3zni6IrsHwuiq/lqUZkOVtY=; b=PAJf/rjM3hFjABSdPd0E49zNW0K4tADo3qLQgKvfqrGCRcGDgMAyJGCR/bvscy3gN6mZFE hw2MbOtUxxUmM+BNa2Dwm2yQ3KHm+Df7R0WXSLLk0baWAvokgWVTqXgKibBx91ytkj3bCa 8F6oyWLlpUvib/07Vf6j13UXkDFib+0qgutiBwOQNxu73+QyhMzM3505WFhcDADE+loeo2 8ZO4JEEUbMT4yghNQgNYCO7SvVfYYJUoGgzsEOzoJaGmd5ZgEoBuvWrhK7K0OOVqo34Q93 PAInbY6zaxU/SqtgfISHLaZ8iwE0daABWvtHBgdQqjLu27wjVQGfwG9gtvmmDg== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1731057950; a=rsa-sha256; cv=none; b=MlwAb94I/tjLHbcIto3kSsN+SKyljr/uuW5ALYmmA9Ep3YBbcfNlFYgPsZN7zho9ppabV2 aW5qQ/Bd1BQtxe0Os2DsFryic47+wq9aCEdCReuKrHKd/X8B9oA/uQnOLaxCOCgF86vwkL irLXmjiTM7uiEUcWcT6CiIXp28N1LT+ysR2fNIjxz/AjusADapvfK4dgrD6KtmsN3C3jO1 FAorVNTsRgNRmEbEIULf0v2Ng8oIEj0iZrLS0wVGh88mP02/vRabBpRD6k80I1n8vG2Y1P 3OhzHsloJYIAP8Y9Z8VYW/V+obE4/m77t5EAV1MiWvo4YwLaeTRKWZ+slS/2jA== Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2610:1c1:1:606c::50:1d]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4XlD7x6RvczNmk for ; Fri, 8 Nov 2024 09:25:49 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.5]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id 4A89Pnjb065453 for ; Fri, 8 Nov 2024 09:25:49 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id 4A89Pnki065452 for bugs@FreeBSD.org; Fri, 8 Nov 2024 09:25:49 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: bugs@FreeBSD.org Subject: [Bug 282621] NFSv4 Inside VNET Jail - Now Broken Date: Fri, 08 Nov 2024 09:25:49 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: new X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: bin X-Bugzilla-Version: 14.1-RELEASE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Many People X-Bugzilla-Who: vermaden@interia.pl X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: bugs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: bug_id short_desc product version rep_platform op_sys bug_status bug_severity priority component assigned_to reporter Message-ID: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated List-Id: Bug reports List-Archive: https://lists.freebsd.org/archives/freebsd-bugs List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-bugs@FreeBSD.org MIME-Version: 1.0 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D282621 Bug ID: 282621 Summary: NFSv4 Inside VNET Jail - Now Broken Product: Base System Version: 14.1-RELEASE Hardware: amd64 OS: Any Status: New Severity: Affects Many People Priority: --- Component: bin Assignee: bugs@FreeBSD.org Reporter: vermaden@interia.pl Hi, Rick Macklem was kind to implement the possibility to run NFS server nfsd(8= ) in a VNET Jail - as described in details in the FreeBSD 2022 Q3 Status Report: - https://freebsd.org/status/report-2022-10-2022-12/#_enable_the_nfs_server_t= o_run_in_a_vnet_prison Relevant commits: - https://freshbsd.org/freebsd/src/commit/bba7a2e89602e6745bb2ec474f5ab714aef= 49f42 - https://freshbsd.org/freebsd/src/commit/d4b4f3b9c356938de6140ccea20d502d207= b18a7 - https://freshbsd.org/freebsd/src/commit/6a76d35cac8e1549f74bd4cdceccc2ee52c= 8e556 - https://freshbsd.org/freebsd/src/commit/99187c3a44c2c3e168e462a30d45af07574= 8195f - https://freshbsd.org/freebsd/src/commit/7926a01ed7ae7cefd81ef4cc2142c35b84d= 81913 - https://freshbsd.org/freebsd/src/commit/7e44856e3a6deb194c2c376e886854b2563= 60c40 - https://freshbsd.org/freebsd/src/commit/4d68605f31fb536722529dc90f16cc47d96= 4882e - https://freshbsd.org/freebsd/src/commit/ab0440af75ba0c2dbf263c5441ccbe40585= 15fff - https://freshbsd.org/freebsd/src/commit/9d329bbc9aea6b8f47df251072bc65403ac= 3e43e - https://freshbsd.org/freebsd/src/commit/f0db2b6022dfa15f375f5fcdd278b9df21c= b88f5 - https://freshbsd.org/freebsd/src/commit/6444662a563ba714fed8563645764262c6f= 5e90f - https://freshbsd.org/freebsd/src/commit/2894c8c96b9b94f35aaa27ee5ef3ac11c27= 6fe3f - https://freshbsd.org/freebsd/src/commit/b039ca0776774036a9e33aa45f50296a7f6= ba547 - https://freshbsd.org/freebsd/src/commit/cd406ac94d8beae7f184adb14a3c94e0583= 66b9a - https://freshbsd.org/freebsd/src/commit/ed03776ca7f43de8275da80cfa89a9ecc47= 32f82 - https://freshbsd.org/freebsd/src/commit/ef6fcc5e2b0714c859d2e4ba23a55b1fd12= f8a4e - https://freshbsd.org/freebsd/src/commit/357492c99597d13bc966441f30bb44f6ef6= 59f08 - https://freshbsd.org/freebsd/src/commit/ef4e8f0cf91f7009745d5a7a90d3bdd2d9e= 25780 - https://freshbsd.org/freebsd/src/commit/4036fcb8053adf3ac54c8428eef0dd076df= c1718 - https://freshbsd.org/freebsd/src/commit/10dff9da9748b0eadd2d02dded3afd2321d= 15537 - https://freshbsd.org/freebsd/src/commit/a90b47abcbdbd48c68fbf7c407546293479= 056de - https://freshbsd.org/freebsd/src/commit/57ff348804f98d956f2e203b665de5a8989= dbf8c - https://freshbsd.org/freebsd/src/commit/4bbbd5875d32f3cbe76235d90243f713eff= 9b9d0 - https://freshbsd.org/freebsd/src/commit/0bb08f21cc5c62d0e2dfcea500521fa8010= 58dd3 - https://freshbsd.org/freebsd/src/commit/84eac070494d81d0e0ded098a0275791874= e5251 - https://freshbsd.org/freebsd/src/commit/9432e798fc6daaad341a496e9abcf9e3b76= 0a63b - https://freshbsd.org/freebsd/src/commit/a1254dcaa869bba20e46d966c53c7473bb2= 4d02b I even created article how to use it:: - https://vermaden.wordpress.com/2023/07/01/nfsv4-server-inside-freebsd-vnet-= jail/ Rick also made important setup guide: - https://people.freebsd.org/%7Ermacklem/nfsd-vnet-prison-setup.txt I contacted Rick to make sure that I did not do any misconfiguration or something like that - an we came to a conclusion that some newer commit/cha= nge must have broke that. I just tried to run (Again) NFSv4 server inside FreeBSD VNET Jail ... and I= can not even mount the NFS share ... this is on 14.1-RELEASE. These are exported NFS shares. nfsd # cat /etc/exports=20 V4: / -sec=3Dsys -network 10.0.0.0/24 /share -sec=3Dsys -maproot=3Droot -network 10.0.0.0/24 I can not mount on 'other' system: poudriere root ~ # mount -o nfsv4 10.1.1.99:/share /mnt mount_nfs: nmount: /mnt: Permission denied ... or even on the nfsd(8) server: nfsd # mount -o nfsv4 10.1.1.99:/share /mnt mount_nfs: nmount: /mnt: Operation not permitted After checking with tcpdump(8) the interesting error message seems to be th= is one: - NFS reply xid 2939613111 reply ERR 20: Auth Credentials are too weak Complete dump below. nfsd # tcpdump -n port 2049 tcpdump: verbose output suppressed, use -v[v]... for full protocol decode listening on epair99b, link-type EN10MB (Ethernet), snapshot length 262144 bytes 18:27:49.336751 IP 10.1.1.123.619 > 10.1.1.99.2049: Flags [S], seq 1126678540, win 65535, options [mss 1460,nop,wscale 6,sackOK,TS val 4116429320 ecr 0], length 0 18:27:49.336788 IP 10.1.1.99.2049 > 10.1.1.123.619: Flags [S.], seq 2606098797, ack 1126678541, win 65535, options [mss 1460,nop,wscale 6,sackOK,TS val 1369673091 ecr 4116429320], length 0 18:27:49.336986 IP 10.1.1.123.619 > 10.1.1.99.2049: Flags [.], ack 1, win 1027, options [nop,nop,TS val 4116429320 ecr 1369673091], length 0 18:27:49.337118 IP 10.1.1.123.619 > 10.1.1.99.2049: Flags [P.], seq 1:45, ack 1, win 1027, options [nop,nop,TS val 4116429320 ecr 1369673091], length 44: NFS request xid 3699330492 40 null 18:27:49.337138 IP 10.1.1.99.2049 > 10.1.1.123.619: Flags [.], ack 45, win 29128, options [nop,nop,TS val 1369673091 ecr 4116429320], length 0 18:27:49.337155 IP 10.1.1.99.2049 > 10.1.1.123.619: Flags [P.], seq 1:29, ack 45, win 29128, options [nop,nop,TS val 1369673091 ecr 4116429320], length 28: NFS reply xid 3699330492 reply ok 24 null 18:27:49.337385 IP 10.1.1.123.619 > 10.1.1.99.2049: Flags [F.], seq 45, ack 29, win 1027, options [nop,nop,TS val 4116429320 ecr 1369673091], length 0 18:27:49.337401 IP 10.1.1.99.2049 > 10.1.1.123.619: Flags [.], ack 46, win 29128, options [nop,nop,TS val 1369673091 ecr 4116429320], length 0 18:27:49.337423 IP 10.1.1.99.2049 > 10.1.1.123.619: Flags [F.], seq 29, ack 46, win 29128, options [nop,nop,TS val 1369673091 ecr 4116429320], length 0 18:27:49.337584 IP 10.1.1.123.619 > 10.1.1.99.2049: Flags [.], ack 30, win 1027, options [nop,nop,TS val 4116429320 ecr 1369673091], length 0 18:27:49.337800 IP 10.1.1.123.620 > 10.1.1.99.2049: Flags [S], seq 42080079, win 65535, options [mss 1460,nop,wscale 6,sackOK,TS val 106674985 ecr 0], length 0 18:27:49.337829 IP 10.1.1.99.2049 > 10.1.1.123.620: Flags [S.], seq 4274899720, ack 42080080, win 65535, options [mss 1460,nop,wscale 6,sackOK,TS val 2800873825 ecr 106674985], length 0 18:27:49.337996 IP 10.1.1.123.620 > 10.1.1.99.2049: Flags [.], ack 1, win 1027, options [nop,nop,TS val 106674985 ecr 2800873825], length 0 18:27:49.338055 IP 10.1.1.123.620 > 10.1.1.99.2049: Flags [P.], seq 1:281, ack 1, win 4352, options [nop,nop,TS val 106674985 ecr 2800873825], length 280: NFS request xid 2939613111 276 getattr fh 0,10/1229193216 18:27:49.338071 IP 10.1.1.99.2049 > 10.1.1.123.620: Flags [.], ack 281, win 29128, options [nop,nop,TS val 2800873825 ecr 106674985], length 0 18:27:49.338090 IP 10.1.1.99.2049 > 10.1.1.123.620: Flags [P.], seq 1:25, ack 281, win 29128, options [nop,nop,TS val 2800873825 ecr 106674985], length 24: NFS reply xid 2939613111 reply ERR 20: Auth Credentials are too weak 18:27:49.338341 IP 10.1.1.123.620 > 10.1.1.99.2049: Flags [F.], seq 281, ack 25, win 4352, options [nop,nop,TS val 106674985 ecr 2800873825], length 0 18:27:49.338356 IP 10.1.1.99.2049 > 10.1.1.123.620: Flags [.], ack 282, win 29128, options [nop,nop,TS val 2800873825 ecr 106674985], length 0 18:27:49.338363 IP 10.1.1.99.2049 > 10.1.1.123.620: Flags [F.], seq 25, ack 282, win 29128, options [nop,nop,TS val 2800873825 ecr 106674985], length 0 18:27:49.338496 IP 10.1.1.123.620 > 10.1.1.99.2049: Flags [.], ack 26, win 4352, options [nop,nop,TS val 106674985 ecr 2800873825], length 0 This is the VNET Jail config. % cat /etc/jail.conf.d/nfsd.conf=20 nfsd { # GLOBAL exec.start =3D "/bin/sh /etc/rc"; exec.stop =3D "/bin/sh /etc/rc.shutdown"; exec.consolelog =3D "/var/log/jail_console_${name}.log"; exec.clean; mount.devfs; host.hostname =3D ${name}; path =3D /jail/${name}; # PERMISSIONS allow.raw_sockets; devfs_ruleset =3D 110; # NFSD/VNET allow.nfsd; enforce_statfs =3D 1; # VNET/VIMAGE vnet; vnet.interface =3D "${if}b"; # NETWORKS/INTERFACES $id =3D "99"; $ip =3D "10.1.1.${id}/24"; $gw =3D "10.1.1.1"; $br =3D "vm-public"; $if =3D "epair${id}"; # ADD TO bridge0 INTERFACE exec.prestart +=3D "ifconfig ${if} create up"; exec.prestart +=3D "ifconfig ${if}a up descr jail:${name}"; exec.prestart +=3D "ifconfig ${br} addm ${if}a up"; exec.start +=3D "ifconfig ${if}b ${ip} up"; exec.start +=3D "route add default ${gw}"; exec.poststop +=3D "ifconfig ${if}a destroy"; } As 14.2-RELEASE is month away from release it would be great to fix that be= fore release. Thanks, vermaden --=20 You are receiving this mail because: You are the assignee for the bug.=