From nobody Thu Nov 16 14:22:20 2023 X-Original-To: freebsd-questions@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4SWMgW3NDPz50jMk for ; Thu, 16 Nov 2023 14:22:31 +0000 (UTC) (envelope-from mp@petermann-it.de) Received: from www444.your-server.de (www444.your-server.de [136.243.160.50]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4SWMgV0tDZz4phg for ; Thu, 16 Nov 2023 14:22:29 +0000 (UTC) (envelope-from mp@petermann-it.de) Authentication-Results: mx1.freebsd.org; dkim=none; spf=pass (mx1.freebsd.org: domain of mp@petermann-it.de designates 136.243.160.50 as permitted sender) smtp.mailfrom=mp@petermann-it.de; dmarc=none Received: from sslproxy04.your-server.de ([78.46.152.42]) by www444.your-server.de with esmtpsa (TLS1.3) tls TLS_AES_256_GCM_SHA384 (Exim 4.94.2) (envelope-from ) id 1r3dG9-000NaD-3R for freebsd-questions@freebsd.org; Thu, 16 Nov 2023 15:22:21 +0100 Received: from [217.92.37.201] (helo=mail.localdomain) by sslproxy04.your-server.de with esmtpsa (TLSv1.3:TLS_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1r3dG8-000Ruf-S0 for freebsd-questions@freebsd.org; Thu, 16 Nov 2023 15:22:20 +0100 Received: from [192.168.2.5] (unknown [192.168.2.5]) by mail.localdomain (Postfix) with ESMTPS id 60634260E7 for ; Thu, 16 Nov 2023 15:22:20 +0100 (CET) Message-ID: <8ada22f7-f25c-47e9-829e-c22a2630a691@petermann-it.de> Date: Thu, 16 Nov 2023 15:22:20 +0100 List-Id: User questions List-Archive: https://lists.freebsd.org/archives/freebsd-questions List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-questions@freebsd.org X-BeenThere: freebsd-questions@freebsd.org MIME-Version: 1.0 User-Agent: Mozilla Thunderbird From: Matthias Petermann To: freebsd-questions@freebsd.org Content-Language: de-DE Subject: Snapdir of Nullfs-mounted Dataset incomplete in Jail Content-Type: multipart/signed; protocol="application/pkcs7-signature"; micalg=sha-256; boundary="------------ms030901070404050301070104" X-Authenticated-Sender: mpeterma@petermann-it.de X-Virus-Scanned: Clear (ClamAV 0.103.10/27095/Thu Nov 16 09:40:23 2023) X-Spamd-Result: default: False [-5.39 / 15.00]; SIGNED_SMIME(-2.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_SHORT(-1.00)[-0.997]; R_SPF_ALLOW(-0.20)[+ip4:136.243.160.50/32]; MIME_GOOD(-0.20)[multipart/signed,text/plain]; XM_UA_NO_VERSION(0.01)[]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; DMARC_NA(0.00)[petermann-it.de]; FREEFALL_USER(0.00)[mp]; RCPT_COUNT_ONE(0.00)[1]; PREVIOUSLY_DELIVERED(0.00)[freebsd-questions@freebsd.org]; RCVD_VIA_SMTP_AUTH(0.00)[]; HAS_X_AS(0.00)[]; MLMMJ_DEST(0.00)[freebsd-questions@freebsd.org]; RCVD_COUNT_THREE(0.00)[3]; TO_DN_NONE(0.00)[]; HAS_ATTACHMENT(0.00)[]; ARC_NA(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:~]; R_DKIM_NA(0.00)[]; MID_RHS_MATCH_FROM(0.00)[]; ASN(0.00)[asn:24940, ipnet:136.243.0.0/16, country:DE]; RCVD_TLS_ALL(0.00)[] X-Rspamd-Queue-Id: 4SWMgV0tDZz4phg X-Spamd-Bar: ----- This is a cryptographically signed message in MIME format. --------------ms030901070404050301070104 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit Hello everyone, I have just made an observation regarding the interaction of ZFS snapshots, Jails, and Nullfs mounts. I have a FreeBSD 13.2-p5 system with a ZFS Root Filesystem. I am using Bastille to manage Jails. There is a Jail whose root filesystem is mounted from the dataset zroot/bastille/jails/dc/root to /usr/local/bastille/jails/dc/root. Furthermore, there is a dataset on the host named zroot/data/shares. I mount this dataset to the Jail's root filesystem using a Nullfs mount at /usr/local/bastille/jails/dc/root/shares. This works so far – I can access /shares within the Jail and effectively use the underlying dataset from both the host and the Jail. Now, I wanted to access the snapshots of zroot/data/shares from within the Jail. So, I quickly set the Snapdir property to visible. Within the Jail, I can now list the existing snapshots via /shares/.zfs/snapshot. Here's the problem: Snapshots are continuously created by Sanoid from the host. Over time, I now get different directory listings of snapdir depending on where I list it from: when listing from the host at /zroot/data/shares/.zfs/snapshot, everything is complete. In the Jail at /shares/.zfs/snapshot, the newer snapshots are missing. If I then go to the host and execute the following: ``` % doas ls -lah /zroot/data/shares/.zfs/snapshot/* ``` ...I get all the snapshots with sub-directories listed as expected. Surprisingly, after that, I can also see the snapshots from within the Jail. A similar issue is discussed in [1] and [2]. Where is my mistake in thinking? Best regards, Matthias [1] https://forums.freebsd.org/threads/zfs-snapshot-directory-access-operation-not-permitted-in-jail-on-freebsd13.80713/ [2] https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=260160 --------------ms030901070404050301070104 Content-Type: application/pkcs7-signature; name="smime.p7s" Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="smime.p7s" Content-Description: S/MIME Cryptographic Signature MIAGCSqGSIb3DQEHAqCAMIACAQExDzANBglghkgBZQMEAgEFADCABgkqhkiG9w0BBwEAAKCC Cc8wggSSMIIDeqADAgECAgh7b+hpj4hxdjANBgkqhkiG9w0BAQsFADBmMQswCQYDVQQGEwJE RTEzMDEGA1UECgwqREdOIERldXRzY2hlcyBHZXN1bmRoZWl0c25ldHogU2VydmljZSBHbWJI MSIwIAYDVQQDDBlkZ25zZXJ2aWNlIENBIDIgVHlwZSBFOlBOMB4XDTIzMDIyNzA5Mjg0N1oX DTI0MDIyNzA5Mjg0N1owcDELMAkGA1UEBhMCREUxITAfBgNVBAUTGDQwMDAwMDAwNjNmYzdh MmMyMzY3MDZjNDEbMBkGA1UEAwwSTWF0dGhpYXMgUGV0ZXJtYW5uMSEwHwYJKoZIhvcNAQkB FhJtcEBwZXRlcm1hbm4taXQuZGUwggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQCv UU/+BQYtChQynhczrQa1mva2NJubL2tUGxh5fzYdir3xRlXM/Y55e42uXJnFevtrdCjC/3gD i5y+aMQTpUvsNlUC+LiSTqLMYnPwRi83L7+tCip4vn/mhHSgnpnwQFsJ6dwYELLTIOa0Hfy8 34WD2U+ti71x4Ov2Ghq+FBX+lQusP8UZmvWP8OGzpn964cBWbAsmNk9JbOOf097U+fXs3cOX eqC5w+46iA+K/LMQY4TqCd7vw1X4eptT6dj/3CP9wxamdgu69C7TXpCLhjSet7k3f9nX2Gxa uqUCHbfxzzyRm7I9ytwpswQRZQYUZ5ztQYN0quOp/h5zk0rybKcFAgMBAAGjggE4MIIBNDAd BgNVHQ4EFgQU9XRgKeE4dGBY+hQKQCaHHxqp8ewwDAYDVR0TAQH/BAIwADAfBgNVHSMEGDAW gBTpxpPR1Q8GZHLqapY+uhDyVFSyeTBWBgNVHSAETzBNMEsGDCsGAQQB+ysCAQMCCDA7MDkG CCsGAQUFBwIBFi1odHRwOi8vc2VjNS5kZ25zZXJ2aWNlLmRlL3BvbGljaWVzL2luZGV4Lmh0 bWwwPgYDVR0fBDcwNTAzoDGgL4YtaHR0cDovL3NlYzUuZGduc2VydmljZS5kZS9jcmwvY3Js Mi10eXBlLWUuY3JsMA4GA1UdDwEB/wQEAwIEsDAdBgNVHSUEFjAUBggrBgEFBQcDAgYIKwYB BQUHAwQwHQYDVR0RBBYwFIESbXBAcGV0ZXJtYW5uLWl0LmRlMA0GCSqGSIb3DQEBCwUAA4IB AQAIuNIxzvPQ003CW1tA4MkfWTtkCHajiB/37Cd9txEu3ielx/KBRYcLYM30W07UkFDLrP2V qx/ct5NyRv9gFBvtk790gsy4+hYcEIn4y0Gr0fxIdHgAjlAFvPt0Rj+oweDpIyqJ8xtZiN8H zhgHq2OYOp911f/nbgZ36Swela3GWghr19ZHirFp9b63DH91mKcFzZPZ1QeaFQ8Wcvb6Hld3 6+MIbuai4eLX3nirv6fSCIXlPH+iI5iV2eJNZfU8jSpTrwv9sRpjrJjSVqOqoBYKuZU43g/n GKzADoJVVd/jGMNZka3Pfqd3JtAp2YZhh+d5S1ZpQFCCXnk6F6nY0C32MIIFNTCCBB2gAwIB AgIIVRxK12atJfYwDQYJKoZIhvcNAQELBQAwYTELMAkGA1UEBhMCREUxMzAxBgNVBAoMKkRH TiBEZXV0c2NoZXMgR2VzdW5kaGVpdHNuZXR6IFNlcnZpY2UgR21iSDEdMBsGA1UEAwwUZGdu c2VydmljZSBSb290IDc6UE4wHhcNMTYxMDI2MDkyMjQxWhcNMjQxMDI2MDkyMjQxWjBmMQsw CQYDVQQGEwJERTEzMDEGA1UECgwqREdOIERldXRzY2hlcyBHZXN1bmRoZWl0c25ldHogU2Vy dmljZSBHbWJIMSIwIAYDVQQDDBlkZ25zZXJ2aWNlIENBIDIgVHlwZSBFOlBOMIIBIjANBgkq hkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEA3KXylD90x6NH0pdmzmujzW0XA2GWhOGVd7yxq3v1 OOOTrEWoTkT3j//S+J8nEyun1GsKQ06jmg8bV2MY6CTQvS5ykcVQf1JAX9IdubzdC9YleCoQ mmPE4pldM9slEpW9jbmeIHQVOvaiZGrKmI/gD/DnEDqNInY/Ta9XpaBw99otCQz5IQY/FX+n Om+5jcp/Mn2WL2Zc81dokP3L6OohS8dsIBu5gpDmfAQBxgxcOk9FCANAZOtGIUTEcSOxl4zM QUANmP116D+Hb0Aw7TDZitK+Q1F6H/O8Nm613LbkNm+MTcBSBK1aAELvH7Z105vYjrWjrFsi zGV+r+bM2kAagQIDAQABo4IB6jCCAeYwEgYDVR0TAQH/BAgwBgEB/wIBADAfBgNVHSMEGDAW gBQBDBYbdHLTPBTuJne3SRRT2PuLGzBbBggrBgEFBQcBAQRPME0wSwYIKwYBBQUHMAGGP2h0 dHA6Ly9yb2NzcC1kZ24uZGduc2VydmljZS5kZTo4MDgwL2VqYmNhL3B1YmxpY3dlYi9zdGF0 dXMvb2NzcDBqBgNVHSAEYzBhMF8GDCsGAQQB+ysCAQQCATBPME0GCCsGAQUFBwIBFkFodHRw Oi8vd3d3LmRnbnNlcnZpY2UuZGUvdHJ1c3RjZW50ZXIvcHVibGljL2RnbnNlcnZpY2UvaW5k ZXguaHRtbDCBmQYDVR0fBIGRMIGOMIGLoIGIoIGFhoGCbGRhcDovL2xkYXAuZGduc2Vydmlj ZS5kZTozODkvQ049Q1JMLTEsTz1ER04lMjBTZXJ2aWNlJTIwR21iSCxDPURFP2NlcnRpZmlj YXRlUmV2b2NhdGlvbkxpc3Q/YmFzZT9vYmplY3RDbGFzcz1jUkxEaXN0cmlidXRpb25Qb2lu dDAdBgNVHQ4EFgQU6caT0dUPBmRy6mqWProQ8lRUsnkwDgYDVR0PAQH/BAQDAgEGMBsGCSsG AQQBwG0DBQQOMAwGCisGAQQBwG0DBQEwDQYJKoZIhvcNAQELBQADggEBAKu8OfpFyfv8U2rd I4k1+Bg1+R1HkKIagLixn6LR3KkKqoxfBulFCNKuItLZ4R7ZXYWazQThK4E4RrE6DWdWNhp9 BUa3K1FPS54UjMHdVydCXuhEUnRIeL13/GMSUBmSTwjyXK25YbntJpsDvZ24SnsBoJYbeVVk h00vL6uQ2t828RRupSHROtleCqnDAIsmNgXy/ILgj7HE1pebcSa42+BDaPZ0anc7FzRBt8ss 4Jsr+s50OKRoE3uROql8eWzulwg6FCuCs9UvNj80eob3MSmaO5Citrrdgd27sFPMan8O9xDB /Ipr+C1v0sefcChJSo4o4x3TaxRv8DEaGw2MzMkxggOCMIIDfgIBATByMGYxCzAJBgNVBAYT AkRFMTMwMQYDVQQKDCpER04gRGV1dHNjaGVzIEdlc3VuZGhlaXRzbmV0eiBTZXJ2aWNlIEdt YkgxIjAgBgNVBAMMGWRnbnNlcnZpY2UgQ0EgMiBUeXBlIEU6UE4CCHtv6GmPiHF2MA0GCWCG SAFlAwQCAQUAoIIB4TAYBgkqhkiG9w0BCQMxCwYJKoZIhvcNAQcBMBwGCSqGSIb3DQEJBTEP Fw0yMzExMTYxNDIyMjBaMC8GCSqGSIb3DQEJBDEiBCC/B+NPPsoQB/WvUJM4PguEUZCmJocT g8ag6a9lRpmNbzBsBgkqhkiG9w0BCQ8xXzBdMAsGCWCGSAFlAwQBKjALBglghkgBZQMEAQIw CgYIKoZIhvcNAwcwDgYIKoZIhvcNAwICAgCAMA0GCCqGSIb3DQMCAgFAMAcGBSsOAwIHMA0G CCqGSIb3DQMCAgEoMIGBBgkrBgEEAYI3EAQxdDByMGYxCzAJBgNVBAYTAkRFMTMwMQYDVQQK DCpER04gRGV1dHNjaGVzIEdlc3VuZGhlaXRzbmV0eiBTZXJ2aWNlIEdtYkgxIjAgBgNVBAMM GWRnbnNlcnZpY2UgQ0EgMiBUeXBlIEU6UE4CCHtv6GmPiHF2MIGDBgsqhkiG9w0BCRACCzF0 oHIwZjELMAkGA1UEBhMCREUxMzAxBgNVBAoMKkRHTiBEZXV0c2NoZXMgR2VzdW5kaGVpdHNu ZXR6IFNlcnZpY2UgR21iSDEiMCAGA1UEAwwZZGduc2VydmljZSBDQSAyIFR5cGUgRTpQTgII e2/oaY+IcXYwDQYJKoZIhvcNAQEBBQAEggEAFMogFHaxzZ8F+TVYMu7nvzHLsUhXYYoc+jrS EomKJnm5QAt+6zSMP5grkE/CiwUbkz8XXvhZ3rIv4HZzXwqX84Prndjl/TK1zqjxDgWMGq1I 6VcsJSrL+5hT64XtDXLhAgtHy2TprZPILIEE0Q0q5igyEFyX+hE2KH4pVReOz0HmgJrlIo2Y 2AO3uLZBx8Sl9QRr5KZDSu056mJMB2rTGsZaTVCw6tVqunzPlPjHnyP0nhc/lq7Z+tSO+7Fs OyZT0wBTTq2OtZBm2QrAelvM9FsHjzhP7PNabhaN2b/x/ahPCS7CbGnyKbIx4m4fzaLWeFP1 3GhIK3+X4DLwDVY9lAAAAAAAAA== --------------ms030901070404050301070104--