From nobody Wed Oct 04 13:33:33 2023 X-Original-To: bugs@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4S0wcs61wLz4vxC8 for ; Wed, 4 Oct 2023 13:33:33 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4S0wcs4M1Fz3XN2 for ; Wed, 4 Oct 2023 13:33:33 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1696426413; a=rsa-sha256; cv=none; b=DrvodGfviKRUcWY8UYsfRcsCghDBC7zNSswoKL5HxFKfma2Sut5GlLJuciWRGZe6vJcPDt R4fns09I29dTE86hydjsYrfgAgJTaOLrVoKzb0JN2fd93ILylZ+qnnT12s59ICFLYrZxqH jYc7XVVgaGocL2mFCneqONvskqhUGc245lT0uo6e2ndHkVy7mdjzjbPVCLNhSMM+UaZ7Qi QE20meLRXUfqWh+PPAsNCrInFL/g6XrKPe2Zpf5gNoluRCKIGvtgT0u1piIfbSs348dYvR 2WjBKg/xaq3U3/VO8Z2dGHdspPD1I1FDDtBa1fAbyWv4PeZV+mtQLKHxn4ntlg== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1696426413; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=5tUE8WwJQutwcMFFZcom8Rminbj1RVN0J2XCXuHhRhg=; b=wXZqCQiTKkqHjY60lKY677YHhDhkM6re5otE8FFO2yO0UoI8KTzpyCxZRlkWL/M/w1nuXA 55W4OV+0E/qVtnxvT2SYb/JsPTDA9ZhAh/KM1Y9Bs+BpvjZ5eqYnJM7h7kAVrJaYy4v09W kjdiMFibQjxK/bKPwrTeU227t/JbKeyKlnl8tAguKBR9PEryuu4rYIGoA1Xbqqy9ZB6b3f qwXcYiYMQFWg5KakKuLH94njHiZabkSxf6ROroV/eQJM8N0iBvWgMrjGcCj387PmFlhS7N MSHQijhlnhWlQ3+QFh5SL5GpHL9xsDwP/TjUSG9oK8J1cwh4sVBmShmkWr9qAw== Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2610:1c1:1:606c::50:1d]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4S0wcs3QcnzfHH for ; Wed, 4 Oct 2023 13:33:33 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.5]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id 394DXX2h062700 for ; Wed, 4 Oct 2023 13:33:33 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id 394DXXwx062699 for bugs@FreeBSD.org; Wed, 4 Oct 2023 13:33:33 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: bugs@FreeBSD.org Subject: [Bug 274263] Access to zfs snapshots within a jail return EPERM after a while of operation Date: Wed, 04 Oct 2023 13:33:33 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: new X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 13.2-RELEASE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: freebsd-bugs@virtualtec.ch X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: bugs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: bug_id short_desc product version rep_platform op_sys bug_status bug_severity priority component assigned_to reporter Message-ID: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated List-Id: Bug reports List-Archive: https://lists.freebsd.org/archives/freebsd-bugs List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-bugs@freebsd.org MIME-Version: 1.0 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D274263 Bug ID: 274263 Summary: Access to zfs snapshots within a jail return EPERM after a while of operation Product: Base System Version: 13.2-RELEASE Hardware: Any OS: Any Status: New Severity: Affects Only Me Priority: --- Component: kern Assignee: bugs@FreeBSD.org Reporter: freebsd-bugs@virtualtec.ch Overview: - host-system with jails, that run hosting environments (webs etc) - jails run in their own zfs datasets, but filesystems are managed by the h= ost, not by the jails - daily zfs snapshots provide jail content of the previous days - customers are taught to, and use these snapshots for easy access to previ= ous content - some time after a reboot (I'm not aware of the specific trigger), access = to /.zfs/snapshot/{SPECIFICSNAP} from within the jail fails with EPERM, no matter whether the user is unprivileged or root. Listing of the snapshot directory itself (/.zfs/snapshot) works fine. - I found that if I list that snapshot directory from the host system (even= as unprivileged user), access from within the jail works fine again (for a while) Specifics (possibly too verbose, sorry): - current host system: 13.1-RELEASE-p6 - sample zfs layout: tank/services/jails/reseller123 tank/services/jails/reseller123/chrimg/someweb.com tank/services/jails/reseller123/chrimg/someweb.com/tmp - corresponding zfs filesystem mountpoints: /services/jails/reseller123 /services/jails/reseller123/services/webs/someweb.com/system /services/jails/reseller123/services/webs/someweb.com/system/tmp - we clone specific php-version based template filesystems to a system directory within a web root to implement different php environments. these clones are all read-only - there's an additional devfs and a nullfs mount per web: devfs on /services/jails/reseller123/services/webs/someweb.com/dev /services/addons nullfs read-only to /services/jails/reseller123/services/webs/someweb.com/system/services/addons the nullfs mount grants access to common typo3 version templates the cust= omer uses within webs - customer accesses /.zfs/snapshot via a symlink we provide in /services/webs/backup, but the observed behavior happens via direct access as well as via symlink, so I don't think this is relevant In the failure case, I get this: $ ls /services/webs/backup/2023-10-04-00:00:04 ls: /services/webs/backup/2023-10-04-00:00:04: Operation not permitted or with ktrace: ... 34393 ls CALL fstatat(AT_FDCWD,0x8006df098,0x7fffffffe2b0,0) 34393 ls NAMI "/services/webs/backup/2023-10-04-00:00:04" 34393 ls RET fstatat -1 errno 1 Operation not permitted 34393 ls CALL=20 fstatat(AT_FDCWD,0x8006df098,0x7fffffffe2b0,0x200) 34393 ls NAMI "/services/webs/backup/2023-10-04-00:00:04" 34393 ls RET fstatat -1 errno 1 Operation not permitted Now, if I do the same ls on the host system (also as unprivileged user), it works fine. And as soon as I've done that, the jail access works again. Could there be permission issues with nullfs in jails, such as in this setu= p?=20 Unfortunately, I've so far not found a way to forcibly cause the failure. --=20 You are receiving this mail because: You are the assignee for the bug.=