From nobody Thu Jun 02 18:35:22 2022 X-Original-To: freebsd-net@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 9895519E902C for ; Thu, 2 Jun 2022 18:35:25 +0000 (UTC) (envelope-from jmg@gold.funkthat.com) Received: from gold.funkthat.com (gold.funkthat.com [IPv6:2001:470:800b::2]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "gate2.funkthat.com", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4LDZSr4H9hz3lFb for ; Thu, 2 Jun 2022 18:35:24 +0000 (UTC) (envelope-from jmg@gold.funkthat.com) Received: from gold.funkthat.com (localhost [127.0.0.1]) by gold.funkthat.com (8.15.2/8.15.2) with ESMTPS id 252IZM4E021212 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Thu, 2 Jun 2022 11:35:22 -0700 (PDT) (envelope-from jmg@gold.funkthat.com) Received: (from jmg@localhost) by gold.funkthat.com (8.15.2/8.15.2/Submit) id 252IZMSE021210; Thu, 2 Jun 2022 11:35:22 -0700 (PDT) (envelope-from jmg) Date: Thu, 2 Jun 2022 11:35:22 -0700 From: John-Mark Gurney To: Rick Macklem Cc: "freebsd-net@FreeBSD.org" Subject: Re: make NFSv3 default now on diskless Message-ID: <20220602183522.GZ88842@funkthat.com> Mail-Followup-To: Rick Macklem , "freebsd-net@FreeBSD.org" References: <20220602032456.GY88842@funkthat.com> List-Id: Networking and TCP/IP with FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-net List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-net@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-Operating-System: FreeBSD 11.3-STABLE amd64 X-PGP-Fingerprint: D87A 235F FB71 1F3F 55B7 ED9B D5FF 5A51 C0AC 3D65 X-Files: The truth is out there X-URL: https://www.funkthat.com/ X-Resume: https://www.funkthat.com/~jmg/resume.html X-TipJar: bitcoin:13Qmb6AeTgQecazTWph4XasEsP7nGRbAPE X-to-the-FBI-CIA-and-NSA: HI! HOW YA DOIN? can i haz chizburger? User-Agent: Mutt/1.6.1 (2016-04-27) X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.4.3 (gold.funkthat.com [127.0.0.1]); Thu, 02 Jun 2022 11:35:22 -0700 (PDT) X-Rspamd-Queue-Id: 4LDZSr4H9hz3lFb X-Spamd-Bar: - Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=none (mx1.freebsd.org: domain of jmg@gold.funkthat.com has no SPF policy when checking 2001:470:800b::2) smtp.mailfrom=jmg@gold.funkthat.com X-Spamd-Result: default: False [-1.25 / 15.00]; TO_DN_EQ_ADDR_SOME(0.00)[]; ARC_NA(0.00)[]; FREEFALL_USER(0.00)[jmg]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-0.997]; MID_RHS_MATCH_FROM(0.00)[]; MIME_GOOD(-0.10)[text/plain]; DMARC_NA(0.00)[funkthat.com]; AUTH_NA(1.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; TO_MATCH_ENVRCPT_SOME(0.00)[]; NEURAL_HAM_SHORT(-0.45)[-0.454]; RCPT_COUNT_TWO(0.00)[2]; MLMMJ_DEST(0.00)[freebsd-net]; FORGED_SENDER(0.30)[jmg@funkthat.com,jmg@gold.funkthat.com]; R_SPF_NA(0.00)[no SPF record]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:6939, ipnet:2001:470::/32, country:US]; FROM_NEQ_ENVFROM(0.00)[jmg@funkthat.com,jmg@gold.funkthat.com]; RCVD_TLS_ALL(0.00)[]; RCVD_COUNT_TWO(0.00)[2] X-ThisMailContainsUnwantedMimeParts: N Rick Macklem wrote this message on Thu, Jun 02, 2022 at 14:44 +0000: > John-Mark Gurney wrote: > > I just booted FreeBSD-current diskless, using NFS root, and I ended > > up having issues because by default, NFS root is only v2. > > > > One of things that happened was disk space available was listed as > > -138G, or -144830429K. I assume this is because the server is reporting > > TBs instead. > Yes. NFSv2 uses 32bit sizes. Should we make the NFS server clamp various sizes then? instead of reporting negative numbers? (Sorry if this is already done on newer versions of FreeBSD, my server is a bit old.) > > If I mount via mount_nfs, the sizes are normal/correct because it mounts > > v3. > I believe most specify "nfsv3" in the "/" mount line of /etc/fstab on the > remote root fs. Then, when the system does a "mount -u" to make it > read/write it gets toggled to NFSv3. well, I tried doing a: mount -u -o nfsv3 / on the system and this didn't work switch over to v3, it was still on v2.. I haven't specified nfsv3 in /etc/fstab, but IMO, this should be the default.. Also, I'm right now booting single user mode, because I'm -mapall to a user, and lots of FreeBSD breaks when files aren't uid 0, and there doesn't appear to be a way to remap the uid to root (that I have found).. > > The other issue that I ran into is that NFSv2 can't access >4GB files > > (or create them). > As above, NFSv2 uses 32bit sizes. > > > Anyone object to adding BOOTP_NFSV3 to GENERIC? > Well, that option only works when used with BOOTP_NFSROOT. > The GENERIC configs for amd64, arm64,... use the other way. > (Just to make it confusing, there are two different ways an NFS root > fs is set up.) > See below. > > > Or maybe making it a > > tunable that defaults to set, because it seems a bit crazy to default > > to v2 these days. > I don't think changing the default to NFSv3 will be a problem. > The reason it was NFSv2 was that, > for some non-FreeBSD NFS servers, the NFSv3 file handle is different > than the NFSv2 one. > > I added NFSv3 support to stand/libsa/nfs.c about 15years ago, so every > system should be running the newer NFS code in the loader and be able > to do NFSv3 booting. > > > This option was added in 432aad0e in 1997 so that the nfs_diskless > > structure didn't need to be filled out. Does anything even > > populate/fill it out anymore? I saw code in i386/i386/locore.s that > > does this, but it doesn't appear anywhere else. > Yes. For "options NFS_ROOT" (the other way), the loader uses > "stand/libsa/nfs.c" to acquire the remote file system's root file handle > and fills it in. (See nfs_setup_diskless() in sys/nfs/nfs_diskless.c.) > Looking at it, it appears to enable NFSv3 so long as it finds > "boot.nfsroot.nfshandlelen" set. Well, I'm not seeing that, and this system is booting via pxeboot+loader, so maybe something is broken? > > There also appears to possibly be a way via mount options, but I can't > > see where it's documented to set them. > I think you just specify "nfsv3" as a mount option in the root fs > line in /etc/fstab on the root fs on the NFS server. See above, this doesn't appear to work, or doesn't work the way I think it should... > I don't think changing the default to NFSv3 will be a problem. > The hassle is testing the various cases, to make sure nothing > breaks. I have no diskless setup to do testing and I don't even know > when installs/upgrades actually replace the loader? Well, this diskless was easier to setup than I expected, partly because I already had most of the infrastructure together (from netbooting another machine). Put pxeboot on a tftp server, configure the dhcp server to send the correct options, extract base.txz to a directory, export it, and it worked. I assume that I'm getting loader from that install since I don't specify it in the dhcp server. As for testing, we have the CI system for that, right? ;p /me needs to get back to work on the lab. I guess we'd need to list the configurations that we care about, the only ones I can think of, off the top of my head are pxeboot (which I'm testing now), and u-boot.. For servers, are there any servers that are NFSv2 only that are in common use today? If you're running an ancient server that is NFSv2 only, I think you deserve to have to rebuild kernels or something instead of making 99% of the rest of us do it.. -- John-Mark Gurney Voice: +1 415 225 5579 "All that I will do, has been done, All that I have, has not."