NFS panic: newnfs_copycred: negative nfsc_ngroups (client HEAD r253033, server 9.1-R)

[Bryan Drewery]

I received this panic on the client while doing heavy parallel
reads/writes over NFS. I only recently moved these files to NFS, so I
don't know whether or not it's a recent regression.

Client: HEAD r253033
Server: 9.1-R

core.txt: http://people.freebsd.org/~bdrewery/nfs.txt

fstab of related paths:

> tank:/tank/distfiles/freebsd                        /mnt/distfiles      nfs             rw,bg,noatime,intr,rsize=65536,wsize=65536,readahead=8,nfsv4    0       0
> tank:/usr/packages/                                 /mnt/all-packages   nfs             rw,bg,noatime,soft,retrycnt=3,rsize=65536,wsize=65536,readahead=8,nfsv4    0       0

Server: params on these paths: -maproot=root -network 10.10.0.0/16

tcpdump at the time:

> 21:43:05.396585 IP 10.10.0.7.4180315003 > 10.10.0.5.2049: 168 getattr fh 0,4/2
> 21:43:05.396589 IP 10.10.0.5.2049 > 10.10.0.7.946: Flags [.], seq 48265029:48266477, ack 4394885, win 29124, options [nop,nop,TS val 1950216660 ecr 596674], length 1448
> 21:43:05.396603 IP 10.10.0.5.2049 > 10.10.0.7.946: Flags [.], seq 48266477:48267925, ack 4394885, win 29124, options [nop,nop,TS val 1950216660 ecr 596674], length 1448
> 21:43:05.396605 IP 10.10.0.7.946 > 10.10.0.5.2049: Flags [.], ack 48266477, win 3916, options [nop,nop,TS val 596674 ecr 1950216660], length 0
> 21:43:05.396608 IP 10.10.0.5.2049 > 10.10.0.7.946: Flags [.], seq 48267925:48269373, ack 4394885, win 29124, options [nop,nop,TS val 1950216660 ecr 596674], length 1448
> 21:43:05.396621 IP 10.10.0.5.2049 > 10.10.0.7.946: Flags [.], seq 48269373:48270821, ack 4394885, win 29124, options [nop,nop,TS val 1950216660 ecr 596674], length 1448
> 21:43:05.396624 IP 10.10.0.7.946 > 10.10.0.5.2049: Flags [.], ack 48269373, win 3870, options [nop,nop,TS val 596674 ecr 1950216660], length 0
> 21:43:05.396641 IP 10.10.0.5.2049 > 10.10.0.7.946: Flags [.], seq 48270821:48272269, ack 4394885, win 29124, options [nop,nop,TS val 1950216660 ecr 596674], length 1448
> 21:43:05.396653 IP 10.10.0.5.2049 > 10.10.0.7.946: Flags [.], seq 48272269:48273717, ack 4394885, win 29124, options [nop,nop,TS val 1950216660 ecr 596674], length 1448
> 21:43:05.396656 IP 10.10.0.7.946 > 10.10.0.5.2049: Flags [.], ack 48272269, win 3825, options [nop,nop,TS val 596674 ecr 1950216660], length 0
> 21:43:05.396659 IP 10.10.0.5.2049 > 10.10.0.7.946: Flags [.], seq 48273717:48275165, ack 4394885, win 29124, options [nop,nop,TS val 1950216660 ecr 596674], length 1448
> 21:43:05.396671 IP 10.10.0.5.2049 > 10.10.0.7.946: Flags [.], seq 48275165:48276613, ack 4394885, win 29124, options [nop,nop,TS val 1950216660 ecr 596674], length 1448
> 21:43:05.396674 IP 10.10.0.7.946 > 10.10.0.5.2049: Flags [.], ack 48275165, win 3780, options [nop,nop,TS val 596674 ecr 1950216660], length 0
> 21:43:05.396676 IP 10.10.0.5.2049 > 10.10.0.7.946: Flags [.], seq 48276613:48278061, ack 4394885, win 29124, options [nop,nop,TS val 1950216660 ecr 596674], length 1448
> 21:43:05.396689 IP 10.10.0.5.2049 > 10.10.0.7.946: Flags [.], seq 48278061:48279509, ack 4394885, win 29124, options [nop,nop,TS val
> Write failed: Broken pipe

I have nfsuserd running on both client/server. nfscbd is running.
nfs_client_enable=yes in rc.conf.

User lookups seem to work fine:

> -rw-r--r--  1 root  bryan  1554804 Jul  6 10:50 /mnt/distfiles/pkg-1.1.4.tar.xz

I ran a find -ls on these paths and all files return a user/group. I am
guessing there is a race condition with files being written and looking
up the associated groups.

-- 
Regards,
Bryan Drewery
bdrewery at freenode/EFNet