From nobody Tue Feb 08 16:24:56 2022 X-Original-To: bugs@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id E86F419AB484 for ; Tue, 8 Feb 2022 16:24:56 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4JtSyw40RRz3t4P for ; Tue, 8 Feb 2022 16:24:56 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2610:1c1:1:606c::50:1d]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 6A5DF1C3D9 for ; Tue, 8 Feb 2022 16:24:56 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.5]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id 218GOubB092736 for ; Tue, 8 Feb 2022 16:24:56 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id 218GOuIt092735 for bugs@FreeBSD.org; Tue, 8 Feb 2022 16:24:56 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: bugs@FreeBSD.org Subject: [Bug 261291] ESX NFS4.1 client hangs, server never responds to EXCHANGE_ID/CREATE_SESSION Date: Tue, 08 Feb 2022 16:24:56 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 13.0-STABLE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: asomers@FreeBSD.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: bugs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated List-Id: Bug reports List-Archive: https://lists.freebsd.org/archives/freebsd-bugs List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-bugs@freebsd.org MIME-Version: 1.0 ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1644337496; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=onlTFekRBjglbkB4tsFmM4eV9XbGfgUKI+NOO+pnw5s=; b=xlL330V6HWd2PKCQKmhWxk3xQZ0wrMVuFc6gtoZu23I33zu00BYzfyBFaAzX82q6Yrwsg9 0uhX+hY6ZI6OsYcg56xUniVQ549wJNA/UKcvvPHbilt1DSz80JpBiEafOeA4ArvK+Ck1zG JcMjCbFWAVu6KTyhvHui12/oBrPSRq5eurRrIDVz/1QYb4pcSOXmmnp01SQ0oXMb0FJKKE 3nPaQr6h7CgHo6Vtj/LAId61u/15kltsAsYQxtYIk1ZDra3Nt8YbGXNniMd4ksxxRI41C4 d/O+kx0KvawJoM4PJU9ZISCuSscrgsL2QkEwf60pRxTcjXaD9sVzWARBehMbrg== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1644337496; a=rsa-sha256; cv=none; b=jVxSbWMactiVtCKqhjhXRw6JYDflp1FmtQP34bgXeZqjGZ3aOURqBYTLUJTFOu87x8z5SZ lcMZPH9dBySTpJOKbsCBdh4rE6bMR02+1LUP0zMubGfsrGRIw0r2Lw8vCoRh2H5jXyxOAq 2e2NkkBywYUdinRDZKaCYL2/I2Sp81HHgBrxi9Oh2vIt6YH65oIOIBs4ymsMirKs1v5b8c KfOq7EcBBD0Mc/EvOy1dPxfDc6YGneU8FJ1YJ5wrGs1NgjAazY3bAOY6yAmFOHIhOiPBow vBO2j87asVtW5xn6tl1Dem991vq0z0PHUIhSXKfnzmAd7cf8Ek+oJrWk+bNu3Q== ARC-Authentication-Results: i=1; mx1.freebsd.org; none X-ThisMailContainsUnwantedMimeParts: N https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D261291 --- Comment #12 from Alan Somers --- I reproduced the issue again. Deliberately this time, by downing one leg of the LAGG during high traffic. This time I tailored the packet capture more narrowly, so it didn't drop any packets (the original pcap file contained omissions just because tcpdump couldn't write to disk fast enough). Crucia= lly, it shows that the client sent a DESTROY_SESSION rpc which didn't show up in= the original pcap file. The sequence looks like this: 1) The client's (172.30.156.243) last regular NFS call is packet 84 2) After that are a ton of TCP segment reassemblies. Probably related to t= he lagg interruption 3) In packet 472, the client sends DESTROY_SESSION 4) The server (172.30.99.32) replies NFS4_OK in packet 474 5) The client sends EXCHANGE_ID in packet 475 6) The server responds with NFS4_OK and clientid 0xd9e0ee6135000000 in pack= et 477 7) The client sends CREATE_SESSION in packet 478 with clientid 0xd9e0ee6135000000 8) The server replies NFS4ERR_STALE_CLIENTID in packet 480 9) Go back to step 5 and loop Could there be a problem in how we handle the DESTROY_SESSION rpc? If you = want to look, I uploaded the new packet trace to my home directory on freefall, named "slc-rb-nesx4-7-feb.create-session.pcap". --=20 You are receiving this mail because: You are the assignee for the bug.=