From nobody Mon Jul 01 08:36:17 2024 X-Original-To: pf@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4WCKBn5pc3z5PsRS for ; Mon, 01 Jul 2024 08:36:17 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R11" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4WCKBn43fbz52HH for ; Mon, 1 Jul 2024 08:36:17 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1719822977; a=rsa-sha256; cv=none; b=SdiK8tcWJFvJ8S8ecsLJy2qKHv1wbHaDW1qwcWVzC4bQEUeRNq0tBrZEKMS8vJqdwGvc32 kejInqwqtTmyhE1v5bR9cJUnSwxPnWK7LCO50rIXqyuSt4p7+2OJYsHLqz6xLzeohi/MRH VPbl9QOEk0bHtUgHqqNusdPQiqJRhR4EJDR8gVoUJ9wxb+XzURy7amei4ZXGKHHZmJpF4D iU6BE+jIGROoja1HqCUXXkhZ89fC2Qw6rIzy9VBf7FZtzg7RmcbCe4cs+seIvPFH1Lp9Xa /aCmdFBvvGXFSQxKkg4OLB7m45F3Rlc95jAZW7nhUoXaaMOgvaKL7v/ZKTjeXA== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1719822977; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=ylghnbqIG7GB79Wv/GmbWFpCbAMjwy/WiGmH3nk1o74=; b=Yw9f0iVL+Y6qe1oSxIuZ5yfYr7Qg9FgoTu9ENoEClL4u/xm3tQHDf4smG3E1XzR05JyWBG GxYhkGw2LfFMIG2NyKZsX+bQxmKFNmo/wFl0EKXeE+8CaXREGqpjrv4AayHrqMeU4m27B5 dWNV0YCLfx2vKIeDgOSnzORLql6QiN6X7sckbPkRdyfQXU9M3s7i05mvW3zTqkE4PfRjz3 ueUn54FOP044esNjunP7hJOoZEpRMizCjZzoEgcjzlm2WBBvq9HehHo+vWH9HPc5eu6uYy 2zWxkz96zq88lsjyh9wg+i1rqRjLOttLNMP0QHZh/lRzfY4abZwFDD6aFC19ZA== Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2610:1c1:1:606c::50:1d]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4WCKBn2dF8zy7D for ; Mon, 1 Jul 2024 08:36:17 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.5]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id 4618aHqE010640 for ; Mon, 1 Jul 2024 08:36:17 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id 4618aHGD010639 for pf@FreeBSD.org; Mon, 1 Jul 2024 08:36:17 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: pf@FreeBSD.org Subject: [Bug 279830] Using AMD EPYC 9374F 32-Core Processor, PF driver gives error when loading for more than 62 VFs Date: Mon, 01 Jul 2024 08:36:17 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 14.0-RELEASE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: vdubey@maxlinear.com X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: pf@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated List-Id: Technical discussion and general questions about packet filter (pf) List-Archive: https://lists.freebsd.org/archives/freebsd-pf List-Help: List-Post: List-Subscribe: List-Unsubscribe: X-BeenThere: freebsd-pf@freebsd.org Sender: owner-freebsd-pf@FreeBSD.org MIME-Version: 1.0 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D279830 --- Comment #5 from vdubey@maxlinear.com --- Anyone can help to check on this issue please. The device which we are using has an on chip memory of 32 MB. Kernel is allocating prefetchable memory for all the 64 VFs. We do not see any issue on Intel CPU to create 64 VFs as the FreeBSD kernel= is able to grow memory as seen from the log: dre_drv0: DRE_drvIovInit: Called with num_vfs 64. pcib7: allocated memory range (0xa3d00000-0xa40fffff) for rid 254 of dre_dr= v0 dre_drv0: Lazy allocation of 0x400000 bytes rid 0x254 type 3 at 0xa3d00000 pcib7: allocated memory range (0xa4100000-0xa41fffff) for rid 260 of dre_dr= v0 dre_drv0: Lazy allocation of 0x100000 bytes rid 0x260 type 3 at 0xa4100000 pcib7: attempting to grow memory window for (0-0xffffffff,0x2000000) front candidate range: 0xa1d00000-0xa3cfffff back candidate range: 0xa6280000-0xa827ffff pcib7: grew memory window to 0xa1d00000-0xa62fffff pcib7: allocated memory range (0xa1d00000-0xa3cfffff) for rid 264 of dre_dr= v0 dre_drv0: Lazy allocation of 0x2000000 bytes rid 0x264 type 3 at 0xa1d00000 But, when we try it on an AMD EPYC CPU, we see following error and it is not able to allocate memory: dre_drv0: DRE_drvIovInit: Called with num_vfs 64. pcib15: allocated memory range (0xf9000000-0xf93fffff) for rid 254 of dre_d= rv0 dre_drv0: Lazy allocation of 0x400000 bytes rid 0x254 type 3 at 0xf9000000 pcib15: allocated memory range (0xf9400000-0xf94fffff) for rid 260 of dre_d= rv0 dre_drv0: Lazy allocation of 0x100000 bytes rid 0x260 type 3 at 0xf9400000 pcib15: attempting to grow memory window for (0-0xffffffff,0x2000000) front candidate range: 0xf7000000-0xf8ffffff back candidate range: 0xfb580000-0xfd57ffff dre_drv0: 0x2000000 bytes of rid 0x264 res 3 failed (0, 0xffffffffffffffff). dre_drv0: DRE_drvIovUnInit: Called. Can anyone help to confirm if its an issue related to FreeBSD kernel. --=20 You are receiving this mail because: You are the assignee for the bug.=