From nobody Sun Jul 03 21:45:06 2022 X-Original-To: bugs@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id A1AE88A4C90 for ; Sun, 3 Jul 2022 21:45:06 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4LbjCQ3KPsz3vxH for ; Sun, 3 Jul 2022 21:45:06 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2610:1c1:1:606c::50:1d]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4D6472C70E for ; Sun, 3 Jul 2022 21:45:06 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.5]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id 263Lj6T1035045 for ; Sun, 3 Jul 2022 21:45:06 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id 263Lj6xK035044 for bugs@FreeBSD.org; Sun, 3 Jul 2022 21:45:06 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: bugs@FreeBSD.org Subject: [Bug 265019] pmap_growkernel wiring excess memory on module load Date: Sun, 03 Jul 2022 21:45:06 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: new X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: CURRENT X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: ashafer@badland.io X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: bugs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: bug_id short_desc product version rep_platform op_sys bug_status bug_severity priority component assigned_to reporter Message-ID: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated List-Id: Bug reports List-Archive: https://lists.freebsd.org/archives/freebsd-bugs List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-bugs@freebsd.org MIME-Version: 1.0 ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1656884706; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=2+dkz7sVaU1su5la3S/pnqIeb+acuQwiYxzI2X+N3Uw=; b=hx0rH1fXynsjG8khwdj9GjtBjtvh70FEvT4VFPUyrRyRowY4e+GDMIxwUM+7P6L/qWYUCC TUGubjM9LD7wh02gCSx5bHK874Gl4/QYCe7LoovPmd+EPj9DOSxEZwVW9kl14siW/9XdLb V4zFRHFFakA4vs4cQ1DLJDKPFEAV0VPUlrrXGrD8dqJoJ5LpaZsgUv+FrDNdyCiHVOPC15 3UpnY/bQ7zagOOuRcBmLoqmX90DphBUsZG2q40INd8CeOPwZ/khHh7VlgXnLa11UswMJIi rxh0T3rz2NXh5C9KaPqduiIpaaOtDU0yUdgNkButZWo4KaX/tcYjv9wDmlmZFw== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1656884706; a=rsa-sha256; cv=none; b=R1ucf1rSIKW13/z84gO5TBjPgn1BcwN9S8cfvKshFjdxG3TWywMM2Efz5qRM4BAJLk1Ek+ lcRYP9N/NoCyQ1kf8gAniSIrqOq+4mRKEY9edqwcD8eNhaHi8Qdel8wkcA9Qw8ex2a9EYF 4ybA69YrdKSzw8KYBHnQw4UPS3eESCmKSC5XcFQ3WdULfEFqLk06Ul75nohSXpsFn/PlDe oWaMktj1aH1m/ktwshTZyDbn9TE+uGSjIU03QPL4vpLc6YO48J2RS/FziWqWxQmWLeeKgM hPCg1yradmGlHJICO6TpfhLB5kmStRbe66y86xnftymgJe8lkISyFL/QmMUZ8A== ARC-Authentication-Results: i=1; mx1.freebsd.org; none X-ThisMailContainsUnwantedMimeParts: N https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D265019 Bug ID: 265019 Summary: pmap_growkernel wiring excess memory on module load Product: Base System Version: CURRENT Hardware: amd64 OS: Any Status: New Severity: Affects Some People Priority: --- Component: kern Assignee: bugs@FreeBSD.org Reporter: ashafer@badland.io When loading a large kernel module (in this case nvidia.ko) a massive amoun= t of memory will be fired. This ends up wiring 4.2Gb out of 8Gb on the system, w= hich significantly hurts usability. There was a lot of discussion on discord about this, I'll try to include it here: https://discord.com/channels/727023752348434432/757305573866733680/99294022= 4063746058 "Maybe it is the addr param itself after all. I'm seeing some weirdness in = the min address used for kernel_vm_end that I haven't explained yet. KERNBASE is 0xffffffff80000000, but I see vm_map_find using 0xfffffe0000000000 as the m= in address to start searching for open space at. So kernel_vm_end appears to s= tart at fffffe00b1600000 when loading the kernel (the kernel log drops some lines there so this is the first value I see for it), and it's raised to find more space. That seems to be the general "neighborhood" of addresses passed to pmap_growkernel . Then at some point when I call kldload it passes some off= set from KERNBASE instead of from that "neighborhood", and since KERNBASE and 0xfffffe0000000000 are so far away it allocates a million pages. Sorry the logic there is a little fuzzy, but tldr is it feels like there are two different ideas about where the kernel VM address starts and kldload triggers us using the wrong one." What we think is happening is that normally vm_map_find is called starting = from VM_MIN_KERNEL_ADDRESS, but link_elf_load_file starts searching at KERNBASE. This ends up wiring massive amount of memory to split the difference. Norma= lly this would be fine since we reserve a bunch of pages after KERNBASE for ker= nel modules, but if the kernel module is too large and overflows this region, t= his error might occur. This PR tracks improving the logic of pmap_growkernel to properly track gro= wing in multiple ways. We might want to have two kernel_vm_ends, and decide the starting/ending points of growing the kernel based on the address given to pmap_growkernel and if it's above KERNBASE --=20 You are receiving this mail because: You are the assignee for the bug.=