From nobody Tue Nov 26 23:32:27 2024 X-Original-To: bugs@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Xyf4X2SgCz5fBBn for ; Tue, 26 Nov 2024 23:32:28 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R10" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Xyf4W69zxz4Q0N for ; Tue, 26 Nov 2024 23:32:27 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1732663947; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=XGk4TrrsxqhjjyTp+w57BbUv/1rSTWhtk4kOgWJlbSo=; b=DDTUYtLmt6ml2oKSYxCfVXvZOK8gKAtB5wyKXl8+5l5lST7Nu7DxcC7SbRbHF7NgUSpUhR 5JduNyxfmsaVKBn+cQgGUzjwL85D+DqqL6sOyLT2cUAXriNShXbpdUJwlULKNKSfID60Xa rpe5e4mETcWEUyBAqcEblKXUCA+BuSn3H3FIywy1DeG224O+eOY+a+oG2KOPNqvexPrK29 iyd7Yy0+eec7/moAJhQFZfHzzbk3hhLtuoBHXKKvGUOsaviV3J1deBcvvNuv1QD41PTv+p qixrZf/XwC5lDXXJgCLla0K47iPxLfbNYgf4KMF9SvzT/QOr7e6bbLr1uH00aA== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1732663947; a=rsa-sha256; cv=none; b=wBg7qmJqIzz0V14FiRtKIpT4exBysLRAs0q0jgaTybZXRuiRQdfMvNcKkAKa5RiVT3xgaD RT6J2pwiZHMPb+OS0dtOF0zqrwFcOGCYqX389yxH+OkPDUQH6y5ZWdXP0HcJNsJesWno8T 5xu7qWUaUCeqDKV4pP4FkuBFhSf9QgXhUfcwVSenVrSJOKeoOsVlCWY0M8gVuKqfmhmc6x e0X2EqrCagu2vYhNrEAGmbwSXew7OVYt3JMAplIQFXLTTRYVjCn8C2Re6Qox32lQgiwgqS mUimU5IGvWoaC0S3mbVJP7FJrSBtLuhQghUedv2LCtOBWojYzF999uHY56et1Q== Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2610:1c1:1:606c::50:1d]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4Xyf4W5nbrzljG for ; Tue, 26 Nov 2024 23:32:27 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.5]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id 4AQNWRsj029369 for ; Tue, 26 Nov 2024 23:32:27 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id 4AQNWRrE029367 for bugs@FreeBSD.org; Tue, 26 Nov 2024 23:32:27 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: bugs@FreeBSD.org Subject: [Bug 282993] awk eats all ram Date: Tue, 26 Nov 2024 23:32:27 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: new X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: bin X-Bugzilla-Version: 14.1-RELEASE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: riccardo@torrini.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: bugs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: bug_id short_desc product version rep_platform op_sys bug_status bug_severity priority component assigned_to reporter Message-ID: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated List-Id: Bug reports List-Archive: https://lists.freebsd.org/archives/freebsd-bugs List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-bugs@FreeBSD.org MIME-Version: 1.0 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D282993 Bug ID: 282993 Summary: awk eats all ram Product: Base System Version: 14.1-RELEASE Hardware: amd64 OS: Any Status: New Severity: Affects Only Me Priority: --- Component: bin Assignee: bugs@FreeBSD.org Reporter: riccardo@torrini.org I have a very simple parser for windows logs (forwarded as json using nxlog= ), created on 13.3 or even before. After update to 14.1 it eats all my memory = (ram and swap). Here the script: # tail -qF {a lot of windows log, rotated every hour} | \ awk -F" {" '{printf( "{%s\n", $2 )}' | \ grep --line-buffered 'AUDIT_FAILURE' | jq '{some parsing}' -j -c -M --unbuffered I can confirm that this script runned for at least 130 days on 13.3 but aft= er 10 hours on 14.1 top showed 17GB used for awk process (confirmed by ps) Actually I changed awk filter to perl (I'm sure it is not the best conversi= on but works) perl -F"[[:space:]]{" -e 'print "{@F[1]" ; ' | \ (please not that ram also grow up with perl filter, but much slower) I have tried to reproduce with [1] [2] and [3] but happens only on [3] :-( [1] ( while true; do cat /etc/motd; done ) | awk -F'-' '{printf( "%s\n", $2= )}' > /dev/null [2] cat /dev/random | hd | awk -F' ' '{printf( "%s\n", $0 )}' > /dev/null [3] tail -qF /var/log/remote/windows/*log | awk -F' {' '{printf( "{%s\n", $2 )}' > /dev/null (while running 1, 2 or 3 use top or ps -o rss to check used ram) VM used to have 8GB of ram and 17GB of swap when 13.3, now has 32GB of ram (but =E2=80=8B=E2=80=8Bthat's not the point, awk behavior is changed someho= w or I'm using it a lot bad) -- Riccardo. --=20 You are receiving this mail because: You are the assignee for the bug.=