[Bug 282993] awk eats all ram

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=282993

            Bug ID: 282993
           Summary: awk eats all ram
           Product: Base System
           Version: 14.1-RELEASE
          Hardware: amd64
                OS: Any
            Status: New
          Severity: Affects Only Me
          Priority: ---
         Component: bin
          Assignee: bugs@FreeBSD.org
          Reporter: riccardo@torrini.org

I have a very simple parser for windows logs (forwarded as json using nxlog),
created on 13.3 or even before. After update to 14.1 it eats all my memory (ram
and swap).

Here the script:

# tail -qF {a lot of windows log, rotated every hour} | \
    awk -F" {" '{printf( "{%s\n", $2 )}' | \
    grep --line-buffered 'AUDIT_FAILURE' | jq '{some parsing}' -j -c -M
--unbuffered

I can confirm that this script runned for at least 130 days on 13.3 but after
10 hours on 14.1 top showed 17GB used for awk process (confirmed by ps)

Actually I changed awk filter to perl (I'm sure it is not the best conversion
but works)
    perl -F"[[:space:]]{" -e 'print "{@F[1]" ; ' | \
(please not that ram also grow up with perl filter, but much slower)

I have tried to reproduce with [1] [2] and [3] but happens only on [3] :-(

[1] ( while true; do cat /etc/motd; done ) | awk -F'-' '{printf( "%s\n", $2 )}'
> /dev/null
[2] cat /dev/random | hd | awk -F' ' '{printf( "%s\n", $0 )}' > /dev/null
[3] tail -qF /var/log/remote/windows/*log | awk -F' {' '{printf( "{%s\n", $2
)}' > /dev/null
(while running 1, 2 or 3 use top or ps -o rss to check used ram)

VM used to have 8GB of ram and 17GB of swap when 13.3, now has 32GB of ram
(but ​​that's not the point, awk behavior is changed somehow or I'm using it a
lot bad)

--
Riccardo.

-- 
You are receiving this mail because:
You are the assignee for the bug.