Logstash failing to process messages

[Kernel Panic]

Thanks for getting back to me, yes I suspect it has something to do with my
filters though I've no idea which one it could be as I'm filtering on beats
and syslog inputs. As a work around I've just added a cron command to
restart Logstash every morning at 01:00, though obviously that means I'm
losing non-beat events whilst it restarts. Please let me know if upgrading
to the latest versions helps you, if it doesn't then perhaps a PR needs to
be filed.

On 24 May 2018 at 11:25, Benny Goemans <benny.goemans at belgacom.net> wrote:

> I have seen the same issue. In my case however, I had about OOM caused by
> parsing long grok patterns. I didn't have these in 5.3 either so I suspect
> it's a memory leak somewhere.
> I have since upgraded everything to 6.x and am waiting to see if the same
> issue persists.
>
> Regards,
> Benny Goemans
>
> On 23-05-2018 17:23, Kernel Panic wrote:
>
>> Hello, I'll just list the versions before I start:
>>
>> FreeBSD 11.1
>>
>> Logstash 6.23
>> Elasticsearch 5.6.8
>> Kibana 5.6.8
>>
>> The issue I'm having is that after a few days Logstash will stop
>> processing
>> any messages; I'm using the same config file that I used with Logstash
>> 5.3.0 which worked without issue and was rock-solid. There's nothing in
>> the
>> Logstash log file apart from messages about a field in my Cisco logs being
>> the wrong type and therefore failing to index, however this has always
>> been
>> the case. I have tried enabling the 'dead letter' feature in Logstash to
>> process these Cisco logs but that just makes Logstash even more unstable.
>>
>> The Logstash service doesn't actually crash, it just stops processing
>> messages and fails to respond to the restart command so I end up having to
>> reboot the server. I should say though that Logstash continues to respond
>> the the monitor API commands.
>>
>> I have tried updating all Logstash plugins however that has not fixed the
>> issue.
>>
>> As I said, I never had any problems with Logstash 5.3.0 but the latest
>> version (and version 5.6.8) just seem to become unstable after a few days.
>>
>> Any help is greatly appreciated.
>> _______________________________________________
>> freebsd-ports at freebsd.org mailing list
>> https://lists.freebsd.org/mailman/listinfo/freebsd-ports
>> To unsubscribe, send any mail to "freebsd-ports-unsubscribe at freebsd.org"
>>
>
>
>