Network Tuning - mbuf

From: Ben Hutton <ben_at_benhutton.com.au>
Date: Fri, 27 Jun 2025 02:17:18 UTC
Hi,

I'm currently having an issue with a spring-boot application (with 
nginx  in front on the same instance) running on FreeBSD 14.1 in AWS. 
Two of our instances at present have had the application go offline with 
the following appearing in the /var/log/messages:

Jun 26 07:57:47 freebsd kernel: [zone: mbuf_jumbo_page] 
kern.ipc.nmbjumbop limit reached
Jun 26 07:57:47 freebsd kernel: [zone: mbuf_cluster] 
kern.ipc.nmbclusters limit reached
Jun 26 07:59:34 freebsd kernel: sonewconn: pcb 0xfffff8021bd74000 
(0.0.0.0:443 (proto 6)): Listen queue overflow: 193 already in queue 
awaiting acceptance (104 occurrences), euid 0, rgid 0, jail 0
Jun 26 08:01:51 freebsd kernel: sonewconn: pcb 0xfffff8021bd74000 
(0.0.0.0:443 (proto 6)): Listen queue overflow: 193 already in queue 
awaiting acceptance (13 occurrences), euid 0, rgid 0, jail 0

Each time this has occurred I have increased the nmbjumbop and 
nmbclusters values. The last time by a huge amount to see if we can 
mitigate the issue. Once I adjust the values the application starts 
responding to requests again.

My question is, is just increasing this the correct course of action or 
should I be investigating something else, or adjusting other settings 
accordingly? Also if this is due to an underlying issue and not just 
network load how would I get to the root cause? Note the application 
streams allot of files in rapid succession which I'm suspecting is what 
is causing the issue.

Thanks
Ben