Re: FreeBSD on KVM, Q35 and kernel structure leak

From: Eugene M. Zheganin <eugene_at_zhegan.in>
Date: Tue, 05 Apr 2022 08:25:33 UTC 
Follow-up:

13.0-RELEASE also has this problem (I upgraded to a recent version 
trying to see if it solves the issue), and of course when running 
similar setup (heavily loaded nginx + grpc application) on the baremetal 
server everything is just fine (regardless of the version - I have 
several 12.x and 13.x). This also may be related to the fact that this 
particular VM is using vtnet(4) driver; but I'm using the same cloud VM 
type in Hetzner to run nginx + prometheus, it has the uptime of 19 days 
and shows no signs of the leakage.

05.04.2022 3:19, Eugene M. Zheganin пишет:
> Hello,
>
> I'm trying to run FreeBSD 13.1-RC1 (releng/13.1-n250066-aef190f298a) 
> on KVM/Qemu, presumably Q35 type VM in Hetzner.
>
> After some uptime and considerable amount of connections, VM leaks 
> tcp_inpcb to the limit:
>
>
> ITEM                   SIZE  LIMIT     USED     FREE      REQ 
> FAILSLEEP XDOMAIN
> tcp_inpcb:              496, 510671,  510281,     391, 617269,11530,   
> 0,   0
>
>
> Causing multiple dmesg errors
>
> sonewconn: pcb 0xfffff8001ac8bd90: pru_attach() failed
> sonewconn: pcb 0xfffff8000ab625d0: pru_attach() failed
> sonewconn: pcb 0xfffff8000af999b0: pru_attach() failed
> sonewconn: pcb 0xfffff8000ab621f0: pru_attach() failed
> sonewconn: pcb 0xfffff8000ab62000: pru_attach() failed
> sonewconn: pcb 0xfffff8000ab625d0: pru_attach() failed
> sonewconn: pcb 0xfffff8000af999b0: pru_attach() failed
> sonewconn: pcb 0xfffff8000af999b0: pru_attach() failed
> sonewconn: pcb 0xfffff8000af993e0: pru_attach() failed
> sonewconn: pcb 0xfffff8000af999b0: pru_attach() failed
> sonewconn: pcb 0xfffff8000ab627c0: pru_attach() failed
>
>
> console errors (even on low number of kern.ipc.numopensockets: 1447)
>
>
> [zone: tcp_inpcb] kern.ipc.maxsockets limit reached
> [zone: tcp_inpcb] kern.ipc.maxsockets limit reached
> [zone: tcp_inpcb] kern.ipc.maxsockets limit reached
> [zone: tcp_inpcb] kern.ipc.maxsockets limit reached
> [zone: tcp_inpcb] kern.ipc.maxsockets limit reached
> [zone: tcp_inpcb] kern.ipc.maxsockets limit reached
> [zone: tcp_inpcb] kern.ipc.maxsockets limit reached
> [zone: tcp_inpcb] kern.ipc.maxsockets limit reached
> [zone: tcp_inpcb] kern.ipc.maxsockets limit reached
> [zone: tcp_inpcb] kern.ipc.maxsockets limit reached
>
>
> and inability to connect to just anything:
>
>
> # telnet 127.0.0.1 4080
> Trying 127.0.0.1...
> telnet: socket: No buffer space available
>
>
> Even when the number of connections id dropped (for instance I remove 
> this VM from balancing) it cannot recover (not freeing tcp_inpcb).
>
>
> is there any hope considering this ?
>
>
> Thanks.
>
> Eugene.
>
>