problem with 9k jumbo clusters

[Yuriy Tabolin]

I would be grateful if you could make patch.

05.12.2014 00:49, Jack Vogel пишет:
> I had wanted to remove the larger cluster sizes from the driver a 
> while back, but for reasons
> I don't remember that code change didn't happen. The new 40G ixl 
> driver does this, it only
> uses standard clusters for anything under 2K, and above that 
> everything uses 4K.
>
> I would be curious to see if this change would resolve your problem, 
> would you like a patch,
> or are you able to hack the code yourself to do this?
>
> Jack
>
>
> On Thu, Dec 4, 2014 at 11:00 AM, Alexander V. Chernikov 
> <melifaro at freebsd.org <mailto:melifaro at freebsd.org>> wrote:
>
>     On 04.12.2014 13:50, Yuriy Tabolin wrote:
>
>         Hi All.
>         I have a server with two Intel 10G NIC. OS FreeBSD
>         10.1-Release amd64. Server works like NFS, samba-server and
>         iSCSI target. Both NICs aggregated into lagg device and set
>         MTU 9014 to them. There are some tuning sysctl.conf:
>         kern.maxfiles=6289601
>         kern.maxfilesperproc=5660640
>         kern.maxvnodes=3339565
>         kern.ipc.nmbclusters=12255588
>         kern.ipc.nmbjumbop=6127794
>         kern.ipc.nmbufs=78435780
>         kern.ipc.maxsockbuf=16777216
>         kern.ipc.maxsockets=6289600
>         net.inet.tcp.sendbuf_max=16777216
>         net.inet.tcp.sendspace=65536
>         net.inet.tcp.recvbuf_max=16777216
>         net.inet.tcp.recvspace=65536
>
>         After some days of working, the errors are appearing:
>         ix1: Interface stopped DISTRIBUTING, possible flapping
>         ix0: Interface stopped DISTRIBUTING, possible flapping
>         ix0: Could not setup receive structures
>         ix1: Could not setup receive structures
>
>     Hello. It looks like
>     https://lists.freebsd.org/pipermail/freebsd-net/2014-May/038630.html
>     is relevant here.
>
>
>         After that errors the NICs stoped working. netstat -m shows:
>         32881/33854/66735 mbufs in use (current/cache/total)
>         16370/8198/24568/12255588 mbuf clusters in use
>         (current/cache/total/max)
>         16370/4807 mbuf+clusters out of packet secondary zone in use
>         (current/cache)
>         0/873/873/6127794 4k (page size) jumbo clusters in use
>         (current/cache/total/max)
>         16383/21517/37900/1815641 9k jumbo clusters in use
>         (current/cache/total/max)
>         0/0/0/1021298 16k jumbo clusters in use (current/cache/total/max)
>         188407K/222004K/410411K bytes allocated to network
>         (current/cache/total)
>         0/0/0 requests for mbufs denied (mbufs/clusters/mbuf+clusters)
>         0/0/0 requests for mbufs delayed (mbufs/clusters/mbuf+clusters)
>         0/0/0 requests for jumbo clusters delayed (4k/9k/16k)
>         0/101414306/0 requests for jumbo clusters denied (4k/9k/16k)
>         0 requests for sfbufs denied
>         0 requests for sfbufs delayed
>         0 requests for I/O initiated by sendfile
>
>         9k jumbo clusters max is too big, but looks like system cannot
>         allocate them. There are huge number of "9k requests for jumbo
>         clusters denied". ifconfig ix down/up don't helped, reboot is
>         needed. Thanks for any help!
>
>
>         -- 
>         Best regards,
>         Tabolin Yuriy
>         System administrator
>         Speech Technology Center
>         _______________________________________________
>         freebsd-net at freebsd.org <mailto:freebsd-net at freebsd.org>
>         mailing list
>         http://lists.freebsd.org/mailman/listinfo/freebsd-net
>         To unsubscribe, send any mail to
>         "freebsd-net-unsubscribe at freebsd.org
>         <mailto:freebsd-net-unsubscribe at freebsd.org>"
>
>
>     _______________________________________________
>     freebsd-net at freebsd.org <mailto:freebsd-net at freebsd.org> mailing list
>     http://lists.freebsd.org/mailman/listinfo/freebsd-net
>     To unsubscribe, send any mail to
>     "freebsd-net-unsubscribe at freebsd.org
>     <mailto:freebsd-net-unsubscribe at freebsd.org>"
>
>
-- 
Best regards,
Tabolin Yuriy
System administrator
Speech Technology Center