kern/138516: [patch] em(4): 4k (page size) mbuf clusters leak

[Kazuaki ODA]

>Number:         138516
>Category:       kern
>Synopsis:       [patch] em(4): 4k (page size) mbuf clusters leak
>Confidential:   no
>Severity:       critical
>Priority:       high
>Responsible:    freebsd-bugs
>State:          open
>Quarter:        
>Keywords:       
>Date-Required:
>Class:          sw-bug
>Submitter-Id:   current-users
>Arrival-Date:   Thu Sep 03 23:50:04 UTC 2009
>Closed-Date:
>Last-Modified:
>Originator:     Kazuaki ODA
>Release:        FreeBSD 8.0-BETA3 amd64
>Organization:
>Environment:
FreeBSD router.aliceblue.jp 8.0-BETA3 FreeBSD 8.0-BETA3 #47: Fri Sep  4 07:05:11 JST 2009     kazuaki at router.aliceblue.jp:/usr/obj/usr/src/sys/ROUTER  amd64
>Description:
I think current em(4) driver has the problem that the driver leaks 4k (page size) mbuf clusters, which are used to transmit packets, when em_xmit() fails with ENOBUFS.
em_mq_start_locked() calls em_xmit() repeatedly to transmit queued packets.  But, when em_xmit() fails, the dequeued packet is left alone and not requeued, so the mbuf cluster which is used for the packet leaks.

>How-To-Repeat:
1) Install Apache2 and PHP5.
2) Modify httpd.conf to enable PHP5.
3) Put large size (more than 100KB) .php file under DocumentRoot.
4) Start up Apache2.
5) Make many concurrent requests to the .php file, which you put at #3, from a remote machine.  For example, run ab (Apache Bench) as the following:
   $ ab -n 100000 -c 100 -k http://host_name_of_the_server/test.php
6) Check the number of allocated 4K mbuf clusters with "netstat -m".

>Fix:
Apply the attached patch.


Patch attached with submission follows:

--- sys/dev/e1000/if_em.c.orig	2009-08-20 03:08:50.000000000 +0900
+++ sys/dev/e1000/if_em.c	2009-09-04 06:58:56.427358096 +0900
@@ -1063,8 +1063,11 @@
                 next = drbr_dequeue(ifp, adapter->br);
                 if (next == NULL)
                         break;
-                if (em_xmit(adapter, &next))
+                if (em_xmit(adapter, &next)) {
+			if ((error = drbr_enqueue(ifp, adapter->br, next)) != 0)
+				return (error);
                         break;
+		}
                 ETHER_BPF_MTAP(ifp, next);
                 /* Set the watchdog */
                 adapter->watchdog_timer = EM_TX_TIMEOUT;


>Release-Note:
>Audit-Trail:
>Unformatted: