sys/dev/em/if_em.c

[Julian Elischer]

Murat Balaban wrote:

>Hello hackers,
>
>I have a special-purpose setting where I have a ng_hub like kernel module (ng_lb)
>which I've been coding. The box I'm using has two em(4) adapters, and I've
>hooked em0's lower with my ng_lb's link0, and em1's lower with ng_lb's link1.
>
>Situation looks like this:
>
>        lower            link0                 link1              lower
>em0 ---------------> -------------> ng_lb --------------> ---------------> em1
>
>Every packet that is received by em0 is handed over to my netgraph module and
>after very little modification in the packet ethernet header (changing destination
>mac addresss) I NG_FWD_ITEM() the packet to em1.
>  
>

somewhat similar to what ng_bridge does.

>I'm generating traffic with a packet generator, and em0 seems to be ok with around
>910 Mbit/s traffic.
>
>However if I write the packets into em1, em1 seems to drop 40-60 Mbit/s (of 910 Mbit/s)
>data. I digged the problem a bit, and found out that, IFQ_HANDOFF, called deep inside
>from NG_FWD_ITEM was returning ENOBUFS.
>
>A little more investigation proved me that the source of ENOBUFS error was that
>the em1 was running out of Tx descriptors. The relative logic in dev/em/if_em.c
>(em_encap) was that if # of Tx descriptors falls below a threshold, the driver
>tries to clean transmit interrupts once. # of available Tx desc. is again checked
>and if the number is still not incresed ENOBUFS error is returned.
>
>What I'd like to ask is, instead of cleaning the transmit interrupts only once,
>why not do it many times till the number of available tx descriptors increases 
>to a moderate level?
>
>The following patch solved my problem, though I wanted to get your opinions about
>it.
>
>Cheers, 
>Murat
>http://www.enderunix.org/murat/
>
>PS: Both cards are plugged into a 64-bit 66 Mhz PCI-X bus. I've polling enabled
>in both interfaces, and HZ set to 10000.
>
>
>--- if_em_murat.c       Wed Jul 26 13:59:22 2006
>+++ if_em.c     Wed Jul 26 14:01:11 2006
>@@ -1177,11 +1177,9 @@
>          * available hits the threshold
>          */
>         if (adapter->num_tx_desc_avail <= EM_TX_CLEANUP_THRESHOLD) {
>-                em_clean_transmit_interrupts(adapter);
>-                if (adapter->num_tx_desc_avail <= EM_TX_CLEANUP_THRESHOLD) {
>-                        adapter->no_tx_desc_avail1++;
>-                        return(ENOBUFS);
>-                }
>+               do {
>+                       em_clean_transmit_interrupts(adapter);
>+               while (adapter->num_tx_desc_avail <= EM_TX_CLEANUP_THRESHOLD);
>         }
>
>  
>

please have some limit to the number of times that the loop can be run.. 
maybe 20 or something.
(and maybe something  to note that it has hit that limit).

>         /*
>_______________________________________________
>freebsd-hackers at freebsd.org mailing list
>http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
>To unsubscribe, send any mail to "freebsd-hackers-unsubscribe at freebsd.org"
>  
>