sys/dev/em/if_em.c

[Eric Anderson]

On 07/26/06 06:16, Murat Balaban wrote:
> Hello hackers,
> 
> I have a special-purpose setting where I have a ng_hub like kernel module (ng_lb)
> which I've been coding. The box I'm using has two em(4) adapters, and I've
> hooked em0's lower with my ng_lb's link0, and em1's lower with ng_lb's link1.
> 
> Situation looks like this:
> 
>         lower            link0                 link1              lower
> em0 ---------------> -------------> ng_lb --------------> ---------------> em1
> 
> Every packet that is received by em0 is handed over to my netgraph module and
> after very little modification in the packet ethernet header (changing destination
> mac addresss) I NG_FWD_ITEM() the packet to em1.
> 
> I'm generating traffic with a packet generator, and em0 seems to be ok with around
> 910 Mbit/s traffic.
> 
> However if I write the packets into em1, em1 seems to drop 40-60 Mbit/s (of 910 Mbit/s)
> data. I digged the problem a bit, and found out that, IFQ_HANDOFF, called deep inside
> from NG_FWD_ITEM was returning ENOBUFS.
> 
> A little more investigation proved me that the source of ENOBUFS error was that
> the em1 was running out of Tx descriptors. The relative logic in dev/em/if_em.c
> (em_encap) was that if # of Tx descriptors falls below a threshold, the driver
> tries to clean transmit interrupts once. # of available Tx desc. is again checked
> and if the number is still not incresed ENOBUFS error is returned.
> 
> What I'd like to ask is, instead of cleaning the transmit interrupts only once,
> why not do it many times till the number of available tx descriptors increases 
> to a moderate level?
> 
> The following patch solved my problem, though I wanted to get your opinions about
> it.


What if adapter->num_tx_desc_avail never gets above 
EM_TX_CLEANUP_THRESHOLD ?  Maybe adding another check (max loop counter) 
or something?


> --- if_em_murat.c       Wed Jul 26 13:59:22 2006
> +++ if_em.c     Wed Jul 26 14:01:11 2006
> @@ -1177,11 +1177,9 @@
>           * available hits the threshold
>           */
>          if (adapter->num_tx_desc_avail <= EM_TX_CLEANUP_THRESHOLD) {
> -                em_clean_transmit_interrupts(adapter);
> -                if (adapter->num_tx_desc_avail <= EM_TX_CLEANUP_THRESHOLD) {
> -                        adapter->no_tx_desc_avail1++;
> -                        return(ENOBUFS);
> -                }
> +               do {
> +                       em_clean_transmit_interrupts(adapter);
> +               while (adapter->num_tx_desc_avail <= EM_TX_CLEANUP_THRESHOLD);
>          }
> 
>          /*



Eric



-- 
------------------------------------------------------------------------
Eric Anderson        Sr. Systems Administrator        Centaur Technology
Anything that works is better than anything that doesn't.
------------------------------------------------------------------------