Advice on a multithreaded netisr patch?

[Julian Elischer]

Sepherosa Ziehau wrote:
> On Mon, Apr 6, 2009 at 7:59 PM, Robert Watson <rwatson at freebsd.org> wrote:
>> m_pullup() has to do with mbuf chain memory contiguity during packet
>> processing.  The usual usage is something along the following lines:
>>
>>        struct whatever *w;
>>
>>        m = m_pullup(m, sizeof(*w));
>>        if (m == NULL)
>>                return;
>>        w = mtod(m, struct whatever *);

while this is true, m_pullup ALWAYS does things so in fact you
want to always put it in a test to see if it is really needed..

from memory it is something like:

  if (m->m_len < headerlen && (m = m_pullup(m, headerlen)) == NULL) {
        log(LOG_WARNING,
           "nglmi: m_pullup failed for %d bytes\n", headerlen);
              return (0);
  }
  header = mtod(m, struct header *);


>>
>> m_pullup() here ensures that the first sizeof(*w) bytes of mbuf data are
>> contiguously stored so that the cast of w to m's data will point at a
>> complete structure we can use to interpret packet data.  In the common case
>> in the receipt path, m_pullup() should be a no-op, since almost all drivers
>> receive data in a single cluster.
>>
>> However, there are cases where it might not happen, such as loopback traffic
>> where unusual encapsulation is used, leading to a call to M_PREPEND() that
>> inserts a new mbuf on the front of the chain, which is later m_defrag()'d
>> leading to a higher level header crossing a boundary or the like.
>>
>> This issue is almost entirely independent from things like the cache line
>> miss issue, unless you hit the uncommon case of having to do work in
>> m_pullup(), in which case life sucks.
>>
>> It would be useful to use DTrace to profile a number of the workfull m_foo()
>> functions to make sure we're not hitting them in normal workloads, btw.
> 
> I highly suspect m_pullup will take any real effect on RX path, given
> how most of drivers allocate the mbuf for RX ring (all RX mbufs should
> be mclusters).
> 
> Best Regards,
> sephe
>