can the scheduler decide to schedule an interrupted but runnable thread on another CPU core? What are the implications for code?

[John Baldwin]

On Friday, February 14, 2014 12:55:03 pm Adrian Chadd wrote:
> On 14 February 2014 08:39, John Baldwin <jhb at freebsd.org> wrote:
> > On Friday, February 14, 2014 4:22:34 am Adrian Chadd wrote:
> >> Ok, so now I remember the other odd thing.
> >>
> >> I was seeing the sending context(s) jumping from one CPU to another
> >> during flowtable_insert_common(), around the locking bits.
> >>
> >> But I thread pinned all the sender user threads!
> >>
> >> So, why would the senders still be scheduled on other CPUs if I've
> >> pinned the userland threads?
> >>
> >> (and yes, I verified that the userland threads weren't moving around.)
> >
> > Can you clarify a bit?  It's not clear how sender thraeds differ from
> > userland threads differ from sender user threads.  (I.e. one reading
> > is that these are all the same thing and should thus all be pinned
> > (I assume you mean using cpuset to bind them to specific cores rather
> > than sched_pin))
> 
> Yup, I'm doing a manual, poor-mans RSS in lieu of merging in roberts stuff:
> 
> * the userland threads are using the cpuset call to map a thread into
> a cpuset, yes
> * the NIC TX/RX ring routines in cxgbe are pinned to the same CPU as
> the userland threads

If they are all cpuset to a single CPU, they should not migrate, though
I think sched_bind() can override that.  However, that requires code to
explicitly call sched_bind() which should be rare.

-- 
John Baldwin