RFC: bus_get_cpus(9)

[John Baldwin]

On Thursday, February 19, 2015 10:15:56 AM Warner Losh wrote:
> > On Feb 19, 2015, at 10:03 AM, John Baldwin <jhb at freebsd.org> wrote:
> > 
> > On Thursday, February 19, 2015 08:28:34 AM Adrian Chadd wrote:
> >> On 19 February 2015 at 07:37, John Baldwin <jhb at freebsd.org> wrote:
> >>> There's nothing preventing the RSS code from calling bus_get_cpus()
> >>> internally to populate the info it returns in its APIs.
> >>> 
> >>> That is, I imagine something like:
> >>> 
> >>> #ifdef RSS
> >>> 
> >>>        queue_info = fetch_rss_info(dev);
> >>>        for (queue in queue_info) {
> >>>        
> >>>                create queue for CPU queue->cpu
> >>>        
> >>>        }
> >>> 
> >>> #else
> >>> 
> >>>        /* Use bus_get_cpus directly and do 1:1 */
> >>> 
> >>> #endif
> >>> 
> >>> That is, I think RSS should provide a layer on top of new-bus, not be a
> >>> bus_foo API.  At some point all drivers might only have the #ifdef RSS
> >>> case
> >>> and not use bus_get_cpus() directly at all, but it doesn't seem like the
> >>> RSS API is quite there yet.
> >> 
> >> I wasn't suggesting we have RSS as a newbus method, just that drivers
> >> don't necessarily need to call the bus method and iterate themselves.
> >> 
> >> I was suggesting that we do what i've done for rss, but as a generic
> >> "how should devices create queues and map them to cpusets / interrupt
> >> locality" and that calls the bus method(s) to discover topology and
> >> query local-interrupt and local-memory sets to do things
> >> appropriately.
> >> 
> >> Then RSS is just a flavour of that API call - network drivers could
> >> either be RSS aware and call it to get the mapping, or call some
> >> higher level bus API call to do the "generic" hints or whatever based
> >> mapping.
> > 
> > Can you provide a sample API (function prototype, etc.)?  Aside from RSS
> > (which will have its own API for other reasons), I don't know of another
> > use case that is well understood enough to let us build an abstraction on
> > yet (we all know the perils of abstracting from one use case), so I'm
> > hesitant to go much further than "these are the best place to do
> > interrupts”.
> 
> Newer LSI cards could benefit from that, but the rest of the storage stack
> may need tweaks to allow for true multi-queue implementations. Interrupts
> would be part of that.

Right, storage in particular I think we don't really know what model we want 
yet, so aren't really ready for an API that tries to be abstract across both 
network and storage.

-- 
John Baldwin