[RFC] Event timers on sparc64/sun4v

[Marius Strobl]

On Tue, Jul 20, 2010 at 01:54:02PM -0400, John Baldwin wrote:
> On Sunday, July 18, 2010 10:05:08 am Marius Strobl wrote:
> > On Sat, Jul 17, 2010 at 07:26:03PM +0300, Alexander Motin wrote:
> > > By the way I have some doubts about tick_get_timecount_mp() correctness.
> > > It tries to bind thread to BSP, but what if it is called inside
> > > interrupt handler, or under lock, or some else. I have doubt binding
> > > will work in that case.
> > 
> > I've no idea whether sched_bind() works under locks etc as
> > there's no man page describing it, however as it requires
> > curthread to be locked and thread_lock() itself uses a
> > spin lock and locking(9) basically says that acquiring a
> > spinlock with any other lock held is okay I assume that
> > the whole thing is fine with any lock held. Also if there
> > were such restrictions I'd expect there some KASSERTs etc
> > to be in place in the functions invovled preventing
> > incorrect use but tick_get_timecount_mp() doesn't trigger
> > such.
> 
> It isn't safe mostly because sched_bind() might need to context switch if you 
> aren't currently running on the desired CPU.  If you do that with another spin 
> lock held you should hit this KASSERT():
> 
> 	KASSERT(td->td_critnest == 1 || (td->td_critnest == 2 &&
> 	    (td->td_owepreempt) && (flags & SW_INVOL) != 0 &&
> 	    newtd == NULL) || panicstr,
> 	    ("mi_switch: switch in a critical section"));
> 
> Since td_critnest will be > 1 and flags will have SW_VOL set instead of 
> SW_INVOL.  Really critical_exit() is the only place that is allowed to call 
> mi_switch() with td_critnest != 1, but ULE doesn't even do that now.

Thanks for clearing that up.

> 
> > Apart from that I'm not really happy about that construct
> > myself but I don't see an alternative to always bind to
> > the same CPU when reading the tick counter in order to
> > get reliable results and in US-IIIi-based machines there
> > just isn't another piece of hardware besides the per-CPU
> > stick and tick counters that could be used as a timecounter
> > available.
> 
> You could check td_critnest perhaps in your routine and if it is non-zero just 
> return the cached value of the timecounter from the last time it was polled 
> from tc_ticktock() (effectively turning those instances of getfootime() into 
> just footime()).  Things like gettimeofday() would still be correct as they 
> can safely bind to the BSP, and I doubt many interrupt handlers are actually
> using getfootime().
> 

I think instead of using sched_bind() I could just IPI the BSP
in case the it's called on an AP. When implementing the tick
reader as direct cross trap that should have minimal overhead
and avoids making any assumptions about the caller of the
tc_get_timecount function.

Marius