rtprio_thread trouble

[John Baldwin]

On Friday, February 22, 2013 2:06:00 pm Ian Lepore wrote:
> I ran into some trouble with rtprio_thread() today.  I have a worker
> thread that I want to run at idle priority most of the time, but if it
> falls too far behind I'd like to bump it back up to regular timeshare
> priority until it catches up.  In a worst case, the system is
> continuously busy and something scheduled at idle priority is never
> going to run (for some definition of 'never').  
> 
> What I found is that in this worst case, even after my main thread has
> used rtprio_thread() to change the worker thread back to
> RTP_PRIO_NORMAL, the worker thread never gets scheduled.  This is with
> the 4BSD scheduler but it appears that the same would be the case with
> ULE, based on code inspection.  I find that this fixes it for 4BSD, and
> I think the same would be true for ULE...
> 
> --- a/sys/kern/sched_4bsd.c	Wed Feb 13 12:54:36 2013 -0700
> +++ b/sys/kern/sched_4bsd.c	Fri Feb 22 11:55:35 2013 -0700
> @@ -881,6 +881,9 @@ sched_user_prio(struct thread *td, u_cha
>  		return;
>  	oldprio = td->td_user_pri;
>  	td->td_user_pri = prio;
> +	if (td->td_flags & TDF_BORROWING && td->td_priority <= prio)
> +		return;
> +	sched_priority(td, prio);
>  }
>  
>  void
> 
> But I'm not sure if this would have any negative side effects,
> especially since in the ULE case there's a comment on this function that
> specifically notes that it changes the the user priority without
> changing the current priority (but it doesn't say why that matters).
> 
> Is this a reasonable way to fix this problem, or is there a better way?

This will lose the "priority boost" afforded to interactive threads when they 
sleep in the kernel in the 4BSD scheduler.  You aren't supposed to drop the
user priority to loose this boost until userret().  You could perhaps try
only altering the priority if the new user pri is lower than your current
priority (and then you don't have to check TDF_BORROWING I believe):

	if (prio < td->td_priority)
		sched_priority(td, prio);

-- 
John Baldwin