ULE realtime scheduler advice needed

Hi,

I'm doing some work with audio and have noticed some problems with the 
ULE scheduler. I have a program that generate audio based on 
key-presses. When no keys are pressed, the load is near 0%, but as soon 
as you start pressing keys, the load goes maybe to 80% of a CPU core. 
This program I run with rtprio 8 xxx. The issue I observe or hear 
actually, is that it takes too long until the scheduler grasps that this 
program needs it's own CPU core and stops time-sharing the program. When 
I however use cpuset -l xxx rtprio 8 yyy everything is good, and the 
program outputs realtime audio in-time.

Or is this perhaps a CPU frequency stepping issue?

Any advice on where to look?

--HPS