Xen (and others Hypervisors) how do they handle IPIs?

[Julian Elischer]

On 9/26/13 5:50 PM, Roger Pau Monné wrote:
> On 26/09/13 03:48, Julian Elischer wrote:
>> On 9/25/13 11:04 PM, Roger Pau Monné wrote:
>>> On 25/09/13 16:56, Julian Elischer wrote:
>>>> If CPUs are mapped around, how are IPIs handled? I assume they must be
>>>> emulated?
>>>>
>>>> I've noticed that under Xen (on both Amazon EC2 and a Redhat server)
>>>> whenever you schedule a thread it always sits on the run queue for 20
>>>> uSecs before it starts running. It looks to me like it's the IPI taking
>>>> a long time to be emulated.
>>> This has been improved on the FreeBSD Xen PVHVM port by using PV IPIs
>>> instead of the emulated ones, see r255331. It should be faster than the
>>> previous emulated implementation.
>>>
>> I missed that.. thanks!
>> Do you (or anyone else) know if this can be used on Amazon EC2?
>> And do you need a specific version/configuration of Xen to be able to
>> use it?

I ran a new GENERIC kernel on an Amazon Xen 4.2 system and it 
performed a bit better.
delays in the running of threads reduced from a constant 20uSecs to a 
variable amount between 1.5uSecs and 9uSecs.
It's still very slow when compared with real hardware.
(timing viewed using Schedgraph)

Is there a way to see whether PV IPIs are active?
(it looks it from the results but that could be so many other things too).

Julian
> The PV IPIs require Xen version 4.0 or greater, and the PV timer
> requires 4.0.1 or greater if I'm not mistaken.
>
> If you are lucky to get an Amazon instance that uses this Xen version
> (or any superior one) they will be activated by default, if not FreeBSD
> will switch to the old event delivery method and PV IPIs and PV timer
> will be disabled in favour of the emulated ones.
>
> I guess it's just a matter of time before Amazon switches all their
> servers to Xen 4.x.
>
> Roger.
>
>
>