Poor virtio performance on Scaleway ARM systems

[Greg V]

On Sun, Sep 16, 2018 at 9:46 PM, Peter Jeremy <peter at rulingia.com> 
wrote:
> I have been playing with the 4-core ARM64 VPSs on 
> https://www.scaleway.com
> and notice that the disk I/O performance (using virtio_blk) is 
> abyssmal.
> Using "dd if=/dev/{vda|vtbd0} of=/dev/null bs=256k count=4096", I get
> 400-500MBps, whilst under FreeBSD-12, I get about 5MBps.  I've 
> checked on a
> couple of instances and both Linux & FreeBSD on the same instance and 
> get
> similar results.  Linux & FreeBSD are both using a virtio block device
> attached to the PCI bus.  Rebuilding FreeBSD to turn off all the 
> debugging
> has no effect.
> 
> The only oddity I've found is that FreeBSD is reporting very high 
> interrupt
> rates on gic0,p11, gic0,s4 and gic0,s5 whilst disk I/O is occurring.
> Unfortunately, I can't tell what is attached to those interrupts (it's
> not obvious from the dmesg and reported as "+").


Hmm, about the only interesting thing I saw in dmesg:

its0: <ARM GIC Interrupt Translation Service> mem 0x8080000-0x809ffff 
on gic0
its0: GITS_BASER0: unable to be updated: a9070000afe80607 != 
a907000040ec0607
device_attach: its0 attach returned 6
its0: <ARM GIC Interrupt Translation Service> mem 0x8080000-0x809ffff 
on gic0
its0: Could not allocate memory
device_attach: its0 attach returned 6
[some repeated attach attempts]

But it's not deterministic, I managed to reboot without the error.