Poor virtio performance on Scaleway ARM systems

[Peter Jeremy]

On 2018-Oct-23 18:46:05 +0300, Greg V <greg at unrelenting.technology> wrote:
>echo 'hw.pci.honor_msi_blacklist="0"' >> /boot/loader.conf
>
>With the fix, we're getting ~1000 interrupts on 'its0,4: virtio_pci1' 
>(instead of ~30000 on gic), latency is no longer awful, and bandwidth 
>is about 220 MB/s. YAY!!

Unfortunately, this isn't working for me on 13-CURRENT.  I am still
seeing ~4.5MBps disk I/O with ~30,000 interrupts/sec on gic0,s5.  I
don't even see an its0,4.  

Looking at a verbose boot, the relevant errors are:

its0: <ARM GIC Interrupt Translation Service> mem 0x8080000-0x809ffff on gic0
its0: GITS_BASER0: unable to be updated: a907000040a40607 != a907000040ee0607
device_attach: its0 attach returned 6
...
virtio_pci0: <VirtIO PCI Network adapter> port 0xf040-0xf05f mem 0x3ee09000-0x3ee09fff,0x3ee00000-0x3ee03fff at device 1.0 on pci0
vtnet0: <VirtIO Networking Adapter> on virtio_pci0
virtio_pci0: host features: 0x79bfffe7 <EventIdx,RingIndirect,0x8000000,NotifyOnEmpty,SetMacAddress,GuestAnnounce,RxModeExtra,VLanFilter,RxMode,ControlVq,Status,MrgRxBuf,TxUFO,TxTSOECN,TxTSOv6,TxTSOv4,RxUFO,RxECN,RxTSOv6,RxTSOv4,TxAllGSO,MacAddress,0x4,RxChecksum,TxChecksum>
virtio_pci0: negotiated features: 0x308fbbe3 <EventIdx,RingIndirect,SetMacAddress,VLanFilter,RxMode,ControlVq,Status,MrgRxBuf,TxTSOECN,TxTSOv6,TxTSOv4,RxECN,RxTSOv6,RxTSOv4,TxAllGSO,MacAddress,RxChecksum,TxChecksum>
vtnet0: bpf attached
vtnet0: Ethernet address: de:2b:88:06:a0:4c
virtio_pci0: attempting to allocate 3 MSI-X vectors (3 supported)
virtio_pci0: attempting to allocate 2 MSI-X vectors (3 supported)
virtio_pci0: using legacy interrupt
virtio_pci1: <VirtIO PCI Block adapter> port 0xf000-0xf03f mem 0x3ee08000-0x3ee08fff,0x3ee04000-0x3ee07fff at device 2.0 on pci0
vtblk0: <VirtIO Block Adapter> on virtio_pci1
virtio_pci1: host features: 0x79000e54 <EventIdx,RingIndirect,0x8000000,NotifyOnEmpty,ConfigWCE,Topology,WriteCache,BlockSize,DiskGeometry,MaxNumSegs>
virtio_pci1: negotiated features: 0x10000e54 <RingIndirect,ConfigWCE,Topology,WriteCache,BlockSize,DiskGeometry,MaxNumSegs>
virtio_pci1: attempting to allocate 2 MSI-X vectors (2 supported)
virtio_pci1: attempting to allocate 2 MSI-X vectors (2 supported)
virtio_pci1: using legacy interrupt
vtblk0: 47683MB (97656250 512 byte sectors)

I haven't had a chance to dig into either of these any further but hope to do so
over the weekend.
--
Peter Jeremy
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 963 bytes
Desc: not available
URL: <http://lists.freebsd.org/pipermail/freebsd-arm/attachments/20181026/74af41bd/attachment.sig>