virtio-net issues on aarch64 QEMU/KVM

[Mark Rutland]

On Thu, Jul 27, 2017 at 04:29:54PM +0200, Andrea Bolognani wrote:
> Hi all,

Hi,

> I'm trying 11.1-RELEASE on QEMU/KVM, and I'm having some
> trouble with VirtIO networking.

Likewise on both counts.

As a (vaguely-related) heads-up, you will find that 11.1 will not work as an
SMP KVM guest, as it doesn't enable SGIs for the boot CPU in the GIC
distributor, leading to a lockup later in the boot process. I think Andrew
Turner is currently looking at fixing that.

> I've imported the official qcow2 image into libvirt by
> running
> 
>   # virt-install \
>     --name freebsd \
>     --arch aarch64 \
>     --machine virt \
>     --ram 2048 \
>     --vcpus 1 \
>     --accelerate \
>     --graphics none \
>     --features acpi=off \
>     --os-variant freebsd11.0 \
>     --import \
>     --disk path=/var/lib/libvirt/images/freebsd.qcow2,bus=virtio,address.type=virtio-mmio \
>     --network network=default,model=virtio,address.type=virtio-mmio
> 
> and the system comes up succesfully; however, when I
> try to activate the network interface by running
> 
>   # dhclient vtnet0
> 
> the process seems to pretty much just sit there. ^C
> doesn't help. Needless to say, if I configure vtnet0 to
> be brought up at boot I never get to the prompt.

I see the same thing with or without KVM using QEMU 2.9.0, upstream EDK2
(commit ef3d1df77bbd5227), and the same 11.1-RELEASE VM image. I'm not using
libvirt, but I see issues when pasing QEMU:

    -device virtio-net-device,netdev=net0 netdev user,id=net0

At boot time, FreeBSD stops making progress after "Feeding entropy: .", and if
I send SIGINFO with ^T, I see:

    load: 1.55  cmd: ifconfig 233 [runnable] 124.01r 0.00u 123.88s 100% 2768k

I dumped a stacktrace using QEMU's gdbserver; it looks like FreeBSD is polling
the virtio queue waiting for something:

#0  0xffff000000170294 in VIRTIO_BUS_POLL (dev=0xfffffd00005ab900) at ./virtio_bus_if.h:159
#1  virtqueue_poll (vq=0xffff00004083e000, len=0x0) at /usr/src/sys/dev/virtio/virtqueue.c:573
#2  0xffff00000017707c in vtnet_ctrl_mac_cmd (sc=0xfffffd00004fa000, hwaddr=<optimized out>) at /usr/src/sys/dev/virtio/network/if_vtnet.c:3163
#3  vtnet_set_hwaddr (sc=0xfffffd00004fa000) at /usr/src/sys/dev/virtio/network/if_vtnet.c:3588
#4  0xffff000000176568 in vtnet_reinit (sc=0xfffffd00004fa000) at /usr/src/sys/dev/virtio/network/if_vtnet.c:3022
#5  vtnet_init_locked (sc=0xfffffd00004fa000) at /usr/src/sys/dev/virtio/network/if_vtnet.c:3069
#6  0xffff000000175ff8 in vtnet_ioctl (ifp=0xfffffd000058f000, cmd=<optimized out>, data=<optimized out>) at /usr/src/sys/dev/virtio/network/if_vtnet.c:1107
#7  0xffff000000373e60 in ifhwioctl (cmd=<optimized out>, ifp=<optimized out>, td=<optimized out>, data=<optimized out>) at /usr/src/sys/net/if.c:2456
#8  ifioctl (so=<optimized out>, cmd=2149607696, data=0xffff000054ae1888 "vtnet0", td=<optimized out>) at /usr/src/sys/net/if.c:2836
#9  0xffff0000002f3b14 in fo_ioctl (fp=<optimized out>, com=2149607696, active_cred=<optimized out>, td=<optimized out>, data=<optimized out>) at /usr/src/sys/sys/file.h:323
#10 kern_ioctl (td=0xfffffd0000a7f000, fd=3, com=2149607696, data=0xffff000054ae1888 "vtnet0") at /usr/src/sys/kern/sys_generic.c:836
#11 0xffff0000002f377c in sys_ioctl (td=0xfffffd0000a7f000, uap=0xffff000054ae1978) at /usr/src/sys/kern/sys_generic.c:745
#12 0xffff000000555408 in syscallenter (td=<optimized out>, sa=<optimized out>) at /usr/src/sys/arm64/arm64/../../kern/subr_syscall.c:135
#13 svc_handler (frame=<optimized out>, td=<optimized out>) at /usr/src/sys/arm64/arm64/trap.c:139
#14 do_el0_sync (td=0xfffffd0000a7f000, frame=<optimized out>) at /usr/src/sys/arm64/arm64/trap.c:366

... unfortunately I'm not all that familiar with how virtio works, so I'm not
sure which end is doing the wrong thing.

Thanks,
Mark.