KVM+FreeBSD+82599 10G NIC doesn't work.
Jack Vogel
jfvogel at gmail.com
Thu May 16 08:02:32 UTC 2013
The environment that I tested in when the virtualization support was put
into
the driver was using SRIOV, not pass-through, we have never validated the
latter, and whenever it crops up it seems to be having problems.
I can put it into my queue to look into, but I can't say how quickly I can
get
to it, very busy here.
Jack
On Thu, May 16, 2013 at 12:45 AM, GemaEvor <legumen at foxmail.com> wrote:
> Hi gurus:
>
>
> I'm working on a FreeBSD virtulizaion project under KVM.
> Everything works fine, except the 10G Intel 82599 SFP NIC.
>
>
> I'm sure there are some body else who have encountered this problem
> before. See:
> FreeBSD guest with VTD NIC not passing traffic
> http://comments.gmane.org/gmane.comp.emulators.kvm.devel/83806
> VT-d not working for FreeBSD 9.0 guest
> http://permalink.gmane.org/gmane.comp.emulators.kvm.devel/91074
>
>
> But, unfortunately, none of them solved it or gave the clearly way to
> solve it. :<
>
>
> Firstly, let's make it clearly. This problem could not be reproduced with:
> 1. Xen hypervisor + FreeBSD guest os
> 2. or KVM + Linux guest os (e.g. Fedora 12)
> So, it must be something wrong with KVM or FreeBSD.
>
>
> Here is my environment:
> Host OS:
> OpenSUSE 12.2
> Linux linux-kj2u 3.4.6-2.10-desktop #1 SMP PREEMPT Thu Jul 26 09:36:26 UTC
> 2012 (641c197) x86_64 x86_64 x86_64 GNU/Linux
> QEMU emulator version 1.1.1 (kvm-1.1.1-1.8.1), Copyright (c) 2003-2008
> Fabrice Bellard
>
>
> Guest OS:
> FreeBSD 9.1 amd64
>
>
> What I did:
> 1.Make the pci device of 82599SFP assignale:
> [CODE]
> 0d:00.0 Ethernet controller: Intel Corporation 82599EB 10-Gigabit SFI/SFP+
> Network Connection (rev 01)
> Subsystem: QLogic, Corp. Device 00e4
> Flags: fast devsel, IRQ 25
> Memory at f8fe0000 (64-bit, prefetchable) [size=128K]
> I/O ports at cc00 [size=32]
> Memory at f8fdc000 (64-bit, prefetchable) [size=16K]
> Capabilities: [40] Power Management version 3
> Capabilities: [50] MSI: Enable- Count=1/1 Maskable+ 64bit+
> Capabilities: [70] MSI-X: Enable- Count=64 Masked-
> Capabilities: [a0] Express Endpoint, MSI 00
> Capabilities: [e0] Vital Product Data
> Capabilities: [100] Advanced Error Reporting
> Capabilities: [140] Device Serial Number 00-00-00-ff-ff-00-00-00
> Capabilities: [150] Alternative Routing-ID Interpretation (ARI)
> Capabilities: [160] Single Root I/O Virtualization (SR-IOV)
> Kernel driver in use: pci-stub
> [/CODE]
>
>
> 2. Passthrough it to the guest FreeBSD os. Here is my vm configuration file
> [FILE]
> <domain type='kvm'>
> <name>FreeBSD9.1</name>
> <uuid>4d56613e-400c-f37d-2a9a-e0332d81ca63</uuid>
> <memory>2097152</memory>
> <currentMemory>2097152</currentMemory>
> <vcpu>2</vcpu>
> <os>
> <type arch='x86_64' machine='pc'>hvm</type>
> <boot dev='hd'/>
> </os>
> <features>
> <pae/>
> <apic/>
> <acpi/>
> </features>
> <clock offset='utc'>
> </clock>
> <on_poweroff>destroy</on_poweroff>
> <on_reboot>restart</on_reboot>
> <on_crash>destroy</on_crash>
> <devices>
> <emulator>/usr/bin/qemu-kvm</emulator>
> <disk type='file' device='disk'>
> <driver name='qemu' type='raw' cache='default'/>
> <source file='/home/vm/FreeBSD9.1/FreeBSD9.1.raw'/>
> <target dev='hdc'/>
> </disk>
> <interface type='bridge'>
> <source bridge='br0'/>
> <mac address='52:54:00:0d:92:6f'/>
> <model type='rtl8139'/>
> </interface>
> <hostdev mode='subsystem' type='pci' managed='yes'>
> <source>
> <address domain='0x0000' bus='0x0d' slot='0x00' function='0x0'/>
> </source>
> </hostdev>
> <input type='mouse' bus='ps2'/>
> <graphics type='vnc' port='-1' autoport='yes'/>
>
> </devices>
> </domain>
> [/FILE]
>
>
> 3.The device could be probed and attached by FreeBSD:
> [CODE]
> ix0: <Intel(R) PRO/10GbE PCI-Express Network Driver, Version - 2.4.8> port
> 0xc020-0xc03f mem 0xfe000000-0xfe01ffff,0xfe020000-0xfe023fff irq 11 at
> device 4.0 on pci0
> ix0: Using MSIX interrupts with 2 vectors
> ix0: Ethernet address: 00:0c:bd:05:7d:4a
> ix0: PCI Express Bus: Speed 5.0Gb/s Width x8
> [/CODE]
>
>
> But no interrupt could be found:
> [CODE]
> interrupt total rate
> irq1: atkbd0 143 0
> irq4: uart0 177 1
> irq15: ata1 1593 10
> cpu0:timer 7140 46
> Total 9053 58
> [/CODE]
>
>
> And ix0 keeps to be "no carrier" (even if I config an IP address on it and
> ping):
> [CODE]
> ix0: flags=8802<BROADCAST,SIMPLEX,MULTICAST> metric 0 mtu 1500
>
> options=401bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,VLAN_HWTSO>
> ether 00:0c:bd:05:7d:4a
> nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
> media: Ethernet autoselect
> status: no carrier
> [/CODE]
>
>
> 4.In the host side, kvm could see the change made by the guest on the
> device, but no irq:
> [CODE]
> linux-kj2u:~ # cat /proc/interrupts
> CPU0 CPU1 CPU2 CPU3 CPU4 CPU5
> CPU6 CPU7 CPU8 CPU9 CPU10 CPU11 CPU12
> CPU13 CPU14 CPU15 CPU16 CPU17 CPU18
> CPU19 CPU20 CPU21 CPU22 CPU23
> 210: 0 0 0 0 0 0
> 0 0 0 0 0 0 0
> 0 0 0 0 0 0
> 0 0 0 0 0 PCI-MSI-edge
> kvm:0000:0d:00.0
> 211: 0 0 0 0 0 0
> 0 0 0 0 0 0 0
> 0 0 0 0 0 0
> 0 0 0 0 0 PCI-MSI-edge
> kvm:0000:0d:00.0
> [/CODE]
>
>
>
>
>
>
> So, anyone who has the same problem or has a clue to solve it, or has some
> ideas to do a debug job, please help.
>
>
> Thanks!
> _______________________________________________
> freebsd-virtualization at freebsd.org mailing list
> http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization
> To unsubscribe, send any mail to "
> freebsd-virtualization-unsubscribe at freebsd.org"
>
More information about the freebsd-virtualization
mailing list