high load in KVM when network is used (stable/10)

[Armin Pirkovitsch]

On 05/06/14 16:59, Bryan Venteicher wrote:
> 
> 
> ----- Original Message -----
>> From: "Armin Pirkovitsch" <sperber at FreeBSD.org>
>> To: stable at freebsd.org
>> Sent: Monday, May 5, 2014 2:18:39 PM
>> Subject: high load in KVM when network is used (stable/10)
>>
>> Hi,
>>
>> has anyone ever experienced a problem with FreeBSD 10 (release as well
>> as stable) in KVMs? (in relation to network activity)
>>
>> Currently the load of my server goes up when I just fetch a file.
>>
>> systat -vmstat
>>>     3 users    Load  0.57  0.28  0.28                  May  5 21:04
>>>
>>> Mem:KB    REAL            VIRTUAL                       VN PAGER   SWAP
>>> PAGER
>>>         Tot   Share      Tot    Share    Free           in   out     in
>>>         out
>>> Act  714476   27764  2951164    33696  14130k  count
>>> All 2002012   83188  3211064   209272          pages
>>> Proc:                                                            Interrupts
>>>   r   p   d   s   w   Csw  Trp  Sys  Int  Sof  Flt        ioflt  4575 total
>>>   2         102       11k   17  25k 4046   38             cow
>>>   atkbd0 1
>>>                                                           zfod        uhci0
>>>                                                           11
>>> 28.6%Sys  14.3%Intr  1.4%User  0.0%Nice 55.7%Idle         ozfod       ata1
>>> 15
>>> |    |    |    |    |    |    |    |    |    |           %ozfod   121
>>> |    |    |    |    |    |    |    |    |    |           cpu0:timer
>>> ==============+++++++>                                    daefr  3255
>>> virtio_pci
>>>                                            dtbuf          prcfr
>>>                                            virtio_pci
>>> Namei     Name-cache   Dir-cache    350329 desvn          totfr   790
>>> virtio_pci
>>>    Calls    hits   %    hits   %      6556 numvn          react   116
>>>    cpu3:timer
>>>        6       6 100                  4181 frevn          pdwak   185
>>>        cpu2:timer
>>>                                                       210 pdpgs   108
>>>                                                       cpu1:timer
>>> Disks vtbd0   cd0 pass0                                   intrn
>>> KB/t    120  0.00  0.00                           1382576 wire
>>> tps     788     0     0                            504540 act
>>> MB/s  92.55  0.00  0.00                            291232 inact
>>> %busy    49     0     0                                   cache
>>>                                                  14130808 free
>>>                                                           buf
>>
>> it's even worse when I access apache on that server:
>>>     3 users    Load  4.63  1.33  0.66                  May  5 21:06
>>>
>>> Mem:KB    REAL            VIRTUAL                       VN PAGER   SWAP
>>> PAGER
>>>         Tot   Share      Tot    Share    Free           in   out     in
>>>         out
>>> Act  727760   27764  2938852    33696  14116k  count
>>> All 2015420   83188  3198752   209272          pages
>>> Proc:                                                            Interrupts
>>>   r   p   d   s   w   Csw  Trp  Sys  Int  Sof  Flt        ioflt     8 total
>>>             103        14    4   49    2    1             cow
>>>             atkbd0 1
>>>                                                           zfod        uhci0
>>>                                                           11
>>> 68.8%Sys  31.2%Intr  0.0%User  0.0%Nice  0.0%Idle         ozfod       ata1
>>> 15
>>> |    |    |    |    |    |    |    |    |    |           %ozfod     1
>>> |    |    |    |    |    |    |    |    |    |           cpu0:timer
>>> ==================================++++++++++++++++        daefr     4
>>> virtio_pci
>>>                                            dtbuf          prcfr
>>>                                            virtio_pci
>>> Namei     Name-cache   Dir-cache    350329 desvn          totfr     1
>>> virtio_pci
>>>    Calls    hits   %    hits   %      6557 numvn          react
>>>    cpu3:timer
>>>        6       6 100                  4136 frevn          pdwak     1
>>>        cpu2:timer
>>>                                                       136 pdpgs     1
>>>                                                       cpu1:timer
>>> Disks vtbd0   cd0 pass0                                   intrn
>>> KB/t    128  0.00  0.00                           1382808 wire
>>> tps       1     0     0                            506072 act
>>> MB/s   0.11  0.00  0.00                            303460 inact
>>> %busy    33     0     0                                   cache
>>>                                                  14116816 free
>>>                                                           buf
>>
>> and you couldn't say that it just hasn't got enough power...
>> ...
>>> CPU: Intel(R) Xeon(R) CPU E5-2620 0 @ 2.00GHz (2000.01-MHz K8-class CPU)
>>>   Origin = "GenuineIntel"  Id = 0x206d7  Family = 0x6  Model = 0x2d
>>>   Stepping = 7
>>>   Features=0xf83fbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,MMX,FXSR,SSE,SSE2,SS>
>>>   Features2=0x9eb82203<SSE3,PCLMULQDQ,SSSE3,CX16,SSE4.1,SSE4.2,x2APIC,POPCNT,AESNI,XSAVE,OSXSAVE,AVX,HV>
>>>   AMD Features=0x2c100800<SYSCALL,NX,Page1GB,RDTSCP,LM>
>>>   AMD Features2=0x1<LAHF>
>>> real memory  = 17179869184 (16384 MB)
>>> avail memory = 16597995520 (15829 MB)
>>> Event timer "LAPIC" quality 400
>>> ACPI APIC Table: <BOCHS  BXPCAPIC>
>>> FreeBSD/SMP: Multiprocessor System Detected: 4 CPUs
>>> FreeBSD/SMP: 4 package(s) x 1 core(s)
>> ...
>>
>> Has someone any suggestions what I could do to improve the performance
>> (and lower the load)?
>>
> 
> 
> This will sound a bit odd - are you using DHCP on this interface? This might
> be another instance of http://www.freebsd.org/cgi/query-pr.cgi?pr=185864. If
> so, can you try with either a static IP or by applying in the workaround
> patch in the PR?

Nope, it's a static ip.

I as well already tried to enable LRO - which made no difference.

Should I try that patch anyway?