[FreeBSD 8.2 amd64 XENHVM] DomU terrible network performance trought NAT

[Laurent Cligny]

Thanks for your quick reply.
> Maybe you should have a look on the tcp window size values.
> I had such a behavior with two gateways who was not playing the same way
> with the TCP window size which made my network damn slow.
> If you have a linux gateway in front of your FreeBSD VM, it should be this.
Well I compared the TCP default values between the FreeBSD Vm and the
Linux PV VM and changed the net.inet.tcp.recvspace and
net.inet.tcp.sendpace to 124928 wich are the values on the Linux VM, but
there's no change on the network performance. Nice idea, much appreciated.
> Command
> Description
> sysctl net.inet.tcp.rfc1323=1 Activate window scaling and timestamp options
> according to RFC 1323.sysctl ipc.maxsockbuf=*[sbmax]* Set maximum size of
> TCP window.sysctl net.inet.tcp.recvspace=*[wstd]* Set default size of TCP
> receive window.sysctl net.inet.tcp.sendspace=*[wstd]* Set default size of
> TCP transmit window.sysctl kern.ipc.nmbclusters View maximum number of mbuf
> clusters. Used for storage of data packets to/from the network interface.
> Can only be set att boot time - see above. sysctl net.inet.tcp.liondmask=7Used
> to enable the "netlion" patch above.
>
> Cordialement,
> *Guillaume Seigneuret*
>
>
> *Network and System Security Architect*
>  *Web* :          http://www.omegacube.fr
>  *Address* :
> Hôtel Technologique Marseille Innovation - BP 100
> 45 rue Frederic Joliot Curie
> Technopôle de Château Gombert
> 13382 Marseille Cedex 13
>
>
>
> 2011/5/12 Laurent Cligny <laurent.cligny at steadinet.fr>
>
>> Hello all,
>>
>> I want to setup a FreeBSD DomU VM as a network NAT gateway in order to
>> provide Internet access to other FreeBSD and Linux DomU VMs.
>> My Dom0 is Xen Cloud Platform 1.0 on a Dell Poweredge 210 with 8 cores,
>> 16GiB RAM and one NIC.
>>
>> All FreeBSD VM are 8.2 amd64 with XENHVM kernel anf the Linux VM is a
>> Paravirtualized Debian amd64.
>>
>> The NAT gateway is a two-NIC box, with one (xn0) configured with a
>> public Internet address and the other (xn1) configured with a class A
>> private address (10.0.0.254).
>> The Internet traffic is very good from my FreeBSD NAT gateway to the
>> Internet (~ 50MiB/s) and also between other VMs on the private network
>> (~ 70MiB/s trough scp) after applying the patch here
>> (http://www.mail-archive.com/freebsd-xen@freebsd.org/msg00855.html)
>> fixing the "too many frags" problem.
>>
>> Here is a schema of my setup with the observed net speed on it:
>>
>> ----------         ---------------         ----------------------
>> |Internet|---xn0---|FBSD DomU NAT|---xn1---|FBSD and Linux DomUs|
>> ----------         ---------------         ----------------------
>>          <------->               <------->
>>           50MiB/s                 70MiB/s
>>
>>          <------------------------------->
>>           0KiB/s <network speed< 5 KiB/s
>>
>> I tried doing NAT on the FBSD gateway with PF and also with NATd after
>> shutting down PF and unloaded the kernel modules pf.ko and pflog.ko, but
>> I have the same perf problems with both solutions. Also I tried
>> disabling net.inet.tcp.tso and did ifconfig -tso on all NICs on NAT
>> gateway and other VMs without change.
>>
>> I don't see anything on the error console or in /var/log/message.
>>
>> When i'm trying going though NAT with a Linux paravirt DomU, the network
>> perf is slightly better but don't go over 5KiB/s.
>>
>> Finally I put my Linux paravirt DomU VM as a NAT gateway in place of the
>> FreeBSD one, and the network perf is now normal through it (50MiB/s)
>> from the FreeBSD DomU VMs.
>>
>> As I'm very happy with FreeBSD for my environment I'd like to keep as
>> much as this lovely OS for all my needs, so if anyone have an idea where
>> the problem can lies, and what to test next to have a clue of what's
>> happening, I will be very thankfull.
>>
>> Thanks in advance.
>>
>> --
>> /Regards,/
>>
>> *Laurent Cligny
>> /Founder/*
>> 02 97 37 71 14
>> 06 25 40 86 48
>> laurent.cligny at steadinet.fr
>>
>> Steadinet <http://www.steadinet.fr>
>>
>>
>> _______________________________________________
>> freebsd-xen at freebsd.org mailing list
>> http://lists.freebsd.org/mailman/listinfo/freebsd-xen
>> To unsubscribe, send any mail to "freebsd-xen-unsubscribe at freebsd.org"
>>
>>
> _______________________________________________
> freebsd-xen at freebsd.org mailing list
> http://lists.freebsd.org/mailman/listinfo/freebsd-xen
> To unsubscribe, send any mail to "freebsd-xen-unsubscribe at freebsd.org"