[FreeBSD 8.2 amd64 XENHVM] DomU terrible network performance trought NAT

[Guillaume Seigneuret]

Maybe you should have a look on the tcp window size values.
I had such a behavior with two gateways who was not playing the same way
with the TCP window size which made my network damn slow.
If you have a linux gateway in front of your FreeBSD VM, it should be this.

Command
Description
sysctl net.inet.tcp.rfc1323=1 Activate window scaling and timestamp options
according to RFC 1323.sysctl ipc.maxsockbuf=*[sbmax]* Set maximum size of
TCP window.sysctl net.inet.tcp.recvspace=*[wstd]* Set default size of TCP
receive window.sysctl net.inet.tcp.sendspace=*[wstd]* Set default size of
TCP transmit window.sysctl kern.ipc.nmbclusters View maximum number of mbuf
clusters. Used for storage of data packets to/from the network interface.
Can only be set att boot time - see above. sysctl net.inet.tcp.liondmask=7Used
to enable the "netlion" patch above.

Cordialement,
*Guillaume Seigneuret*



*Network and System Security Architect*
 *Web* :          http://www.omegacube.fr
 *Address* :
Hôtel Technologique Marseille Innovation - BP 100
45 rue Frederic Joliot Curie
Technopôle de Château Gombert
13382 Marseille Cedex 13



2011/5/12 Laurent Cligny <laurent.cligny at steadinet.fr>

> Hello all,
>
> I want to setup a FreeBSD DomU VM as a network NAT gateway in order to
> provide Internet access to other FreeBSD and Linux DomU VMs.
> My Dom0 is Xen Cloud Platform 1.0 on a Dell Poweredge 210 with 8 cores,
> 16GiB RAM and one NIC.
>
> All FreeBSD VM are 8.2 amd64 with XENHVM kernel anf the Linux VM is a
> Paravirtualized Debian amd64.
>
> The NAT gateway is a two-NIC box, with one (xn0) configured with a
> public Internet address and the other (xn1) configured with a class A
> private address (10.0.0.254).
> The Internet traffic is very good from my FreeBSD NAT gateway to the
> Internet (~ 50MiB/s) and also between other VMs on the private network
> (~ 70MiB/s trough scp) after applying the patch here
> (http://www.mail-archive.com/freebsd-xen@freebsd.org/msg00855.html)
> fixing the "too many frags" problem.
>
> Here is a schema of my setup with the observed net speed on it:
>
> ----------         ---------------         ----------------------
> |Internet|---xn0---|FBSD DomU NAT|---xn1---|FBSD and Linux DomUs|
> ----------         ---------------         ----------------------
>          <------->               <------->
>           50MiB/s                 70MiB/s
>
>          <------------------------------->
>           0KiB/s <network speed< 5 KiB/s
>
> I tried doing NAT on the FBSD gateway with PF and also with NATd after
> shutting down PF and unloaded the kernel modules pf.ko and pflog.ko, but
> I have the same perf problems with both solutions. Also I tried
> disabling net.inet.tcp.tso and did ifconfig -tso on all NICs on NAT
> gateway and other VMs without change.
>
> I don't see anything on the error console or in /var/log/message.
>
> When i'm trying going though NAT with a Linux paravirt DomU, the network
> perf is slightly better but don't go over 5KiB/s.
>
> Finally I put my Linux paravirt DomU VM as a NAT gateway in place of the
> FreeBSD one, and the network perf is now normal through it (50MiB/s)
> from the FreeBSD DomU VMs.
>
> As I'm very happy with FreeBSD for my environment I'd like to keep as
> much as this lovely OS for all my needs, so if anyone have an idea where
> the problem can lies, and what to test next to have a clue of what's
> happening, I will be very thankfull.
>
> Thanks in advance.
>
> --
> /Regards,/
>
> *Laurent Cligny
> /Founder/*
> 02 97 37 71 14
> 06 25 40 86 48
> laurent.cligny at steadinet.fr
>
> Steadinet <http://www.steadinet.fr>
>
>
> _______________________________________________
> freebsd-xen at freebsd.org mailing list
> http://lists.freebsd.org/mailman/listinfo/freebsd-xen
> To unsubscribe, send any mail to "freebsd-xen-unsubscribe at freebsd.org"
>
>