FreeBSD 10-R, Xen 4.1 guest, pf/NAT performance question

seanrees at seanrees at
Tue Apr 8 20:55:48 UTC 2014

I did notice one more odd thing today:
  Good VPS: throughput at ~5mbps:  ~1800 context switches
  Bad VPS: throughput at 0.5mbps: ~240 context switches

measured using systat -v.

In general, the number of context switches, traps, and system calls were
about an order of magnitude more on the good VPS vs. the bad VPS.

I'm wondering if this perhaps rings any bells? (or opens up avenues of
tuning or investigation)


On Mon, Apr 7, 2014 at 3:50 PM, seanrees at <seanrees at>wrote:

> Thanks for the tip. Yes, I had already run into TSO4 causing issues with
> this VM, so it was switched off (ifconfig xn0 -tso4).
> I also set net.inet.tcp.tso=0 (was 1) and tried again - no change. :(
> Sean
> On Mon, Apr 7, 2014 at 3:10 PM, Mark Felder <feld at> wrote:
>> On 2014-04-07 07:57, seanrees at wrote:
>>> Hi there freebsd-questions,
>>> I've been batting my head against this problem for a few days now and not
>>> having much progress, so I'm hoping to get pointers at what to look at
>>> next.
>>> I've got a FreeBSD 10-R guest in Xen 4.1 (I am just a customer of the Xen
>>> provider; I don't run the Xen hypervisor myself). I use this instance to
>>> terminate a VPN, for which I also NAT VPN clients with PF. I am seeing
>>> unusually slow packet forwarding performance: 0.5mbit internet -> vpn
>>> client, 2.0 mbit vpn client -> internet. (the numbers should be closer to
>>> 10mbit/5mbit).
>>> This guest is a duplicate of another Xen instance I have in another data
>>> centre. I manage the configurations and packages centrally and aside from
>>> IP address differences, the machines are configured identically. The
>>> differences: it's 30ms closer to me and runs in Xen 3.4. I see
>>> performance
>>> from this machine in the 10mbps range.
>>> I've eliminated the obvious:
>>>   - The problem VPS is fine network wise; can download tarballs from the
>>> Internet at 100mbps.
>>>   - VPS -> Home is fine; can download at ~10mbps; the problem is isolated
>>> to forwarding Home -> VPS -> Internet and back.
>>>   - I excluded OpenVPN as the cause by replicating the setup with ssh -w;
>>> same performance.
>>>   - SSH port forwarding (ssh -L) is fast; indicating to me the issue is
>>> somewhere in the PF/kernel.
>>>   - I checked TCP options by capturing traffic at varying points; these
>>> seem fine. I see a good deal of TCP retransmits but the window sizes stay
>>> the same.
>>> Any thoughts on what to check next?
>> Have you turned off TSO?
>> ifconfig xn0 -tso
>> or
>> sysctl net.inet.tcp.tso=0
>> _______________________________________________
>> freebsd-questions at mailing list
>> To unsubscribe, send any mail to "freebsd-questions-
>> unsubscribe at"

More information about the freebsd-questions mailing list