Possible bug in TSO or in pf on bce

[Pyun YongHyeon]

On Mon, Feb 15, 2010 at 10:11:41PM +0100, Albert Shih wrote:
> Hi all, 
> 
> I'm not a tcp/ip guru, so I don't known if it's a bug or not.
> 
> The situation is little complexe, so I'm going to explain that. 
> 
> I've one server with tree interfaces two bce and one bge. All test is on
> two bce.
> 
> This server running FreeBSD-7.2-p6 and have lot of jail (but the problem is
> the same for one jail, so I assume I've just one jail). The bce0 and bce1
> are in different vlan. 
> 
> The jail is on bce1 (meaning the jail IP is on the bce1 subnet). 
> 
> The default gateway is on bce0
> 
> So to make all traffic of the jail pass only throught bce1 and not using
> bce0 I'm using pf with something like 
> 
> 	pass out route-to (bce1 bce1_subnet_gw) from jail_IP to ! bce1_subnet keep state
> 	pass in on bce1 reply-to (bce1 bce1_subnet_gw) from ! bce1_subnet to jail_IP keep state
> 
> if I do that all traffic pass through the right interface (bce1), but...the
> bandwith drop to  ~60kb/s (on gigabit interface). 
> 
> So I find the problem is with TSO, if I deactivated the TSO the bandwith is
> return to normal. 
> 
> I don't knwon if it's a bug in PF (the problem is same if I use scrub or
> not) or in the TSO support of bce.
> 

At first I thought you hit one of edge case of TSO on bce(4). But
it seems the issue comes from pf's route handling. When I ported pf
from OpenBSD, there was no TSO capability in FreeBSD at that time
so the pf_route() had no special handling code for TSO. Since it
was long time ago I'm not sure whether it's correct or not but try
attached patch.

Apart from TSO FreeBSD got several new features like fib, 
flow-table and vnet. We may need to check whether these new
features are still working with pf(4).
-------------- next part --------------
A non-text attachment was scrubbed...
Name: pf.routeto.patch
Type: text/x-diff
Size: 790 bytes
Desc: not available
Url : http://lists.freebsd.org/pipermail/freebsd-pf/attachments/20100216/f4cdad9f/pf.routeto.bin