Issues with a Large Fat pipe Network simulation

[Luigi Rizzo]

On Tue, Jun 21, 2005 at 09:11:57PM +0200, Pieter de Boer wrote:
> Luigi Rizzo wrote:
> 
> > oh yes one thing... you are using 'via foo0' in your rule,
> > which means the packet is intercepted both in the input and
> > output path, which causes further contention on the queues.
> Well, when using 'ip from client to server recv em0', packets get 

i said 'in recv em0' - you missed the 'in' keyword.

> > I am pretty sure there is some issue there, also related to some
> > timing issues and tcp window opening mode (slow start vs. linear)
> I went to see if there were any sysctl's I could tune a bit. I found these:
> net.inet.ip.intr_queue_maxlen: 50
> net.inet.ip.intr_queue_drops: 382136
> 
> I don't like drops. So I set intr_queue_maxlen to 400, and -poof-, the 

whoops... of course, i forgot that one too... which is not much
of an issue if you use polling or bridging, that's why i forgot :)

> speed went up to around 700mbit/s. Still not as fast as it was with 64KB 
> send/recv spaces, but it's a huge improvement nonetheless.
> 
> I guess we probably should tune a bit more until we're confident that 
> the middle-box behaves correctly, before adding things like latency and 
> packet-loss :)
> 
> Thanks for the advice! If you know other settings to tune on the 
> dummynetting host, I'd very much like to hear them. I'm pondering about 
> polling (which means we can't do SMP on the dummynet system, but it's 
> only pushing packets, so that shouldn't matter too much). At least we 
> have somewhat more info to work with now :)

yes you should definitely enable polling if you can, and forget about
smp - it's a router anyways, and multiple processors won't help.

cheers
luigi