Issues with a Large Fat pipe Network simulation

[Pieter de Boer]

Luigi Rizzo wrote:

>>When testing without any extra delay on 'network' and send/recvspaces of
>>65535 bytes, we can sustain around 800mbit/s. The interrupts on
>>'network' may be the limiting factor here. However, when we set the
>>send/recv space to 65535*2, we can only sustain around 200-300mbit/s. It
> 
> 
> have you checked that these two
> 	kern.ipc.maxsockbuf: 262144
> 	kern.ipc.sockbuf_waste_factor: 8
> 
> aren't responsible for the trouble ?

I've seen 200-300mbit when using a 1MB maxsockbuf. Raising it to 2, 4 or 
8MB doesn't seem to have much (if any) effect. Altering 
sockbuf_waste_factor doesn't seem to do anything, either.

However.. when I deleted the pipe rules on 'network', the speed suddenly 
went up to around 800mbit/s too! I remade them, and voila, 200mbit/s.

So apparantly it's an issue in the dummynet configuration I have. So, to 
reiterate:

On 'network':
pipe 1 from client to server via em0
pipe 2 from server to client via em1
allow ip from any to any

Will give me 200MBit/s when using 128KB window sizes, but 800MBit/s when 
using 64KB window sizes.

On 'network':
allow ip from any to any

Will give me 800Mbit/s when using 64KB or 128KB window sizes. I haven't 
configured the pipes in any way, though.


So it appears it's due to the 'network' box afterall..

-- 
Pieter