Throughput problems with dummynet and high delays

[Pieter de Boer]

Andy Jones wrote:


> What happens is as the TCP window grows larger (about 3-4MB), the sender
> spends most of its time processing interrupts (80-90% as reported by top)
> and throughput peaks at about 300Mbps. I've dug into the dummynet code and
> I've found that a large amount of time is spent in the routine
> transmit_event(struct dn_pipe *p) which dequeues packets from a pipe and
> calls ip_output. It appears that ip_output is the culprit, but what it is
> doing with its time, I'm not sure. Packet drops are not being lost 
> according
> to TCP and dummynet. I suspect either pfil_run_hooks(...) or (*
> ifp->if_output) (...) calls in ip_output are taking too much time, but I'm
> not sure.
> 
> Any suggestions on what could be happening would be appreciated!
> 
Deja vu :)

You're doing exactly the same stuff a fellow student and me did over a 
year ago; even using the same hardware. Too bad the report was in Dutch, 
otherwise a reference may have been useful. We used 5.4 and MTU 1500, 
differing from your setup.

What we saw was that the sending machine had major issues. When we 
pinged it on a interface different from where the tests were being run 
on, we saw a lot of packet loss and very high (read: up to 5 -seconds-) 
latency. As you say, the kernel was busy trying to get the packets out, 
but didn't have time for anything else. To us it appeared the TCP code 
became very slow with growing buffers, although we didn't find out if 
this was in the output path or in processing the incoming 
acknowledgments. If I recall correctly, using delayed acknowledgments on 
the receiving system did not have any influence on the sending systems, 
indicating that the problem may be in the output path rather in 
processing the acks.

Sorry to only have a 'me too!'-response, but let's hope someone could 
help out here..

-- 
Pieter