using ipfw seems to interfere with socket communication

[Robert Watson]

On Sun, 2 Jul 2006, Stefan Bethke wrote:

> Essentially, dummynet delays processing of that "two" line just long enough 
> to break the code's assumption that TCP over the loopback interface is 
> instantaneous. If my fading memory of TCP/IP Illustrated Vol 2 serves me 
> right, that was actually the case at least back then: the sendto system call 
> would push the data all the way down to lo0, which would immediately pass it 
> back up until it ended up in the receiving socket buffer.  Dummynet will 
> queue the packet, regardless, so it won't get delivered until the next time 
> dummynet processes queues.

With TCP (and UDP), the ability to immediately deliver has never been 
guaranteed due to socket buffer and window limits, scheduling priority, etc. 
TCP over the loopback interface really is TCP -- data is broken up into 
segments, wrapped in TCP/IP headers, processed by the firewall and dummynet, 
etc.  UNIX domain sockets should be used if it's desirable to avoid this, as 
that implementation delivers directly to the remote socket buffer.  However, 
due to socket buffer sizing and scheduling, blocking is possible there also. 
Stream sockets do not guarantee message boundary atomicity or immediate 
delivery, especially in the presence of SMP.

Robert N M Watson
Computer Laboratory
University of Cambridge