in-kernel tcp server

[Julian Elischer]

John Polstra wrote:
> Julian Elischer wrote:
> 
>> I would actually like to address the performance issues.
>>
>> is there any chance the oldest version (4.x based) might be released,
>> or at least it would be nice to get the code snippet that attaches to 
>> eh ng_ksocket and
>> reads and writes the stream..
>>
>> I could make a TCP ECHO node that way and use it for tracking down the 
>> bottlenecks
>> I'm not too interested in the actual webserver itself.
> 
> I don't have the ksocket version any more.  It was an early experiment 
> (in 2001) that I discarded pretty quickly.
> 
> The later 4.x-based version that bypassed the TCP stack and socket layer 
> performed well on uniprocessor systems.  I didn't feel netgraph was a 
> performance problem at all on that version.  But as multiprocessor 
> systems became more mainstream, the 4.x version wasn't able to take 
> advantage of the added CPUs.  Also, it didn't support ACPI and had 
> trouble booting on some of the newer hardware.
> 
> For those reasons, I updated to a 7.x-based system.  At that point, the 
> newer SMP-friendly netgraph started to impact performance pretty 
> seriously.  The allocation/deallocation of netgraph's queue items seemed 
> to be a big part of the problem.  In 4.x we just passed mbufs around, 
> without any other allocations or deallocations.  In 7.x, the mbufs are 
> wrapped up in queue items that have to be allocated and freed, and that 
> added a lot of overhead.  I think also that the reader-writer locking in 
> netgraph was impacting performance.  It's a really elegant locking 
> scheme, but my node graphs were so simple that I didn't really need it.

Hmm ok,

but if you did find some old ksocket based code sitting around,
i'd love to try it in -current and work on the bottlenecks..

I do need to try fix netgraph bottlenecks...

I'll certainly look at what I can do about the queue items.
I may make a per-cpu cache of them.

> 
> I don't view netgraph as having serious performance problems.  It's just 
> that I was aiming for maximal performance (in terms of HTTP sessions per 
> second), and was willing to do otherwise unreasonable things to get it.
> 
> John