slow transfers on webshare service

[Miroslav Lachman]

Ivan Voras wrote:
> Miroslav Valenta wrote:
> 
>> Hi
>> i have problem with slow transfers on my web filesharing server. I'm
>> running FreeBSD 6.2
>> files are sending by lighttpd 1.4.18 :
>>
>> server.max-worker = 8
>> server.max-fds = 8192
>> server.network-backend = "writev"
>>
>> -----
>>
>> HW:
>> xeon 3160 4GB of RAM
>> areca arc-1260 disk controller with 1GB cache + 8x 500GB sata2 hdd
>>
>>
>> file transfers slow down when i'm about 500 downloading connections
>> but when i send same file from same storage by same line it's  sending
>> fast.
> 
> 
> Please characterize "slow" and "fast" - how is it slow and how fast do 
> you think it should be?
> 
> Some general tips/ideas:
> 
> - Do you really need 8 workers? Lighttpd is async server so it's about 
> as fast as it gets. If you only serve static files, larger number of 
> worker processes (and CPUs...) won't make a difference.
> - Do you use the "kqueue" extension for Lighttpd?
> - 500 parallel downloads probably mean lots of seeking on the disk drive 
> array, have you verified you can sustain the speed you need with the 
> drives?

I am running Lighttpd for about 2 years on download server. With 1 
worker, lighttpd seems limited to 110Mbps. After change to 4 workers
throughput increase to about 190Mbps serving 250-400 clients. (daily 
traffic is ~750GB)

I am using these settings:
server.event-handler = "freebsd-kqueue" # needed on OS X
#server.network-backend = "freebsd-sendfile" # better for small files
server.network-backend = "writev" # better for large files?
server.max-keep-alive-requests = 6
server.max-keep-alive-idle = 5
server.max-read-idle = 60
server.max-write-idle = 180

Miroslav Lachman