Real Insight on Performance

[Rahul]

Thanks Alexander,

I need multi-threaded support because I will need to share data
between threads. I am not just serving static pages, I am actually
performing data manipulation using data that comes in the request as
well as what's in memory. The memory could grow to at least 4 GB.
Actually, I would like to use at least 10GB for that process
(shouldn't be a problem with 16GB RAM on 64Bit OS with Intel Quad Core
- Q6600 or Q9450). Although, I do prefer to launch a thread and keep
it around for when there is work to do than launching a new process
for every request because launching a process is more expensive to
begin with and if it has to fetch data from parent process' memory
somehow, it only makes things worse.

I have used MySQL in the past and it has been able to handle large
volume of data without complains (I am talking about 20K rows per
second with 25 fields per row). So I was hoping to use MySQL this time
around. I guess I should have been more clear in my question, by
querying DB and reading flat files, I was meaning to ask about I/O
storage and retrieval mechanism, are there any optimizations in
FreeBSD regarding I/O?

By hinting at possible run time, I meant it would be running 20 hours
under heavy work load. So it could possibly get far more than 200
requests per second. At peak time it could get around 2500 per second.
So it would be important for OS to not crash or show flaky behavior
with memory management with so much memory to maintain throughout the
day.

Sorry if I seem to be neat-picking but I would like to pick the right
OS and stick to as the demand grows and I add more servers. I would
like them all to be running same OS as hardware is likely to change
more frequently.


Thanks.


On Wed, Jul 16, 2008 at 2:05 AM, Alexander Leidinger
<Alexander at leidinger.net> wrote:
> Quoting Rahul <rahulone at gmail.com> (from Tue, 15 Jul 2008 20:17:46 -0400):
>
>> For example: I would like to run a high performance web server that
>> can handle up to 200 connections per second and serve them with great
>> speed. That's where the concern for multi-threaded support. Depending
>
> Serving 200 connections per second for static data is not hard. And you
> don't need multi-threading for that. Multiple processes is actually better
> than multi-threading in this regard, as you don't have to do locking of
> filedescriptors in the the webserver over multiple threads. The difference
> between linux and FreeBSD should be not big for single CPU/core systems, but
> if you increase the number of CPUs/cores this may be different. See the
> postgresql (it uses processes, not threads like mysql) graphs in
> http://people.freebsd.org/~kris/scaling/7.0%20and%20beyond.pdf to get an
> idea about the scaling of more or less independent processes in FreeBSD.
>
>> on the request, I may have to load (and possibly unload) dynamic
>> modules to perform calculations, and if need be, fetch data from
>> either DB or flat file. It could involve connecting to a process on
>
> As soon as you have calculations and/or DB accesses involved, it mostly
> depends upon the DB optimizations ("good" tables, indexes, data volume,
> queries, good concurrency of the DB, ...) and the computation, not on the
> OS. So without any specific workload, we can not really give recommendations
> (besides giving FreeBSD a try and working with us if there's problem).
>
>> another box to get request specific command strings. This process
>> could run for almost 20 hours straight and OS still has to be able to
>> keep in shape.
>
> The OS doesn't care about how long a process runs. But if you talk about
> good responsiveness of the OS while a process uses a lot of memory and CPU,
> FreeBSD will handle it good (and from what I heard and seen better than
> Linux, but I don't have numbers at hand).
>
> Bye,
> Alexander.
>
> --
>  Leela: Well, someone's in a good mode.
>
> http://www.Leidinger.net    Alexander @ Leidinger.net: PGP ID = B0063FE7
> http://www.FreeBSD.org       netchild @ FreeBSD.org  : PGP ID = 72077137
>