Losing confidence in FreeBSD 6.x in a loaded environment ... :(

[Marc G. Fournier]

Okay, now this is getting ridiculous .. and for all those that have been 
helping me debug things over the past few days, this isn't a rant against 
"people", its a rant against the state of FreeBSD 6.x :(

Its got SERIOUS problems.

As easy as it is to 'blame the hardware', I'm up to my third FreeBSD 6.x 
system that is having problems now, all of which ran flawlessly under 
FreeBSD 4-STABLE under some serious load ...

And by 'serious load' .. jupiter (the one that was giving me the SegFaults 
under those two kernels I tried) was up to 100 vServers running on it 
while I was clearning things off of pluto to upgrade her to 6.x ... and 
something like 209 days uptime ...

Right now, I have 3 FreeBSD 4.x left in production, and 4 FreeBSD 6.x ...

One 4.x server is running 87 vServers, has been up for 74 days now, and 
vmstat 5 shows:

# vmstat 5
  procs      memory      page                    disks     faults      cpu
  r b w     avm    fre  flt  re  pi  po  fr  sr da0 da1   in   sy  cs us sy id
  3 5 0 3594432 228088   60   3   3   1  73 310   0   0  584  562 569 28 29 43
  3 5 0 3578556 227384  546   0   1   7 592   0   0  18  434 1869 1647  7 11 83
  2 5 0 3564464 225092  807   0   0   0 505   0   2   4  382 2698 2684  8  9 83

One of my 6.x just went down for the second time today ... locked up solid 
... first time it did it today, it was hitting maxpipekva, which seems to 
be my biggest headache so far with 6.x ...

Pluto, the one that we've been pretty much fighting over this past 
weekend, is running 69 vServers, 1363 processes, a loadavg <1 ... and I'm 
lucky to keep it running for 24 hours ...

maxpipekva is set to:

            kern.ipc.maxpipekva: 67108864 - kern.ipc.pipekva: 35782656

It just doesn't feel like 6.x is as robust under loaded conditions as 
4-STABLE was ... :(

I don't expect anything to get fixed based on this email ... it was more 
to get off my chest about those that have been suggesting that I'm 
overloading the server(s) and causing the problems ... I loaded them worse 
under 4.x, with less problems ... hell, I got better uptimes under load 
when I was using unionfs then 6.x right now is giving me *without* any 
"funny mounts" :(


----
Marc G. Fournier           Hub.Org Networking Services (http://www.hub.org)
Email . scrappy at hub.org                              MSN . scrappy at hub.org
Yahoo . yscrappy               Skype: hub.org        ICQ . 7615664