9-STABLE panic on intensive fork

[Dmitry Sivachenko]

On 29.08.2013, at 22:45, Konstantin Belousov <kostikbel at gmail.com> wrote:

> On Wed, Aug 28, 2013 at 06:20:29PM +0400, Dmitry Sivachenko wrote:
>> Hello!
>> 
>> I am using very recent FreeBSD-9-STABLE snapshot:
>> 9.2-PRERELEASE FreeBSD 9.2-PRERELEASE #0 r254986: Wed Aug 28 17:18:57 MSK 2013
>> 
>> I run uwsgi program (ports/www/uwsgi) on that machine.
>> 
>> When uwsgi starts, it forks pre-configured number of worker processes.
>> If I raise workers parameter high enough (128), I get kernel panic (100% reproducible):
>> 
>> Fatal trap 12: page fault while in kernel mode
>> 
>> If I compile kernel with KDB enabled, I get the following stack:
>> 
>> pmap_demote_pde_locked()
>> pmap_copy()
>> vmspace_fork()
>> fork1()
>> sys_fork()
>> 
>> I have only remote console for that machine, so I made 2 screenshots:
>> 
>> 1) http://people.freebsd.org/~demon/screen1.jpg
>> Panic screen when kernel has no KDB support compiled in
>> 
>> 2) http://people.freebsd.org/~demon/screen2.jpg
>> Panic screen (2nd part) with the above stack shown.
> Look up the source line for the pmap_demote_pde_locked()+0x471 for your
> kernel.  Dump the core from the panic.


Kernel dump is not generated (despite it is configured at boot), there is no "Dumping...." message on console.
These screenshots shows everything I see on console.

I performed some more investigations on this:
I have several (14) totally identical configured machines running exactly the same software.
Hardware is a bit different though.  I tried to analyze motherboard differences but failed to find common things for the affected machines.

Under conditions described in my initial e-mail, some of them crash (exactly the same way), some of them do not.
I am confident there is no hardware problems, these machines run for months without reboot, as for now I discovered the only way to crash them.

I updated one of the affected servers to 10-current and I can state it does not crash anymore with the same usage scenario.