Consistent file system hang with RELENG_6 of today ...

[Robert Watson]

On Fri, 29 Jul 2005, Marc G. Fournier wrote:

> Oh, before I forget ... how do you break to DDB from a serial console? 
> I'll be running a Portmaster for this ...

Compile BREAK_TO_DEBUGGER into the kernel.  Then issue a serial break and 
it should go into the debugger if the low-level console is set to the 
serial port.

On my port-master, I issue a telnet break using "send break" after hitting 
^] to generate a serial break for a serial port hooked up to TCP.

There's also ALT_BREAK_TO_DEBUGGER, which is documented in NOTES, and 
provides for an alternative key sequence that doesn't use a break signal.

Robert N M Watson

>
> On Fri, 29 Jul 2005, Robert Watson wrote:
>
>> 
>> On Thu, 28 Jul 2005, Marc G. Fournier wrote:
>> 
>>> 'k, I'm starting to play with 6.x, for our new server ... my priority 
>>> right now is to just have it run the existing 'jail' environments from my 
>>> 4.x machine, while I work on getting all of our servers up to 6.x, and 
>>> then will worry about the jail's themselves ...
>>> 
>>> When I try and startup the 4.x jail on my 6.x machine, it "hangs" the file 
>>> system that the jail directory hierarchy happens to be mounted on though 
>>> ... twice in a row so far ...
>>> 
>>> Now, I'm suspecting (and am going to try without it) that it might be 
>>> because I'm mounting devfs within the 4.x jail, but even then, it 
>>> shouldn't hang things up, only generate a whack of errors ...
>>> 
>>> I have a good dump (CTL-ALT-ESC -> panic), but do not have a clue what to 
>>> offer from within there that might be of any use ...
>>> 
>>> If anyone is interested ... ?
>> 
>> If you can get into DDB and have serial console output, the following would 
>> be useful:
>> 
>> The output of 'show pcpu'
>> 
>> The output of 'show pcpu X' for each present cpu, starting with 0.
>> 
>> The output of 'ps'
>> 
>> The output of 'trace' for the currently running thread, and each non-idle 
>> thread shown in the show pcpu output
>> 
>> The output of 'show lockedvnods'
>> 
>> It would also be useful if, relating to the startup of the jail, you can 
>> identify the point in the jail boot where it wedges, and if you hit Ctrl-T, 
>> what process is shown as running and what state it is in, and using DDB, 
>> trace that process.
>> 
>> If you could show the trace output for each process listed in "show 
>> lockevnods".
>> 
>> Likely, there is a leaked lock or a low buffer condition.  However, once we 
>> have the above output we should be able to say more.  The above will 
>> hopefully tell us whether it's a vnode deadlock, and ideally, the 
>> approximate source.
>> 
>> Robert N M Watson
>> 
>> 
>
> ----
> Marc G. Fournier           Hub.Org Networking Services (http://www.hub.org)
> Email: scrappy at hub.org           Yahoo!: yscrappy              ICQ: 7615664
>