Dead console on FreeBSD 9.1

[Matthew Rezny]

I have now observed this on more than one machine so it's time to
report it rather than ignore it as a fluke.

Over a month ago, I saw this several times on FreeBSD/ppc64 9.1-RC. The
ppc64 port is not exactly solid, so with more pressing issues to deal
with I ignored it.

Now, I see the same on a box running FreeBSD/amd64 9.1-STABLE. Seeing
the same issue on multiple machines indicates it's a real problem.

What happens when the console goes dead is that there is no further
output and there is no response to the keyboard. There should be output
to the console from a running program. The program continues to run,
but the screen is not updated. I cannot switch virtual consoles using
the keyboard.

I can ssh in and continue to use the machine. Everything seems to run
fine and I can see further output in dmesg. I always considered the
console as the last resort, so to loose it when the box is still
running is very troublesome indeed.

The only correlation I can come up with is that both machine have disks
on a SAS card using the mpt driver and there may have been a device
disconnecting from the bus at the time of the console hang. It is
impossible to tell exactly when the console hung so I can't be sure
that the output from mpt in dmesg correlates to the moment of the hang.

On the ppc64 machine, I was using ddrescue on a troublesome disk which
periodically disconnects from the SAS bus when the firmware takes too
long attempting to correct errors. I just wanted to get through imaging
the disk, so I let it run and used ssh to check the status. The active
virtual console was running ddrescue, which continued to run without
interruption after the console hang. Any output from the driver would
have been on the first virtual console, which was not the active one at
the time, and without the ability to switch to it I can only look at the
end of dmesg and take a guess.

On the amd64 machine, I was doing a zfs send/receive from one pool to
another. Again, a troublesome disk is present which periodically
disconnects. The active virtual console was running "zpool iostat -v 1"
to monitor the status. The zfs send/recive pipe was in another virtual
console and I had top running in yet another. Again, the console hangs,
I cannot switch to the first virtual console to see what it might say,
but I can see mpt errors at the end of dmesg. I left it be to finish
the zfs send/receive operation while monitoring status via ssh.

The important work is done for the moment. I have not rebooted the
system so it has a dead console but I still have ssh access. Any
suggestions what to look at while it is in this state to attempt to
determine the cause?