5.3-STABLE frozen on heavy network load

[Lukas Ertl]

On Wed, 17 Nov 2004 10:22:14 +0000 (GMT), Robert Watson
<rwatson at freebsd.org> wrote:
> 
> On Wed, 17 Nov 2004, Lukas Ertl wrote:
> 
> > I'm seeing complete freezes on a 5.3-STABLE SMP (with HTT) kernel from
> > Fri Nov 12.  The machine is acting as a newsserver, thus it has heavy
> > network and disk load.
> 
> Do you know if the freeze happens with 5.3-RELEASE "as released"?

No, as I went directly from some 5-CURRENT to RELENG_5.

> If you set 'debug.mpsafenet=0', do the freezes keep happening?
> 
> What happens if you run with INVARIANTS on?

I'll check that.

> Is the system too slow with WITNESS to run your workload?

Unfortunately, yes.
 
> Could you send dmesg output?

Can be found at <http://people.freebsd.org/~le/newscore.dmesg>.

> Do you have an estimate of how long it takes to go from boot to hang?

Somewhere between one, two days and one, two weeks.

> If/when this recurs, could I get you to run the following commands in DDB,
> and send output:
> 
> - ps
> - show lockedvnods
> - show pcpu
> - show pcpu X, for each valid value of X (0 ... maxcpus-1)
> - do trace on each thread active on a CPU
> - do trace on any network device driver ithread, on the netisr, and any
>   other thread that appears to be involved in network activity

OK, will do.
 
> Using the current core, could you go to frame #29, and print *td,
> *td->td_proc, *uio, *active_cred, and *fp.  Go to frame #28 and print *so.
> If possible, please keep this dump around, I may also ask you to inspect
> *so_pcb once we know what to cast it to (given that it's a news server,
> could well be TCP, in which cast *(struct inpcb *)so->so_pcb, as well as
> the tcpcb reached through that).

Can be found at <http://people.freebsd.org/~le/debug2.log>.

> Oh, one more thing that would be useful: if you compile with
> BREAK_TO_DEBUGGER, are you able to get into the debugger using a console
> break or a serial break?  If so, which?  I assume that because you're
> using MP_WATCHDOG, you can't, but it's worth asking.  Right now, syscons
> requires Giant, so if you can get into the debugger via the serial link
> but not syscons, it will suggest something is spinning with Giant.

Unfortunately, I don't have a serial link available. MP_WATCHDOG was
my last resort to get at some info.

Hope that helps,
thanks,
le