RELENG_8 pf stack issue (state count spiraling out of control)

[Daniel Hartmeier]

On Mon, May 02, 2011 at 06:58:54PM -0700, Jeremy Chadwick wrote:

> The next thing I tried was "/etc/rc.d/pf stop", which worked.  Then I
> did "/etc/rc.d/pf start", which also worked.  However, what I saw next
> surely indicated a bug in the pf layer somewhere -- "pfctl -s states"
> and "pfctl -s info" disagreed on the state count:

This can be explained. Note that "/etc/rc.d/pf start" does first flush
all states by calling pfctl -F all.

This calls pf_unlink_state() for every state in the kernel, which
marks each state with PFTM_UNLINKED, but doesn't free it yet.

Such states do not show up in pfctl -s state output, but are still
counted in pfctl -s info output. Normally, they are freed the next
time the pfpurge thread runs (once per second).

It looks like the pfpurge thread was either

  a) sleeping indefinitely, not returning once a second from

	tsleep(pf_purge_thread, PWAIT, "pftm", 1 * hz);

     or

  b) constantly failing to acquire a lock with

	if (!sx_try_upgrade(&pf_consistency_lock))
		return (0);

Maybe a) is possible when CLOCK_MONOTONIC is decreasing? And the
"POKED TIMER" messages you get from BIND, too?

Kind regards,
Daniel