bin/121684: : dump(8) frequently hangs
rwatson at FreeBSD.org
Tue Sep 2 07:55:33 UTC 2008
On Mon, 1 Sep 2008, Jeremy Chadwick wrote:
> On Tue, Sep 02, 2008 at 09:39:20AM +0300, Danny Braniss wrote:
>> take a look at:
>> http://www.freebsd.org/cgi/query-pr.cgi?pr=117603 danny
> That PR may or may not be relevant, depending upon what FreeBSD version
> users are using, and what kernel build date.
> The bug mentioned in that PR got addressed in HEAD on March 13th, 2008, and
> the fix MFC'd to RELENG_7 on April 19th, 2008. It was never MFC'd to
> If there are users on RELENG_7 with kernels built with sources after April
> 19th 2008 who are experiencing the problem, then the PR is probably not
Part of the "problem" here is that a whole class of possible bugs have
near-identical symptoms. While each of the following means quite different
things to a kernel developer working on the problem, and may reflect quite
different types of bugs, they are all often described as "dump hangs" or "dump
- the system deadlocks
- dump fails to complete and/or exit, but cannot be killed
- dump fails to complete and/or exit, but can be killed
When snapshots were first introduced, the problems tended to be at the top end
of that list, corresponding to VFS locking and resource starvation deadlocks.
As the snapshot code has matured, new problems in both the kernel scheduler
and dump code have arisen as parallelism has increased, reflecting a
combination of old bugs in the dump code and new bugs in the kernel scheduler.
Unfortunately, these bugs don't tend to get discovered much during testing in
-CURRENT -- perhaps people don't back up their -CURRENT boxes much :-).
I think we need to rigorously do the following:
- For each bug report, determine whether it is reporting one or more of the
above types of "hangs". If multiple types are reported, track them with
different bug reports.
- Establish as early as possible whether a fix resolves the problems in each
report. Because we're dealing with many bugs over time, it's possible to
end up with accidentally "omnibus" reports that remain open and are never
closed, even though committed and released fixes may correct the problems
experienced by the reporter. It is almost impossible, btw, to rewind and
years later determine if any particular fix would have corrected any
particular report, because the original submitter will have moved on.
Dump happens to be particularly sensitive to bugs of these sorts because it
uses snapshots and it uses multiple workers that signal each other, so it's a
good lithmus test of stability of both of those features. However, it's easy
to conclude that dump is much less stable than it proves in practice because
we have a lot of stale and confusing bug reports. What we do need is a dump
bug report owner, who can keep track of the outstanding set, try to
agressively close the ones that are fixed, which will among other things allow
us to better track regressions vs bugs from inception.
Robert N M Watson
University of Cambridge
More information about the freebsd-stable