UFS Filesystem issues, and the loss of my hair...
Kenneth D. Merry
ken at kdm.org
Thu Aug 13 15:23:10 UTC 2009
On Wed, Aug 12, 2009 at 15:47:21 +0300, Panagiotis Christias wrote:
> On Mon, Aug 10, 2009 at 05:20:44PM -0500, Hearn, Trevor wrote:
> > Yes, it does seem like it was part of one of the other messages. The isp(4)
> > driver was just recently updated in HEAD by mjacob@ who has maintained that
> > driver in the past. He may have some insight if there is an isp(4)-specific
> > problem.
> > --
> > John Baldwin
> > Heh. Ok, I just watched the same error message scroll across the screen
> > for about 5 minutes now, with a different offset, same length. The fun
> > part is that it is not touching the device, /dev/da1p7 at all. From the
> > systat -vmstat display, I see all of the traffic coming from the
> > /dev/mfid0 drives. It ran for a while, then stopped. So, no access to
> > the drive in question, da1p7, but on the root drive, mfid0. Odd. The
> > partition is mapped to the root drive. I wonder if the driver lost
> > itself, and it tried to access the file on the empty folder on the root
> > drive. Sigh. Anyone?
> we faced a similar problem here (major greek university) about a year ago
> . Our setup consists of Dell 2950 servers, QLogic 2462 HBAs (PCI-E)
> and an EMC CLARiiON CX3-40. As soon as we tried to do a simple "tar zxf
> ports.tgz" on a SAN volume the system would freeze or/and panic (same error
> messages as yours). Oleg Sharoiko suggested that we could decrease the
> number of tag openings (tag queue depth). Decreasing it would make the
> system a bit more stable but did not eliminate the problem.
> Then, I contacted Matthew Jacob and tested his latest isp code  along
> with alternative solutions like zfs and gjournal. Matthew was kind enough
> to offer his support but eventually I ran out time and patience, so I moved
> a couple of servers to centos in order to put the storage into production.
> That was around December last year.
> About a month ago Kenneth Merry announced that a new version of isp was
> available  which corrected bugs and added new functionality. I thought
> it was worth trying so I set up FreeBSD 7-stable in two Dell boxes, added
> the isp patches, recompiled the kernel and started the stress tests. I
> also looked around for more info and hints regarding qlogic hbas. The
> Linux driver (ql2xxx) has a 32 max queue depth by default (see
> ql2xmaxqdepth) which is also the recommended value by EMC. There are also
> similar references for Solaris (see sd:sd_max_throttle). Some mention
> even smaller values depending the storage.
> Currently, I am running stress tests, using fsx, ffsb, postmark, iozone,
> bonnie++, blogbench and other home-made scripts (any other suggestion?) on
> two 7-stable-amd64 + isp_diffs.releng7.20090629 boxes. So far, at 32 maximum
> tag openings, everything looks good, I have not seen any panics and the
> following fsck run cleanly. I will keep running more tests for a week or two
> hoping that they will help draw a conclusion.
Thanks for the report! I'm glad to hear it is working for you.
The driver has gone into -current, and will be in 8.0-RELEASE. Hopefully
it'll get propagated back into RELENG_7 before too long.
ken at kdm.org
More information about the freebsd-fs