Hastd activemap cache flush and "request failed: FLUSH.: Operation not supported by device" messages.

[Mikolaj Golub]

On Tue, 29 May 2012 15:52:33 +0300 Juris Krumins wrote:

 JK> Hi everybody.
 JK> I've recently setup two FreeBSD nodes to use them as a storage.
 JK> So I'm using hast over "MegaRAID SAS 1078" and ZFS filesystem created on
 JK> HAST resource devices from /dev/hast/*.
 JK> Everything looks good, but when I execute iozone tests, I'm constantly
 JK> getting following messages in /var/log/message:

 JK> primary node:
 JK> May 29 15:44:15 rook hastd[2313]: [mfid2] (primary) Local request failed
 JK> (Operation not supported by device): FLUSH.
 JK> May 29 15:44:15 rook hastd[2311]: [mfid0] (primary) Local request failed
 JK> (Operation not supported by device): FLUSH.
 JK> May 29 15:44:15 rook hastd[2312]: [mfid1] (primary) Local request failed
 JK> (Operation not supported by device): FLUSH.
 JK> May 29 15:44:15 rook hastd[2312]: [mfid1] (primary) Remote request
 JK> failed (Operation not supported by device): FLUSH.
 JK> May 29 15:44:15 rook hastd[2313]: [mfid2] (primary) Remote request
 JK> failed (Operation not supported by device): FLUSH.
 JK> May 29 15:44:15 rook hastd[2311]: [mfid0] (primary) Remote request
 JK> failed (Operation not supported by device): FLUSH.

 JK> secondary node:
 JK> May 29 15:43:41 rook2 hastd[2458]: [mfid1] (secondary) Request failed:
 JK> FLUSH.: Operation not supported by device.
 JK> May 29 15:43:41 rook2 hastd[2457]: [mfid0] (secondary) Request failed:
 JK> FLUSH.: Operation not supported by device.
 JK> May 29 15:43:41 rook2 hastd[2462]: [mfid2] (secondary) Request failed:
 JK> FLUSH.: Operation not supported by device.

 JK> I'v also found following topic
 JK> http://lists.freebsd.org/pipermail/freebsd-fs/2012-January/013385.html.
 JK> I've updated src, rebuild world and kernel and rebooted. I've also added
 JK> metaflush "off" options to /etc/hast.conf configuration file. But still
 JK> getting previously mentioned messages.

Note, there are two sources of BIO_FLUSH operations in HAST:

1) BIO_FLUSH operations generated by HAST on metadata (activemap) update, if
enabled (not disabled) in config.

2) BIO_FLUSH operations sent by a HAST user (filesystem).

If the underlying provider doesn't support BIO_FLUSH, in the first case a warning
"The provider doesn't support flushing write cache. Disabling it." will be
generated and there will be no more attempt to flush on activemap update.

In the second case the error 'Request failed: FLUSH.: Operation not supported
by device' will be observed (as in your case). Before r225832 it logged on
every BIO_FLUSH coming. Since r225832 HAST will try only once and only one
error in logs should be observed after hastd start.

 JK> Can somebody point me to the right direction how to configure hastd to
 JK> throw this warning only once.

 JK> I'm using:
 JK> FreeBSD primary. 9.0-STABLE FreeBSD 9.0-STABLE #0: Tue May 29 12:53:44
 JK> EEST 2012     root at primary.:/usr/obj/usr/src/sys/GENERIC  amd64

 JK> FreeBSD secondary. 9.0-STABLE FreeBSD 9.0-STABLE #0: Tue May 29 12:53:44
 JK> EEST 2012     root at secondary.:/usr/obj/usr/src/sys/GENERIC  amd64

r225832 has been merged to 9-STABLE on 4 Jan 2012 as r229509 and you shouldn't
observe 'FLUSH: Operation not supported by device' constantly repeating. If
you do this is very strange and you should recheck your build. Please provide
the output of

  ident /sbin/hastd

command.

 JK> and hast.conf configuration from both machines:
 JK> I've decided to put metaflush option on every single level just to be
 JK> sure. No effect.

No need in disabling metaflush in your case. It will disable automatically
after the first attempt failed.

-- 
Mikolaj Golub