hast, zfs, unable to flush disk cache

[Pawel Jakub Dawidek]

On Fri, Mar 01, 2013 at 10:25:50PM +0200, Mikolaj Golub wrote:
> On Fri, Mar 01, 2013 at 11:39:23AM -0600, Chad M Stewart wrote:
> 
> > I've setup a 2 node cluster using HAST.  I'd previously had this
> > running 9.0 and then rebuilt the nodes using 9.1.  Under 9.0 I used
> > vfs.zfs.vdev.bio_flush_disable=1 but that setting does not appear to
> > work in 9.1.  The other difference, previous build had root disk on
> > UFS, while this build has only ZFS based file systems.
> > 
> > FreeBSD node1.san 9.1-RELEASE FreeBSD 9.1-RELEASE #0 r243826: Tue Dec  4 06:55:39 UTC 2012     root at obrian.cse.buffalo.edu:/usr/obj/usr/src/sys/GENERIC  i386
> > 
> > 
> > Mar  1 17:07:25 node1 hastd[1446]: [disk5] (primary) Unable to flush disk cache on activemap update: Operation not supported by device.
> > Mar  1 17:07:28 node1 hastd[1440]: [disk3] (primary) Unable to flush disk cache on activemap update: Operation not supported by device.
> > Mar  1 17:07:28 node1 hastd[1437]: [disk2] (primary) Unable to flush disk cache on activemap update: Operation not supported by device.
> > Mar  1 17:07:28 node1 hastd[1434]: [disk1] (primary) Unable to flush disk cache on activemap update: Operation not supported by device.
> > Mar  1 17:07:28 node1 hastd[1446]: [disk5] (primary) Unable to flush disk cache on activemap update: Operation not supported by device.
> > Mar  1 17:07:28 node1 hastd[1443]: [disk4] (primary) Unable to flush disk cache on activemap update: Operation not supported by device.
> > 
> > I tried setting zfs set zfs:zfs_nocacheflush=1 <pool> but that
> > diddn't appear to work either.  I get a lot of lines in my log file
> > because of this.  I have also tried zfs set sync=disabled <pool> but
> > HAST still outputs those lines in the log file.
> 
> These flushes were generated by HAST itself when it tried to flush
> activemap updates, that is why disabling BIO_FLUSH for ZFS dis not
> help much.
> 
> Setting "metaflush off" in hast.conf should help though.
> 
> BTW, hastd tries to detect devices that do not support BIO_FLUSH,
> checking for the returned errno, and automatically disable flushes if
> the errno is EOPNOTSUPP (Operation not supported). Unfortunately, your
> device returned ENODEV (Operation not supported by device).
> 
> What device do you have?
> 
> Pawel, do you think it would be a good idea to automatically disable
> activemap flush for ENODEV case too?

It would be better to find the driver that returns ENODEV and fix it, IMHO.

-- 
Pawel Jakub Dawidek                       http://www.wheelsystems.com
FreeBSD committer                         http://www.FreeBSD.org
Am I Evil? Yes, I Am!                     http://tupytaj.pl
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 196 bytes
Desc: not available
URL: <http://lists.freebsd.org/pipermail/freebsd-questions/attachments/20130301/81b8e69f/attachment.sig>