ZFS and mdconfig -t vnode - Unexpected behavior

[Dan Rue]

I have discovered an unexpected behavior when using ZFS against a vnode.
When running mdconfig -d against a vnode, there is a long delay before
the blocks are fully written to the backing store file. Consider the
following test script: 

#!/bin/sh
rm foo
dd if=/dev/zero of=foo bs=4096 count=0 seek=1955584
du -k foo
md=`mdconfig -a -t vnode -f foo`
dd if=/dev/random of=/dev/${md} bs=4096 count=100
mdconfig -d -u $md
while [ 1 ]; do
    fsync foo
    du -k foo
    blocks=`du -k foo | awk '{print $1; }'`
    if [ $blocks -gt 100 ]; then
        exit
    fi
    sleep 1
done


A good run looks like this (UFS): 
# time sh t.sh
0+0 records in
0+0 records out
0 bytes transferred in 0.000017 secs (0 bytes/sec)
48      foo
100+0 records in
100+0 records out
409600 bytes transferred in 0.179925 secs (2276505 bytes/sec)
464     foo

real    0m0.203s
user    0m0.001s
sys     0m0.018s


A bad run looks like this (ZFS):
# time sh t.sh
0+0 records in
0+0 records out
0 bytes transferred in 0.000015 secs (0 bytes/sec)
1       foo
100+0 records in
100+0 records out
409600 bytes transferred in 0.089111 secs (4596522 bytes/sec)
1       foo
1       foo
1       foo
1       foo
1       foo
1       foo
1       foo
1       foo
1       foo
1       foo
1       foo
1       foo
1       foo
1       foo
1       foo
1       foo
1       foo
1       foo
1       foo
1       foo
1       foo
1       foo
1       foo
1       foo
1       foo
1       foo
1       foo
515     foo

real    0m27.370s
user    0m0.009s


Under ZFS, it can take as long as 30 seconds before the block size as
reported by du -k has been updated. The fsync appears to be a noop.

Under UFS, du -k shows the correct block size every time, immediately
after mdconfig -d. This is the expected behavior.

This has been tested against ZFS on FreeBSD 8.1, 8.2, and 9 stable, in
several different environments.

Are there any ZFS tunables that could be related to this? What could be
the cause of this behavior? 

Thanks, 
drue