gmirror + ZFS issues

[Nicolas Haller]

Hello,

I've got a new server and I installed FreeBSD 10 on it. I have a problem 
to create a new zfs pool. The command stalls on IO wait (D state / 
zio->io_cv).

The device for the pool is a 1.7T partition (index 4) of a gmirror device.

root at daedalus:~ # uname -a
FreeBSD daedalus 10.0-RELEASE-p3 FreeBSD 10.0-RELEASE-p3 #0: Tue May 13 
18:31:10 UTC 2014 
root at amd64-builder.daemonology.net:/usr/obj/usr/src/sys/GENERIC  amd64


root at daedalus:~ # dmesg | grep ada
ada0 at ahcich0 bus 0 scbus0 target 0 lun 0
ada0: <HGST HUS724020ALA640 MF6OAA70> ATA-8 SATA 3.x device
ada0: Serial Number PN2134P6HE028P
ada0: 600.000MB/s transfers (SATA 3.x, UDMA6, PIO 8192bytes)
ada0: Command Queueing enabled
ada0: 1907729MB (3907029168 512 byte sectors: 16H 63S/T 16383C)
ada0: Previously was known as ad4
ada1 at ahcich1 bus 0 scbus1 target 0 lun 0
ada1: <HGST HUS724020ALA640 MF6OAA70> ATA-8 SATA 3.x device
ada1: Serial Number PN2134P6HAZTVP
ada1: 600.000MB/s transfers (SATA 3.x, UDMA6, PIO 8192bytes)
ada1: Command Queueing enabled
ada1: 1907729MB (3907029168 512 byte sectors: 16H 63S/T 16383C)
ada1: Previously was known as ad6


root at daedalus:~ # gmirror status
       Name    Status  Components
mirror/gm0  COMPLETE  ada0 (ACTIVE)
                       ada1 (ACTIVE)


root at daedalus:~ # gpart show
=>        63  3907029104  mirror/gm0  MBR  (1.8T)
           63  3907029042           1  freebsd  [active]  (1.8T)
   3907029105          62              - free -  (31K)

=>         0  3907029042  mirror/gm0s1  BSD  (1.8T)
            0   204800000             1  freebsd-ufs  (98G)
    204800000    20971520             2  freebsd-swap  (10G)
    225771520  3681257522             4  freebsd-ufs  (1.7T)

So the blocked command is the following:
root at daedalus:~ # zpool create tank /dev/mirror/gm0s1d

root at daedalus:~ # ps -O comm,mwchan
   PID COMMAND MWCHAN   TT  STAT    TIME COMMAND
  1104 getty   ttyin    v0  Is+  0:00.00 /usr/libexec/getty Pc ttyv0
  1105 getty   ttyin    v1  Is+  0:00.00 /usr/libexec/getty Pc ttyv1
  1106 getty   ttyin    v2  Is+  0:00.00 /usr/libexec/getty Pc ttyv2
  1107 getty   ttyin    v3  Is+  0:00.00 /usr/libexec/getty Pc ttyv3
  1108 getty   ttyin    v4  Is+  0:00.00 /usr/libexec/getty Pc ttyv4
  1109 getty   ttyin    v5  Is+  0:00.00 /usr/libexec/getty Pc ttyv5
  1110 getty   ttyin    v6  Is+  0:00.00 /usr/libexec/getty Pc ttyv6
  1111 getty   ttyin    v7  Is+  0:00.00 /usr/libexec/getty Pc ttyv7
85215 csh     pause     2  Ss   0:00.12 /bin/csh
86210 ps      -         2  R+   0:00.00 ps -O comm,mwchan
85081 csh     pause     0  Is   0:00.01 -csh (csh)
85084 screen  pause     0  S+   0:00.11 screen
85086 csh     pause     1  Is   0:00.12 /bin/csh
85159 zpool   zio->io_  1  D+   0:00.00 zpool create tank /dev/mirror/gm0s1d

root at daedalus:/proc/85159 # cat status
zpool 85159 1 85159 85086 pts/1 ctty 1400242849,897702 0,0 0,1876 
zio->io_cv 0 0 0,0,5 -


There is no activity on disks:
root at daedalus:~ # iostat 1
        tty            ada0             ada1            pass0 
   cpu
  tin  tout  KB/t tps  MB/s   KB/t tps  MB/s   KB/t tps  MB/s  us ni sy 
in id
    0    41 124.91 457 55.80  125.92 454 55.80   0.00   0  0.00   0  0 
0  0 99
    0   465  0.00   0  0.00   0.00   0  0.00   0.00   0  0.00   0  0  0 
  0 100
    0   155  0.00   0  0.00   0.00   0  0.00   0.00   0  0.00   0  0  0 
  0 100
    0   151  0.00   0  0.00   0.00   0  0.00   0.00   0  0.00   0  0  0 
  0 100

I have also a FreeBSD 9.0-RELEASE-p3 setup the same way and it works 
like a charm. (but not the same hardware on this one)

Actually, I'm trying to zeroing the partition, we never know :-)

Anyone has an idea ?

Thanks,

-- 
Nicolas Haller