CAM Target Layer, Linux and camcontrol readcap

[Nikolay Denev]

Hi,

I'm running RELENG_9 and I'm trying to play with CTL.
Initially I've setup an isp(4) interface in TARGET mode and tried to export a LUN to a directly connected
Linux RHEL host, but for some reason that failed with the block backend (ramdisk was exported properly) :

This is how I export the volume :

zfs create -V1000G tank/oracle
ctladm create -b block -o file=/dev/zvol/tank/oracle -S ZFSSERIAL001 -d ZFSLUN001
ctladm port -o on
ctladm realsync off 

camcontrol and ctladm show the device correctly :

[10:25]root at goliath:/home/ndenev# ctladm port -l    
Port Online Type     Name         pp vp WWNN               WWPN              
0    YES    IOCTL    CTL ioctl    0  0  0                  0                 
1    YES    INTERNAL ctl2cam      0  0  0x500000091a1f4700 0x500000091a1f4702
2    YES    INTERNAL CTL internal 0  0  0                  0                 
3    YES    FC       isp0         0  0  0x20000024ff376b98 0x21000024ff376b98
4    YES    FC       isp1         1  0  0x20000024ff376b99 0x21000024ff376b99

[10:26]root at goliath:/home/ndenev# ctladm devlist -v
LUN Backend       Size (Blocks)   BS Serial Number    Device ID       
  0 block            2097152000  512 ZFSSERIAL001     ZFSLUN001       
      lun_type=0
      num_threads=14
      file=/dev/zvol/tank/oracle
[10:26]root at goliath:/home/ndenev# camcontrol devlist
<FREEBSD CTLDISK 0001>             at scbus2 target 1 lun 0 (da0,pass0)


This is what I see on the Linux host for the exported zvol:

qla2xxx 0000:0a:00.1: LOOP UP detected (8 Gbps).
qla2xxx 0000:0a:00.1: qla2xxx_eh_host_reset: reset succeeded
qla2xxx 0000:0a:00.1: scsi(4:0:0): Abort command issued -- 1 c 2002.
sd 4:0:0:0: scsi: Device offlined - not ready after error recovery
sd 4:0:0:0: rejecting I/O to offline device
sd 4:0:0:0: rejecting I/O to offline device
sd 4:0:0:0: rejecting I/O to offline device
sdq : READ CAPACITY failed.
sdq : status=0, message=00, host=1, driver=00 
sdq : sense not available. 
sd 4:0:0:0: rejecting I/O to offline device
sdq: Write Protect is off
sdq: Mode Sense: 00 00 00 00
sd 4:0:0:0: rejecting I/O to offline device
sdq: asking for cache data failed
sdq: assuming drive cache: write through
sd 4:0:0:0: Attached scsi disk sdq
sd 4:0:0:0: Attached scsi generic sg18 type 0
sd 4:0:0:0: rejecting I/O to offline device
sd 4:0:0:0: rejecting I/O to offline device
sd 4:0:0:0: rejecting I/O to offline device
sd 4:0:0:0: rejecting I/O to offline device
sd 4:0:0:0: rejecting I/O to offline device
sd 4:0:0:0: rejecting I/O to offline device

I've noticed the READ CAPACITY failed message and tried to issue a several "camcontrol readcap" commands,
but they got stuck and are now unkillable :

[10:21]root at goliath:/home/ndenev# ps axuw|grep cam
root    29739    0.0  0.0  16300 1628  0- DL+   8:09AM     0:00.00 camcontrol readcap da0
root    30033    0.0  0.0  16300 1628  1- DL+   8:12AM     0:00.00 camcontrol readcap 2:1:0
root    30135    0.0  0.0  16300 1632  2  D+    8:21AM     0:00.00 camcontrol start pass0

procstat shows the same kernel stack for them :

[10:23]root at goliath.:/home/ndenev# procstat -kk 30033
  PID    TID COMM             TDNAME           KSTACK                       
30033 101161 camcontrol       -                mi_switch+0x186 sleepq_wait+0x42 _sleep+0x390 cam_periph_runccb+0x5a passioctl+0x171 devfs_ioctl_f+0x7b kern_ioctl+0x115 sys_ioctl+0xfd amd64_syscall+0x546 Xfast_syscall+0xf7 

Also there is this suspicious message on the FreeBSD machine :

cfcs_action: unsupported CCB type 0x918

Then I've tried to remove the volume from CTL but the command also got stuck :

[10:29]root at goliath:/home/ndenev# ps axuww|grep ctladm
root    54269    0.0  0.0  18484 1692  3  I     8:24AM     0:00.00 ctladm remove -b block -l 0
root    20969    0.0  0.0  16280 1720  5  S+   10:30AM     0:00.00 grep ctladm

[10:30]root at goliath:/home/ndenev# procstat -kk 54269
  PID    TID COMM             TDNAME           KSTACK                       
54269 101580 ctladm           -                mi_switch+0x186 sleepq_catch_signals+0x2cc sleepq_wait_sig+0xc _sleep+0x2b9 ctl_be_block_ioctl+0x9aa ctl_ioctl+0x9e6 devfs_ioctl_f+0x7b kern_ioctl+0x115 sys_ioctl+0xfd amd64_syscall+0x546 Xfast_syscall+0xf7 

Some Linux mailing lists suggest that maybe a firmware upgrade on the Linux box can help, but
this does not explain the stuck readcap and ctladm commands.

Any ideas on how to debug this further are welcome.