CAM Target Layer and dev/isp

Tue Jul 17 03:14:34 UTC 2012

Hi Ken,

So, first off, nice work on CTL!  I accidentally stumbled onto this little
gem yesterday and was shocked I hadn't heard of it before.  (Although it
seems I'm not alone, even Google only seems to know about your original
mailing list posts -- `+freebsd +ctladm` returns surprisingly little.)

(Somewhat related question before I get into my main issue: `ctladm`
should remove the need for share/examples/scsi_target, right?  The CAM
Target Layer stuff encompasses all the original functionality exposed in
scsi_target?)

I'm not sure how to expose LUNs via my isp devices.  Here's what I've got
so far:

[root at s24/ttypts/1(../misc/isp/bin)#] for i in {0..3}; do; ./isp_getrole
/dev/isp$i; done
/dev/isp0 chan 0: role target
/dev/isp1 chan 0: role target
/dev/isp2 chan 0: role target
/dev/isp3 chan 0: role target

[root at s24/ttypts/1(../misc/isp/bin)#] ctladm devlist -v
LUN Backend       Size (Blocks)   BS Serial Number    Device ID
  0 ramdisk             2097152  512 MYSERIAL   0     MYDEVID   0
      lun_type=0

[root at s24/ttypts/1(../misc/isp/bin)#] ctladm port -l
Port Online Type     Name         pp vp WWNN               WWPN

0    YES    IOCTL    CTL ioctl    0  0  0                  0

1    YES    INTERNAL ctl2cam      0  0  0x5000000357375b00
0x5000000357375b02
2    YES    INTERNAL CTL internal 0  0  0                  0

3    YES    FC       isp0         0  0  0x200000e08b146178
0x210000e08b146178
4    YES    FC       isp1         1  0  0x400000007f000009
0x400000007f000009
5    YES    FC       isp2         2  0  0x200000e08b102f5b
0x210000e08b102f5b
6    YES    FC       isp3         3  0  0x200000e08b302f5b
0x210100e08b302f5b

It's not obvious from the docs how to export CTL LUNs through my isp
devices.  I tried passing -D /dev/isp[n] to `ctladm create`, but that
returns: 'ctladm: cctl_create_lun: error issuing CTL_LUN_REQ ioctl:
Inappropriate ioctl for device'.

Sample output from one of the switches on my fabric (s24's /dev/isp1 is
plugged into port 8 of sf1):

sf1:admin> switchshow
switchName:	sf1
switchType:	17.2
switchState:	Online
switchMode:	Native
switchRole:	Principal
switchDomain:	1
switchId:	fffc01
switchWwn:	10:00:00:60:69:5a:1a:40
switchBeacon:	OFF
Zoning:      	ON (cfg_2012_07_16)
port  0: id N2 Online         F-Port 50:06:0b:00:00:13:18:72
port  1: id N2 No_Light
port  2: id N2 Online         E-Port 10:00:00:60:69:5a:09:c0 "sf2"
(downstream) 
port  3: id N2 No_Light
port  4: id N2 No_Light
port  5: id N2 Online         F-Port 21:01:00:e0:8b:ab:cf:be
port  6: id N2 Online         F-Port 21:01:00:e0:8b:a6:98:ca
port  7: id N2 Online         F-Port 21:01:00:e0:8b:30:2f:5b
port  8: id N1 Online         L-Port 1 public
port  9: id N2 No_Light
port 10: id N2 No_Light
port 11: id N2 Online         L-Port 8 public
port 12: id N2 Online         L-Port 8 public
port 13: id N2 Online         L-Port 8 public
port 14: id N2 Online         L-Port 8 public
port 15: id N2 No_Light
sf1:admin> portshow 8
portName:   
portHealth: No License
Authentication: None
portFlags:  0x223806b	portLbMod:  0x0	 PRESENT ACTIVE F_PORT L_PORT U_PORT
LOGIN NOELP LED ACCEPT WAS_EPORT
portType:   4.1
portState:  1	Online
portPhys:   6	In_Sync
portScn:    6	F_Port
portRegs:   0x81100000
portData:   0x102b8f40
portId:     010800
portWwn:    20:08:00:60:69:5a:1a:40
portWwn of device(s) connected: 	40:00:00:00:7f:00:00:09
Distance:   normal
Speed:      N1Gbps 

Interrupts:        10126      Link_failure: 0          Frjt:         0

Unknown:           51         Loss_of_sync: 10058      Fbsy:         0

Lli:               10084      Loss_of_sig:  2          Lip_in:       1

Proc_rqrd:         21         Protocol_err: 0          Lip_out:      2

Timed_out:         0          Invalid_word: 0          Lip_rx:       F7,F7
Rx_flushed:        0          Invalid_crc:  0
Tx_unavail:        0          Delim_err:    0
Free_buffer:       0          Address_err:  1
Overrun:           0          Lr_in:        0
Suspended:         0          Lr_out:       0
Parity_err:        0          Ols_in:       0
                              Ols_out:      0
sf1:admin> 

Ideally I'd like to be able to use CTL to export multiple ZFS zvols as
separate targets (i.e. all with unique WWNN/WWPNs), such that, from the
fabric's point of view, the port would look like just another FC-AL JBOD,
like, say, port 11:

sf1:admin> portshow 11
portName:   
portHealth: No License
Authentication: None
portFlags:  0x223806b	portLbMod:  0x0	 PRESENT ACTIVE F_PORT L_PORT U_PORT
LOGIN NOELP LED ACCEPT WAS_EPORT
portType:   4.1
portState:  1	Online
portPhys:   6	In_Sync
portScn:    6	F_Port
portRegs:   0x81130000
portData:   0x11dc6770
portId:     010b00
portWwn:    20:0b:00:60:69:5a:1a:40
portWwn of device(s) connected: 	21:00:00:14:c3:ca:23:ca
	21:00:00:14:c3:c1:9c:90
	21:00:00:14:c3:c4:41:9a
	21:00:00:14:c3:c1:23:6b
	21:00:00:14:c3:ca:23:df
	21:00:00:14:c3:c4:47:97
	21:00:00:14:c3:ca:20:0e
	21:00:00:14:c3:c4:40:ca

That'll allow me to zone the zvols just as if they were actual JBOD disks.
 Which will be awesome.

Is that possible?

Also, the HA stuff sounds bad-ass.  What's the best way to stay up to date
with CTL development?  Watch commits to sys/cam/ctl?  (There's no separate
list or anything for this stuff, right?)

Regards,

	Trent.