Driver development question

Wed Jul 29 17:23:48 UTC 2009

Warning LONG message below, I figured I'd give as much information as
possible to try and help me along.

> On Tuesday 21 July 2009 8:07:13 pm Alfred Perlstein wrote:
> > * John Baldwin <jhb at freebsd.org> [090721 14:44] wrote:
> > > On Tuesday 21 July 2009 2:34:21 am Marc Loerner wrote:
> > > > Am Dienstag 21 Juli 2009 00:38:56 schrieb Sam Leffler:
> > > > > John Baldwin wrote:
> > > > > > On Friday 17 July 2009 11:10:17 am Chris Harrer wrote:
> > > > > >> Hi All,
> > > > > >>
> > > > > >> I'm hoping someone can point me in the right direction...  I'm
> > > > > >> developing a FreeBSD driver for a PCIe card.  The driver
controls a
> > > > > >> hardware device that has DRAM and various state information on
it.  
> I'm
> > > > > >> trying to mimic functionality I have for other OS support such
that 
> I
> > > > > >> can dump memory and state information from the card to a file I

> create
> > > > > >> from within my driver (kernel module).
> > > > > >>
> > > > > >> For example, in a Linux driver I use filp_open to create the
dump 
> file
> > > > > >> (represented by fp), then use fp->f_op->write to put
information 
> into
> > > > > >> the file.
> > > > > >>
> > > > > >> FreeBSD doesn't have filp_* API's.  I've tried searching for 
> example
> > > > > >> drivers and googling for file API's from kernel modules to no 
> avail. 
> > > > > >> Can someone please offer some guidance as to how I might
proceed 
> here?
> > > > > >>
> > > > > >> Thanks in advance and any insight would be most appreciated!
> > > > > >
> > > > > > You can look at sys/kern/kern_ktrace.c to see how the ktrace() 
> system
> > > > > > call creates a file.  I think in general you will wind up using
> > > > > > NDINIT/namei() (to lookup the vnode for a pathname) and then 
> vn_open() /
> > > > > > vn_rdwr() / vn_close().
> > > > >
> > > > > man alq(9).
> > > > >
> > > > > 	
> > > > 
> > > > Why not use kern_open, kern_close, kern_preadv, kern_pwritev?
> > > 
> > > Those affect the state of the current process by opening a new file 
> > > descriptor, etc.  That is generally bad practice for a device driver
to be 
> > > interfering with a process' state, and it will not work for kernel 
> threads.  
> > > You can rather easily have userland open a file and then pass the file

> > > descriptor to a driver which can then do operations on a file
directly.
> > 
> > If the vnode operations are annoying to wrap ones head around, one
> > could have the driver defer this this to a kernel resident process
> > that the driver would create on attach.
> 
> Kernel processes don't have file descriptor tables.

Hi All,

I'm hoping you can provide a bit more guidance here as I'm still struggling
with this and I'm running out of ideas.  Again, what I want to do is, some
event (interrupt from an intelligent PCIe card indicating it's "dead") will
cause me to:

1) Open/create a file
2) write raw/random/non-patterned/variable sized data to the file
3) close/save the file

from within my device driver.  To this point, I've tried several things
using vn_open, vn_open_cred, alq_open and cutting/massaging the code from
alq_open with no success.  I've done this from within the context of my
interrupt handler as well as a timer (callout_init, callout_reset) thread as
well as from a thread I create with kproc_create.  In each and every
instance, I get the following crash:

(gdb) bt 15
#0  _mtx_lock_flags (m=0xc0, opts=0x0, file=0xffffffff80611dc0
"/src/FreeBSD/HECTOR/src/sys/kern/vfs_subr.c", line=0x847) at
/src/FreeBSD/HECTOR/src/sys/kern/kern_mutex.c:195
#1  0xffffffff803c8431 in vref (vp=0x0) at
/src/FreeBSD/HECTOR/src/sys/kern/vfs_subr.c:2119
#2  0xffffffff803c02de in namei (ndp=0xffffff80003bfdf2) at
/src/FreeBSD/HECTOR/src/sys/kern/vfs_lookup.c:229
#3  0xffffffff803d6100 in vn_open_cred (ndp=0xffffff80003bfdf2,
flagp=0xffffff8000045a2c, cmode=0x180, cred=0xffffff00014d5000, fp=0x0) at
/src/FreeBSD/HECTOR/src/sys/kern/vfs_vnops.c:133
#4  0xffffffff8023497a in sxg_dump_open_file (adapter=0xffffff80003b4000) at
/src/FreeBSD/HECTOR/src/sys/dev/sxg/sxg.c:5064
#5  0xffffffff8023524a in sxg_dump_thread (arg=<value optimized out>) at
/src/FreeBSD/HECTOR/src/sys/dev/sxg/sxg.c:5170
#6  0xffffffff80235398 in sxg_check_for_hang_timer (arg1=<value optimized
out>) at /src/FreeBSD/HECTOR/src/sys/dev/sxg/sxg.c:4956
#7  0xffffffff80350741 in softclock (arg=<value optimized out>) at
/src/FreeBSD/HECTOR/src/sys/kern/kern_timeout.c:411
#8  0xffffffff80318b08 in intr_event_execute_handlers (p=<value optimized
out>, ie=0xffffff00014d5600) at
/src/FreeBSD/HECTOR/src/sys/kern/kern_intr.c:1146
#9  0xffffffff80319762 in ithread_loop (arg=0xffffff00014da8a0) at
/src/FreeBSD/HECTOR/src/sys/kern/kern_intr.c:1159
#10 0xffffffff80316a9a in fork_exit (callout=0xffffffff803196b0
<ithread_loop>, arg=0xffffff00014da8a0, frame=0xffffff8000045c90) at
/src/FreeBSD/HECTOR/src/sys/kern/kern_fork.c:829
#11 0xffffffff8054dd5e in fork_trampoline () at
/src/FreeBSD/HECTOR/src/sys/amd64/amd64/exception.S:552
#12 0x0000000000000000 in ?? ()

The offending "SIGSEGV, Segmentation fault" is from the following code
(frame 2):

	if (dp == NULL) {
		dp = fdp->fd_cdir;    <----- fdp->fd_cdir = 0
		VREF(dp);
		FILEDESC_SUNLOCK(fdp);
		if (ndp->ni_startdir != NULL) {
			vfslocked =
VFS_LOCK_GIANT(ndp->ni_startdir->v_mount);
			vrele(ndp->ni_startdir);
			VFS_UNLOCK_GIANT(vfslocked);
		}
	}

The *fdp structure looks like:

(gdb) print *fdp
$4 = {
  fd_ofiles = 0xffffff0001345680, 
  fd_ofileflags = 0xffffff0001345720 "", 
  fd_cdir = 0x0, 
  fd_rdir = 0x0, 
  fd_jdir = 0x0, 
  fd_nfiles = 0x14, 
  fd_map = 0xffffff0001345738, 
  fd_lastfile = 0xffffffff, 
  fd_freefile = 0x0, 
  fd_cmask = 0x12, 
  fd_refcnt = 0x1, 
  fd_holdcnt = 0x1, 
  fd_sx = {
    lock_object = {
      lo_name = 0xffffffff805fc20d "filedesc structure", 
      lo_flags = 0x2330000, 
      lo_data = 0x0, 
      lo_witness = 0xffffff8000206c00
    }, 
    sx_lock = 0x11
  }, 
  fd_kqlist = {
    slh_first = 0x0
  }, 
  fd_holdleaderscount = 0x0, 
  fd_holdleaderswakeup = 0x0
}

In this particular instance, I created a nameidata structure via NDINIT that
is:

(gdb) print *ndp
$2 = {
  ni_dirp = 0xffffff8000045a30 "/var/tmp/sxgdump", 
  ni_segflg = UIO_SYSSPACE, 
  ni_startdir = 0x0, 
  ni_rootdir = 0x0, 
  ni_topdir = 0x0, 
  ni_dirfd = 0xffffff9c, 
  ni_vp = 0x0, 
  ni_dvp = 0x0, 
  ni_pathlen = 0x11, 
  ni_next = 0x0, 
  ni_loopcnt = 0x0, 
  ni_cnd = {
    cn_nameiop = 0x1, 
    cn_flags = 0x520000c, 
    cn_thread = 0xffffff00014eb390, 
    cn_cred = 0xffffff00014d5000, 
    cn_lkflags = 0x0, 
    cn_pnbuf = 0xffffff000620c000 "/var/tmp/sxgdump", 
    cn_nameptr = 0x0, 
    cn_namelen = 0x0, 
    cn_consume = 0x0
  }
}

I appreciate the guidance you've given to date and I hope you can provide a
few more words of wisdom to me.

Thanks in advance!

Chris