Writing a plain text file to disk from kernel space

[Lawrence Stewart]

OK well that's cleared up. Thanks DES.

So I went back to trying the kernio code I found here 
(http://people.freebsd.org/~pjd/misc/kernio/) which does operate 
directly on vnodes (first time I tried it I was getting kernel panics, 
so I assumed the code was a bit dated and carried on down the path of 
trying to get file descriptors working).

Anyways, I modified the filewriter kernel module that I attached to my 
previous post so that it used the kio code, and verified that the code 
does work when called across different threads (in the module's init and 
deinit functions). However, when I ported it into my other module that 
I'm actually working on that uses the pfil hooks, I started getting hard 
resets again.

After further investigation, it turns out that the pfil input hook I'm 
using, which catches packets as they traverse up the network stack, has 
no problems, and will happily write to the file using the kio_write 
function. However, in the pfil output hook, a call to kio_write causes a 
hard reset, with the following text shown on tty0:

Sleeping thread (tid 100069, pid 613) owns a non-sleepable lock
panic: sleeping thread

If I comment out the kio_write code and put a printf instead, there are 
no such problems, so it seems the kio_write function is doing something 
that is upsetting the kernel, but only when called from a function that 
is acting as a pfil output hook? Strikes me as odd behaviour. I don't 
understand which thread the error is in relation to, why that thread is 
sleeping or which lock it is referring to.

I tried wrapping the call to kio_write in a mutex, in case there was a 
race condition caused by multiple threads trying to write to the file at 
the one time, but that hasn't made a difference at all.

I've attached the code that demonstrates the problem, but be warned: 
I've intentionally left it in a state that demonstrates the problem, and 
it will therefore hard reset any machine you run it on.

Any thoughts on what's going on and how I can make the kio code not 
break the kernel when called from within a function that is acting as a 
pfil output hook?

Cheers,
Lawrence


Dag-Erling Smørgrav wrote:
> Lawrence Stewart <lstewart at room52.net> writes:
>   
>> I suspect that you can't use a file descriptor that was opened in one
>> thread in a completely different thread, but I'm not sure if this is
>> true, and if it is true, how to get around it.
>>     
>
> A file descriptor is an index into a file table.  Different threads have
> different file tables.
>
> If you want to read from or write to files within the kernel, you need
> to operate directly on vnodes, not on file descriptors.
>
> DES
>   

-------------- next part --------------
A non-text attachment was scrubbed...
Name: filewriter.ko
Type: application/octet-stream
Size: 6223 bytes
Desc: not available
Url : http://lists.freebsd.org/pipermail/freebsd-hackers/attachments/20070522/2f052053/filewriter.obj
-------------- next part --------------
/*-
 * Copyright (c) 2006 Pawel Jakub Dawidek <p.dawidek at wheel.pl>
 * All rights reserved.
 *
 * Downloaded from http://people.freebsd.org/~pjd/misc/kernio/
 */

#ifndef _SYS_KERNIO_H_
#define	_SYS_KERNIO_H_

struct vnode *kio_open(const char *file, int flags, int cmode);
void kio_close(struct vnode *vp);
int kio_write(struct vnode *vp, void *buf, size_t size);

#endif	/* _SYS_KERNIO_H_ */
-------------- next part --------------
SRCS=vnode_if.h subr_kernio.c filewriter.c
KMOD=filewriter

.include <bsd.kmod.mk> 

-------------- next part --------------
/*-
 * Copyright (c) 2006 Pawel Jakub Dawidek <p.dawidek at wheel.pl>
 * All rights reserved.
 *
 * Downloaded from http://people.freebsd.org/~pjd/misc/kernio/
 */

#include <sys/cdefs.h>
__FBSDID("$FreeBSD$");

#include <sys/param.h>
#include <sys/systm.h>
#include <sys/kernel.h>
#include <sys/kthread.h>
#include <sys/namei.h>
#include <sys/proc.h>
#include <sys/filedesc.h>
#include <sys/vnode.h>
#include <sys/malloc.h>
#include <sys/unistd.h>
#include <sys/fcntl.h>

#include "kernio.h"

struct vnode *
kio_open(const char *file, int flags, int cmode)
{
	struct thread *td = curthread;
	struct nameidata nd;
	int error;

	if (td->td_proc->p_fd->fd_rdir == NULL)
		td->td_proc->p_fd->fd_rdir = rootvnode;
	if (td->td_proc->p_fd->fd_cdir == NULL)
		td->td_proc->p_fd->fd_cdir = rootvnode;

	flags = FFLAGS(flags);
	NDINIT(&nd, LOOKUP, NOFOLLOW, UIO_SYSSPACE, file, td);
	error = vn_open_cred(&nd, &flags, cmode, td->td_ucred, -1);
	NDFREE(&nd, NDF_ONLY_PNBUF);
	if (error != 0)
		return (NULL);
	/* We just unlock so we hold a reference. */
	VOP_UNLOCK(nd.ni_vp, 0, td);
	return (nd.ni_vp);
}

void
kio_close(struct vnode *vp)
{
	struct thread *td = curthread;

	vn_close(vp, FWRITE, td->td_ucred, td);
}

int
kio_write(struct vnode *vp, void *buf, size_t size)
{
	struct thread *td = curthread;
	struct mount *mp;
	struct uio auio;
	struct iovec aiov;

	bzero(&aiov, sizeof(aiov));
	bzero(&auio, sizeof(auio));

	aiov.iov_base = buf;
	aiov.iov_len = size;

	auio.uio_iov = &aiov;
	auio.uio_offset = 0;
	auio.uio_segflg = UIO_SYSSPACE;
	auio.uio_rw = UIO_WRITE;
	auio.uio_iovcnt = 1;
	auio.uio_resid = size;
	auio.uio_td = td;

	/*
	 * Do all of the junk required to write now.
	 */
	vn_start_write(vp, &mp, V_WAIT);
	vn_lock(vp, LK_EXCLUSIVE | LK_RETRY, td);
	VOP_LEASE(vp, td, td->td_ucred, LEASE_WRITE);
  // TODO: Ideally, the flags passed into VOP_WRITE on the next line shouldn't
  //       be hardcoded. Is there a way to calculate them based on the flags
  //       passed into kio_open?
	VOP_WRITE(vp, &auio, IO_UNIT | IO_SYNC | IO_APPEND, td->td_ucred);
	VOP_UNLOCK(vp, 0, td);
	vn_finished_write(mp);
	return (0);
}