Example network protocol implementation

[Robert Watson]

On Sat, 9 Dec 2006, Vishal Patil wrote:

> Could someone point me to an example that shows a SIMPLE network protocol 
> implemented over TCP/IP inside the FreeBSD kernel. I think I could look at 
> the NFS client driver but is there an example simpler than that. Also is 
> there a guide explaining how to go about developing TCP/IP based network 
> protocols for FreeBSD. Thanks

Here are some consumers of sockets in the kernel:

- NFS client, which creates and connects both UDP and TCP sockets, uses them
   for I/O, etc.

- NFS server, which uses UDP and TCP sockets for I/O.  Unlike the NFS client,
   it doesn't open the sockets in kernel, rather, it relies on a user process
   (nfsd) passing validated sockets into the kernel.

- System V streams (dev/streams), which uses socket pairs to implement
   streams.  Does creation and I/O.

- fifofs, which implements POSIX fifos using a pair of UNIX domain sockets.
   Again, does creation and I/O.

- portalfs, which implements the portal file system using sockets.

- ng_ksocket, which provides a netgraph interface to sockets in the kernel.

- netncp, which provides an NCP RPC interface over SPX/IPX for nwfs.

- netsmb, which provides an SMB RPC interface over TCP/IP for smbfs.

- rpclnt, which is used by the nfs4client, and is functionally similar to the
   NFS client RPC code for NFS2/NFS3 in nfsclient.

- bootp_subr.c and krpc_subr.c, which are used by the NFS root code to set up
   NFS access during a diskless boot: they perform the bootp exchange to
   retrieve an IP address, and then the necessary RPC mount protocol to query a
   root file handle to set up the NFS client for the file system root.

All of these examples have upsides and downsides, and vary in maturity.  I'd 
probably start by looking at the NFS client and fifofs.  One of the biggest 
questions you'll need to answer is what your event model is and how it will 
relate to any worker threads you may have.  Many of the in-kernel socket 
consumers use socket upcalls to get direct notifications of socket events from 
within the network stack, allowing for fast socket draining and TCP acking. 
On the other hand, in the netisr/ithread context, you can't perform blocking 
memory allocation and disk I/O, so if that will be involved, you'll need 
worker threads in the style of the NFS server.

Robert N M Watson
Computer Laboratory
University of Cambridge