EVFILT_VNODE doesn't scale to large directory trees?

[Kenton Varda]

That doesn't answer my question.  I'm not even using make.  I could write a
few thousand words describing exactly what I'm trying to do and why it does,
in fact, make sense, but it's really beside the point.  I just want to know
if there is any scalable way to monitor a very large directory tree for
changes.  Is there?

On Sun, Oct 24, 2010 at 9:46 PM, Robert Bonomi <bonomi at mail.r-bonomi.com>wrote:

> > From owner-freebsd-questions at freebsd.org  Sun Oct 24 22:17:42 2010
> > Date: Sun, 24 Oct 2010 18:05:34 -0700
> > From: Kenton Varda <temporal at gmail.com>
> > To: freebsd-questions at freebsd.org
> > Subject: EVFILT_VNODE doesn't scale to large directory trees?
> >
> > Hi all,
> >
> > I am trying to write some code which monitors a possibly-large directory
> > tree for changes.  Specifically, it's a build system, and I want it to
> > automatically start rebuilding whenever I modify a source file.
> >
> > So far the approach I've taken is to use EVFILT_VNODE to watch every file
> > and directory in the tree.  This seems to work OK so far, but it worries
> me
> > that I have to open() every single file.  When I ran the same code on
> > Darwin, it promptly hit the open file descriptor limit, and I'm worried
> that
> > FreeBSD will do the same on larger code trees.
> >
> > Is there any better way to accomplish this?  Hate to say it, but Linux's
> > inotify() seems more scalable here.  From what I can tell from the docs,
> it
> > doesn't require opening the watched files and it will even watch all
> files
> > in a directory with one call.
>
>
> You're re-inventing the wheel.
>
> 1) Set up a 'makefile' for the entire tree.
>
> 2) set up a daemon task that
>     a) cd's to the root direcory of the build tree,
>     b) executes a loop, consisting of
>         1) the 'make all' command,
>         2) a reasonably short 'sleep'
>
>
> If 'efficiency' is a concern, then establish a procedure for checking-out/
> checking-in files from the repository.  When a file is checked in, check
> for (a) it being a new file, *OR* (b) having changes from the prior
> version.
> If either condition is true, fire off 'make' to do the necessary re-build.
>
> NOTE:  'cvs' has the above feature as a built-in option.  simply specify
> 'make' as a program to be run when you do a 'cvs commit' to store changes
> back into the repository.
>
> Did I say soemthing about re-inventing the wheel??   <grin>
>
>
>
>