ZFS trouble: unbelievably large files created

[Jeremy Chadwick]

On Fri, Oct 15, 2010 at 02:11:36PM -0400, David Samms wrote:
> On 10/15/10 13:48, Jeremy Chadwick wrote:
> >On Fri, Oct 15, 2010 at 01:07:37PM -0400, David Samms wrote:
> >>I am running into issues with ZFS where perl, specifically amavisd
> >>running spamassassin creates unbelievably large files, as in files
> >>100x the size of the hard disk.
> >>
> >>My setup is a host run a dozen jails.  The host is amd64 FreeBSD
> >>8.1-RELEASE-p1 #6 as of Thu Sep 30.  The jails are running 8.0
> >>release as I have not yet upgraded them.  This setup has been very
> >>stable till I introduced ZFS.  Below is the zpool setup:
> >>
> >>zpool status
> >>   pool: m1012
> >>  state: ONLINE
> >>  scrub: none requested
> >>config:
> >>
> >>         NAME        STATE     READ WRITE CKSUM
> >>         m1012       ONLINE       0     0     0
> >>           mirror    ONLINE       0     0     0
> >>             ad10    ONLINE       0     0     0
> >>             ad12    ONLINE       0     0     0
> >>
> >>Last night I moved six jails from my UFS+S RAID5 file system to the
> >>new ZFS file system.  Since then perl occasionally runs at 100% disk
> >>usage accessing files in /var/amavis/.spamassassin  Here is the
> >>current directory listing:
> >>
> >>ls -l .spamassassin
> >>total 13352131
> >>-rw-------  1 vscan  vscan           684032 Oct 15 12:02 auto-whitelist
> >>-rw-------  1 vscan  vscan               40 Oct 15 12:37 bayes.lock
> >>-rw-------  1 vscan  vscan          1294336 Oct 15 12:38 bayes_seen
> >>-rw-------  1 vscan  vscan          4227072 Oct 15 12:38 bayes_toks
> >>-rw-------  1 vscan  vscan     553184002048 Oct 15 12:38
> >>bayes_toks.expire3515
> >>-rw-------  1 vscan  vscan  140743122878464 Oct 15 12:14
> >>bayes_toks.expire97254
> >>
> >>The last file is 140TB which is pretty good sized for a 1TB disk.
> >>Disk compression is OFF.  du -hs reports 13G int the .spamassassin
> >>directory.
> >>
> >>Any thoughts?
> >
> >Thread titled "Strange ZFS problem, filesystem claims to be full when
> >clearly not full":
> >
> >http://lists.freebsd.org/pipermail/freebsd-fs/2010-September/thread.html#9610
> >
> 
> I don't think it is related, but could be.  I am not seeing the disk
> as full, although I do know that if I let perl continue to write it
> does fill the ZFS partition.
> 
> df -h
> Filesystem      Size    Used   Avail Capacity  Mounted on
> m1012/shawxp     40G     19G     21G    48%    /
> 
> zfs list
> m1012/shawxp        20.0G  20.9G  19.1G  /m1012/shawxp

You need to read the entire thread and not skim it.

-- 
| Jeremy Chadwick                                   jdc at parodius.com |
| Parodius Networking                       http://www.parodius.com/ |
| UNIX Systems Administrator                  Mountain View, CA, USA |
| Making life hard for others since 1977.              PGP: 4BD6C0CB |