Re: help with full zfs "partitions" - can't delete files

From: David Christensen <dpchrist_at_holgerdanske.com>
Date: Mon, 03 Jun 2024 22:24:50 UTC
On 6/3/24 13:28, William Dudley wrote:
> The problem:
> 
> FreeBSD 13.3 amd64 system, with
> a zfs pool built from two physical drives.
> The zfs pool has 7 "partitions" (is that what they're called?)
> 
> I was copying files over from another machine and didn't realize that
> I filled one of the partitions.
> 
> I can't proceed now with this one full partition.
> Every single command fails due to "out of space".
> 
> That includes:
> rm (one file or many)
> dd if=/dev/zero of=(some file)
> truncate (somefile)
> zfs destroy poolname/partitionname
> cannot destroy 'poolname/partitionname': out of space
> 
> There are no snapshots, I never created any.
> 
> Extensive googling has not shown any more than bug reports acknowledging
> that this is a problem.
> 
> How do I fix this, short of burning the machine to the ground and starting
> over?
> 
> Thanks,
> Bill Dudley
> 
> This email is free of malware because I run Linux.


On 6/3/24 13:53, William Dudley wrote:
 > zfs get all m2pool/gU4

<begin sort>

 > NAME        PROPERTY              VALUE                  SOURCE
 > m2pool/gU4  aclinherit            restricted             default
 > m2pool/gU4  aclmode               discard                default
 > m2pool/gU4  acltype               nfsv4                  default
 > m2pool/gU4  atime                 on                     default
 > m2pool/gU4  available             0B                     -
 > m2pool/gU4  canmount              on                     default
 > m2pool/gU4  casesensitivity       sensitive              -
 > m2pool/gU4  checksum              on                     default
 > m2pool/gU4  compression           off                    default
 > m2pool/gU4  compressratio         1.00x                  -
 > m2pool/gU4  context               none                   default
 > m2pool/gU4  copies                1                      default
 > m2pool/gU4  createtxg             480                    -
 > m2pool/gU4  creation              Sun Dec  9 19:13 2018  -
 > m2pool/gU4  dedup                 off                    default
 > m2pool/gU4  defcontext            none                   default
 > m2pool/gU4  devices               on                     default
 > m2pool/gU4  dnodesize             legacy                 default
 > m2pool/gU4  encryption            off                    default
 > m2pool/gU4  exec                  on                     default
 > m2pool/gU4  filesystem_count      none                   default
 > m2pool/gU4  filesystem_limit      none                   default
 > m2pool/gU4  fscontext             none                   default
 > m2pool/gU4  guid                  16119321983578430568   -
 > m2pool/gU4  jailed                off                    default
 > m2pool/gU4  keyformat             none                   default
 > m2pool/gU4  keylocation           none                   default
 > m2pool/gU4  logbias               latency                default
 > m2pool/gU4  logicalreferenced     2.34T                  -
 > m2pool/gU4  logicalused           2.34T                  -
 > m2pool/gU4  mlslabel              none                   default
 > m2pool/gU4  mounted               no                     -
 > m2pool/gU4  mountpoint            /u4                    local
 > m2pool/gU4  nbmand                off                    default
 > m2pool/gU4  normalization         none                   -
 > m2pool/gU4  objsetid              84                     -
 > m2pool/gU4  overlay               on                     default
 > m2pool/gU4  pbkdf2iters           0                      default
 > m2pool/gU4  primarycache          all                    default
 > m2pool/gU4  quota                 none                   default
 > m2pool/gU4  readonly              off                    default
 > m2pool/gU4  recordsize            128K                   default
 > m2pool/gU4  redundant_metadata    all                    default
 > m2pool/gU4  refcompressratio      1.00x                  -
 > m2pool/gU4  referenced            2.34T                  -
 > m2pool/gU4  refquota              none                   default
 > m2pool/gU4  refreservation        none                   default
 > m2pool/gU4  relatime              off                    default
 > m2pool/gU4  reservation           none                   default
 > m2pool/gU4  rootcontext           none                   default
 > m2pool/gU4  secondarycache        all                    default
 > m2pool/gU4  setuid                on                     default
 > m2pool/gU4  sharenfs              off                    default
 > m2pool/gU4  sharesmb              off                    default
 > m2pool/gU4  snapdev               hidden                 default
 > m2pool/gU4  snapdir               hidden                 default
 > m2pool/gU4  snapshot_count        none                   default
 > m2pool/gU4  snapshot_limit        none                   default
 > m2pool/gU4  special_small_blocks  0                      default
 > m2pool/gU4  sync                  standard               default
 > m2pool/gU4  type                  filesystem             -
 > m2pool/gU4  used                  2.34T                  -
 > m2pool/gU4  usedbychildren        0B                     -
 > m2pool/gU4  usedbydataset         2.34T                  -
 > m2pool/gU4  usedbyrefreservation  0B                     -
 > m2pool/gU4  usedbysnapshots       0B                     -
 > m2pool/gU4  utf8only              off                    -
 > m2pool/gU4  version               5                      -
 > m2pool/gU4  volmode               default                default
 > m2pool/gU4  vscan                 off                    default
 > m2pool/gU4  written               2.34T                  -
 > m2pool/gU4  xattr                 on                     default

<end sort>

 > Thanks,
 > Bill Dudley
 > This email is free of malware because I run Linux.


When posting console sessions, please be complete -- prompt, exact 
command entered, exact output obtained.  For example:

2024-06-03 15:15:29 toor@vf2 ~
# freebsd-version -kru; uname -a
13.3-RELEASE-p1
13.3-RELEASE-p1
13.3-RELEASE-p2
FreeBSD vf2.tracy.holgerdanske.com 13.3-RELEASE-p1 FreeBSD 
13.3-RELEASE-p1 GENERIC amd64


Looking at the output of `zfs get all m2pool/gU4`, above:

 > m2pool/gU4  type                  filesystem             -

     The ZFS dataset "m2pool/gU4" is a file system.

 > m2pool/gU4  available             0B                     -

     The file system has zero bytes of available space.

 > m2pool/gU4  readonly              off                    default

     The file system is read-write.

 > m2pool/gU4  mountpoint            /u4                    local

     The file system mount point has been set to "/u4".

 > m2pool/gU4  canmount              on                     default

     The file system can be mounted.

 > m2pool/gU4  mounted               no                     -

     The file system is not mounted.


As root, please mount the file system:

# zfs mount m2pool/gU4


Then try removing files and/or directories under /u4.


If either of the above fails, please post the complete console session. 
Also, please post a complete console session for the following commands:

# freebsd-version -kru; uname -a

# zpool list m2pool

# zpool status m2pool

# zpool get all m2pool | sort

# mount | grep m2pool

# ls -ld / /u4


David