Filesystem full, but df says not.

[Mark Millard]

On 2017-Dec-14, at 12:56 PM, Rodney W. Grimes <freebsd-rwg at pdx.rh.CN85.dnsmgr.net> wrote:

>>> On 2017-Dec-14, at 11:00 AM, bob prohaska <fbsd at www.zefox.net> wrote:
>>> 
>>> 
>>>> An rpi2 running -current reported errors during boot like this on the
>>>> serial console after a graceful reboot:
>>>> 
>>>> UFS /dev/ufs/rootfs (/) cylinder checksum failed: cg 0, cgp: 0x4c0a5f41 != bp: 0x38b82866
>>>> UFS /dev/ufs/rootfs (/) cylinder checksum failed: cg 3, cgp: 0x58e2c1f5 != bp: 0x903c297
>>>> UFS /dev/ufs/rootfs (/) cylinder checksum failed: cg 0, cgp: 0x4c0a5f41 != bp: 0x38b82866
>>> 
>>> Believe the above low-level messages.
>> 
>> And be aware that when a cg checksum fails that CG's freespace is no
>> longer avaliable to be used until the CG checksum has been corrected
>> by a fsck.
>> 
>> You are probably moving your file system accross the "has CG checksum"
>> and "does not have CG checksum" boundary and when you do that your
>> older kernel does not write the checksum and your newer kernel gets
>> upset about that.
>> 
>> 
>>>> /: write failed, filesystem is full
>>>> cp: /etc/motd: No space left on device
>> 
>> This error is correct, as the free space in your CG is not avaliable as
>> that CG has failed chksum. 

I see. Just not how I took the phrase "filesystem is full".
Now the file system can be full when df shows space available.
That will tend to seem odd to folks.

>>> 
>>> My guess:
>>> 
>>> Other places likely translate the more detailed error
>>> classification to more generic classifications that
>>> hopefully result in an appropriate handling of the issue
>>> but is otherwise not necessarily correct.
>> 
>> It is correct as stated, for the reasons I state.
>> 
>>> 
>>> In other words: do not believe the later related messages
>>> in all its detail.
>> 
>> Oh believe it!

Sorry for the misinterpretation.

>> .
>> Mounting late filesystems:.
>> Dec 14 10:08:56 www kernel: pid 1394 (cp), uid 0 inumber 53912 on /: filesystem full
>> 
>> Root is on the microSD card, /usr /var /tmp and swap are on usb flash.
>> 
>> Nevertheless, it reached multi-user and allowed me  to  ssh in and run df,
>> which reported
>> Filesystem             1K-blocks     Used    Avail Capacity  Mounted on
>> /dev/ufs/rootfs          1473116   479936   875332    35%    /
>> devfs                          1        1        0   100%    /dev
>> /dev/msdosfs/MSDOSBOOT     51140     7588    43552    15%    /boot/msdos
>> /dev/da0e               52221244 28697844 19345704    60%    /usr
>> /dev/da0d                3044988   517860  2283532    18%    /tmp
>> /dev/da0a                2031132   122868  1745776     7%    /var

The above sort of result becomes the odd thing
for people to understand the classification.

> This activity probably did not depend on the bad cylinder
> checksums.
> 
>> Still, any activity that wrote to disk repeated the filesystem full error.
>> 
>> This happened with three different kernels, dating Dec 12, 7 and Aug 26.
>> Running fsck -fy once in single user didn't seem to help, although it 
>> finished without obvious errors. Running fsck -fy repeatedly in single-user 
>> seems to have cleared the error, but it's a surprising development.
> 

===
Mark Millard
markmi at dsl-only.net