GELI: Regression between STABLE-10 and STABLE-11?

[Stefan Esser]

Hi all,

I'm administrating an SVN server for a small company, which is used
to archive work results, but also customer contracts and information
received under NDA.

The system uses pure ZFS (root on ZFS) and part of the "data" pool
is a ZVOL that is used as a GELI provider to hold the confidential
data.

I just tried to upgrade this system to STABLE-11 (or rather 11-BETA1)
and found, that I could not attach the GELI protected partition with:

# geli attach -d -k /root/MY_GELI_KEYFILE /dev/zvol/data/geli.vol

The command failed with "invalid password" (or along that line, sorry
for not writing the exact text down).

The system was running with consistent STABLE-11 kernel and world,
and there was no sign of any other problem.

I performed a roll-back to STABLE-10 and could attach the GELI
partition without any problem with the key-file and password that
had failed under STABLE-11.

This problem is not critical for me (I can create an encrypted backup
of the encrypted data and restore that into a GELI partition created
under STABLE-11), but it might be a general problem - that's why I'm
reporting this failure ...


Some more details:

$ uname -a
FreeBSD XXX.com 10.3-STABLE FreeBSD 10.3-STABLE #0 r318284: Mon May 15
11:58:47 CEST 2017     root at s...  amd64

The (abridged) ZFS pool status is:

$ zpool status
  pool: sys
config:

	NAME              STATE     READ WRITE CKSUM
	sys               ONLINE       0     0     0
	  mirror-0        ONLINE       0     0     0
	    gpt/System-1  ONLINE       0     0     0
	    gpt/System-2  ONLINE       0     0     0

  pool: data
config:
	NAME            STATE     READ WRITE CKSUM
	data            ONLINE       0     0     0
	  mirror-0      ONLINE       0     0     0
	    gpt/Data-1  ONLINE       0     0     0
	    gpt/Data-2  ONLINE       0     0     0

  pool: crypto
config:
	NAME                      STATE     READ WRITE CKSUM
	crypto                    ONLINE       0     0     0
	  zvol/data/geli.vol.eli  ONLINE       0     0     0

$ zfs list -t volume
NAME            USED  AVAIL  REFER  MOUNTPOINT
data/geli.vol  94.5G  78.5G  37.9G  -

I know about the problem of ZFS on ZFS and this will be fixed (I'm
going to convert the file-system in the ZVOL to UFS), but it was a
valid setup when the server was installed a number of years ago.
(And I use "vfs.zfs.vol.recursive=1" as a work-around to disable
the safe-guard that has been implemented to prevent ZFS on ZPOOL.)

I'm able to work around the problem, since the amount of data in the
encrypted partition is small and I wanted to transfer it into an UFS
file-system on a GELI partition, anyway.

Since I had only reserved a short maintenance window for the attempted
upgrade, I could not perform many tests and I lost all logs during the
rollback to STABLE-10. (I had not considered, this could be a problem
that might affect others, at that time.)

Regards, STefan