UFS: unaligned read from GELI with 8k sectorsize

[Konstantin Belousov]

On Sat, Oct 01, 2016 at 02:45:36PM +0300, Konstantin Belousov wrote:
> On Sat, Oct 01, 2016 at 12:42:09AM -0400, Anton Yuzhaninov wrote:
> > Hi all.
> > 
> > I'm trying to install FreeBSD with an encrypted root.
> > 
> > Main difference from commonly used configuration - I want to use geli 
> > with 8k sectorsize.
> > 
> > I've created a geli provider, made UFS with 64k block and 8k fragment, 
> > extracted files to this FS.
> > 
> > While booting from installed system kernel panics after entering geli 
> > passphrase:
> > 
> > g_vfs_done():ada1p6.eli:[READ(offset=21938548736, length=8192)]error = 22
> > vnode_page_generic_gatpager_done: I/O read error 5
> > inid died (signal 6, exit 0)
> > panic: Going nowhere without my init!
> > 
> > errno 22 is EINVAL and it probably returned because 21938548736 is not 
> > multiple of 8192 (geli sectorsize).
> > 
> > Why UFS tries to read with offset, which is not multiple of frag size? 
> > And why this error happens only while mounting geli as root? If I boot 
> > FreeBSD from a USB stick I can attach this geli, mount UFS and read all 
> > files without problem.
> > 
> > FreeBSD 11 / amd64.
> 
> FreeBSD vnode pager assumes that it can read at page granularity.
> Since x86 page size is 4k, sometimes page-in has to occur not on the
> fragment boundary.  In other words, fragment size > 4k are effectively
> not supported.
> 
> Boot needs to execute files from the root mount, which results in the
> mmap(2) attempts on your file system.  While mount and reads/writes do
> not involve the pager, which does not trigger further bugs in the
> buffer cache code.  It should break if you try to execute badly aligned
> ELF binary from your stick, or just mmap() a file from the stick with
> non-8k aligned offset.

It might be not too hard to make this case working, although the speed
of the pagein will be detrimental.  Try this, please.

diff --git a/sys/vm/vnode_pager.c b/sys/vm/vnode_pager.c
index a80a9c2..01e2ea2 100644
--- a/sys/vm/vnode_pager.c
+++ b/sys/vm/vnode_pager.c
@@ -796,7 +796,7 @@ vnode_pager_generic_getpages(struct vnode *vp, vm_page_t *m, int count,
 	 * getting pages via VOP_READ.
 	 */
 	error = VOP_BMAP(vp, foff / bsize, &bo, &bp->b_blkno, &after, &before);
-	if (error == EOPNOTSUPP) {
+	if (error == EOPNOTSUPP || (error == 0 && bo->bo_bsize > PAGE_SIZE)) {
 		relpbuf(bp, freecnt);
 		VM_OBJECT_WLOCK(object);
 		for (i = 0; i < count; i++) {