svn commit: r304321 - in head/sys: boot/efi/boot1 boot/efi/loader boot/i386/boot2 boot/i386/gptboot boot/i386/gptzfsboot boot/i386/zfsboot boot/userboot/ficl boot/userboot/userboot boot/userboot/zf...

[Toomas Soome]

> On 22. aug 2016, at 11:09, Andriy Gapon <avg at FreeBSD.org> wrote:
> 
> On 22/08/2016 10:20, Andriy Gapon wrote:
>> This commit breaks boot process for me and in a quite weird way.
>> I don't have a serial console, so a couple of screenshots.
>> This is what happens with this change:
>> https://people.freebsd.org/~avg/boot-fail-1024x768.jpg
>> This is what I have with the previous loader:
>> https://people.freebsd.org/~avg/boot-success-1024x768.jpg
>> 
>> As you can see somehow the HDD gets misdetected as a floppy, BIOS disk ID is 0x0
>> as opposed to 0x80.  Also, the disk size is incorrect too.  Additionally the
>> firewire is not detected.
>> 
>> I suspect that the problem may have to do with the increased loader size.
>> I must add that I have these in src.conf:
>> LOADER_BZIP2_SUPPORT=yes
>> LOADER_FIREWIRE_SUPPORT=yes
> 
> Removing both of those options allows the boot to succeed.
> Which sort of, maybe confirms the hypothesis.
> 

yep, just enabling firewire does not blow it. 


> Also, as extra data points, this is how SMAP is reported by a good zfsloader:
> https://people.freebsd.org/~avg/boot-smap-1024x768.jpg
> And it seems to be corrupted when using the bad zfsloader:
> https://people.freebsd.org/~avg/boot-smap-corrupt-1024x768.jpg
> 
> Base memory size is 634880 (almost enough for everyone).
> Extended memory is ~ 3.5GB and the high memory is 64MB at the top of it.
> 
> File size of the loader that does not work is 483328 bytes.
> $ size /usr/obj/usr/src/sys/boot/i386/zfsloader/zfsloader.bin
>    text    data      bss      dec       hex   filename
>  438000   26416   130896   595312   0x91570
> /usr/obj/usr/src/sys/boot/i386/zfsloader/zfsloader.bin
> 
> File size of the loader that works is 450560 bytes.
> $ size /usr/obj/usr/src/sys/boot/i386/zfsloader/zfsloader.bin
>    text    data     bss      dec       hex   filename
>  410920   23304   50636   484860   0x765fc
> /usr/obj/usr/src/sys/boot/i386/zfsloader/zfsloader.bin
> 
> So, it seems that there is a practical limit on a loader size for real-world x86
> BIOS-based systems.  We are very close to the limit with the default ZFS loader
> and we cross that limit if additional loader features are enabled.
> 
> My opinion is that we should stop cramming all possible ZFS features into the
> loader.  Instead we should admit that the boot pool, or at least boot
> filesystem, must have certain limitations on features that it can use.  Then
> there is no need to add support for those features to the loader.


Well, it is arguable if its more important to have bzip in loader or support for checksum algorithms, especially as zfs does compression, but lets not get onto this slippery ground;)

but, yes, the root cause of the issue is that if additional [relatively large] code is included by src.conf setup, we will hit the ceiling of <1MB memory area, as client base (0xa000) + code + data are all occupying sequential memory and we are reaching EBDA area in low memory.

And indeed, it does mean more conservative approach about what is included and how.

btw, just to compare, zfsloader with ficl 4 + zfs additionally with gzip (as gzip is needed for for other reasons anyhow, so its “free lunch”) in my illumos build is:

-r--r--r--   1 root     sys       434176 aug 21 16:04 /boot/zfsloader

and the current head + firewire in freebsd is:

-rw-r--r--  1 root  wheel  454656 Aug 22 11:54 /boot/zfsloader

I do suspect the size difference there is partially due to ficl, in illumos (ficl 4):

-rw-r--r--   1 tsoome   staff     132508 aug 22 09:18 libficl.a

and freebsd (ficl 3):

-rw-r--r--  1 root  wheel  213748 Aug 19 01:57 libficl.a

so, there definitely is some space… 

rgds,
toomas

> 
> Personally, I would prefer that this commit is backed out unless it can be
> strongly justified.  Unless we can quickly find a way to run the loader at a
> different, less restricted memory location.
> 
>> My memory of loader's memory placement and layout has faded, so I can't
>> elaborate further on my guess.
>> 
>> Also, there is this report: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=212038
>> That problem could have a different cause.  It should be easier to analyze as
>> the it happens with bhyveload, i.e., in userland.
> 
> 
> -- 
> Andriy Gapon