The future of ZFS in FreeBSD

[Matthew Macy]

On Wed, Dec 19, 2018 at 14:47 Steven Hartland <killing at multiplay.co.uk>
wrote:

> Thanks for the write up most appreciated. One of the more meaty
> differences is that FreeBSD ZFS still has the only merged and production
> ready TRIM support so my question would be are their any plans to address
> this before creating the new port as going back to a world without TRIM
> support wouldn’t be something I’d look forward to.
>

Well, then please follow up on the request I CC'd you on a week ago asking
that you engage on the deadlist based TRIM  PR. That's a better forum for
discussing these details than lamenting in public lists.

Thanks.

-M




> On Wed, 19 Dec 2018 at 06:51, Matthew Macy <mmacy at freebsd.org> wrote:
>
>> The sources for FreeBSD's ZFS support are currently taken directly
>> from Illumos with local ifdefs to support the peculiarities of FreeBSD
>> where the Solaris Portability Layer (SPL) shims fall short. FreeBSD
>> has regularly pulled changes from Illumos and tried to push back any
>> bug fixes and new features done in the context of FreeBSD. In the past
>> few years the vast majority of new development in ZFS has taken place
>> in DelphixOS and zfsonlinux (ZoL). Earlier this year Delphix announced
>> that they will be moving to ZoL
>> https://www.delphix.com/blog/kickoff-future-eko-2018 This shift means
>> that there will be little to no net new development of Illumos. While
>> working through the git history of ZoL I have also discovered that
>> many races and locking bugs have been fixed in ZoL and never made it
>> back to Illumos and thus FreeBSD. This state of affairs has led to a
>> general agreement among the stakeholders that I have spoken to that it
>> makes sense to rebase FreeBSD's ZFS on ZoL. Brian Behlendorf
>> has graciously encouraged me to add FreeBSD support directly to ZoL
>> https://github.com/zfsonfreebsd/ZoF so that we might all have a single
>> shared code base.
>>
>> A port for ZoF can be found at https://github.com/miwi-fbsd/zof-port
>> Before it can be committed some additional functionality needs to be
>> added to the FreeBSD opencrypto framework. These can be found at
>> https://reviews.freebsd.org/D18520
>>
>> This port will provide FreeBSD users with multi modifier protection,
>> project quotas, encrypted datasets, allocation classes, vectorized
>> raidz, vectorized checksums, and various command line improvements.
>>
>> Before ZoF can be merged back in to ZoL several steps need to be taken:
>> - Integrate FreeBSD support into ZoL CI
>> - Have most of the ZFS test suite passing
>> - Complete additional QA testing at iX
>>
>> We at iX Systems need to port ZoL's EC2 CI scripts to work with
>> FreeBSD and make sure that most of the ZFS Test Suite (ZTS) passes.
>> Being integrated in to their CI will mean that, among other things,
>> most integration issues will be caught before a PR is merged upstream
>> rather than many months later when it is MFVed into FreeBSD. I’m
>> hoping to submit the PR to ZoL some time in January.
>>
>> This port will make it easy for end users on a range of releases to
>> run the latest version of ZFS. Nonetheless, transitioning away from an
>> Illumos based ZFS is not likely to be entirely seamless. The
>> stakeholders I’ve spoken to all agree that this is the best path
>> forward but some degree of effort needs to be made to accommodate
>> downstream consumers. The current plan is to import ZoF and unhook the
>> older Illumos based sources from the build on April 15th or two months
>> after iX systems QA deems ZoF stable - which ever comes later. The
>> Illumos based sources will be removed some time later - but well
>> before 13. This will give users a 3 month period during which both the
>> port and legacy Illumos based ZFS will be available to users. Pools
>> should interoperate between ZoF and legacy provided the user does not
>> enable any features available only in ZoF. We will try to accommodate
>> any downstream consumers in the event that they need that date pushed
>> back. We ask that any downstream consumers who are particularly
>> sensitive to changes start testing the port when it is formally
>> announced and report back any issues they have. I will do my best to
>> ensure that this message is communicated to all those who it may
>> concern. However, I can’t ensure that everyone reads these lists. That
>> is the responsibility of -CURRENT users.
>>
>> -M
>>
> _______________________________________________
>> freebsd-fs at freebsd.org mailing list
>> https://lists.freebsd.org/mailman/listinfo/freebsd-fs
>> To unsubscribe, send any mail to "freebsd-fs-unsubscribe at freebsd.org"
>>
>