FreeBSD Quarterly Status Report - Second Quarter 2021

Go to: [ bottom of page ] [ top of archives ] [ this month ]

From: Daniel Ebdrup Jensen <debdrup_at_FreeBSD.org>
Date: Sat, 24 Jul 2021 07:09:36 UTC

Introduction

This report covers FreeBSD related projects for the period between April and
June, and is the second of four planned reports for 2021.

Some of this reports highlights include but are not limited to work on an
experimental installer, changes to pf, additional work on the Linuxulator,
updates on the state of kernel sanitizers, coverage of the raidz expansion
feature for ZFS, and some news about resource accounting.

Yours,
Daniel Ebdrup Jensen, with status hat on.

━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
Table of Contents

• FreeBSD Team Reports
□ FreeBSD Foundation
□ FreeBSD Release Engineering Team
□ Cluster Administration Team
□ Continuous Integration
□ Ports Collection
□ Documentation Engineering Team
• Projects
□ BSDDialog - TUI Widgets
□ Experimental installer
□ Git Migration Working Group
□ LLDB Debugger Improvements
□ Performance Monitoring Counters
□ syzkaller on FreeBSD
• Kernel
□ NXP DPAA driver
□ ENA FreeBSD Driver Update
□ Graphics Driver Update from Linux 5.7
□ Intel Wireless driver support
□ Kernel Sanitizers
□ Linux compatibility layer update
□ NXP LS1028A SoC support
□ Marvell ARM64 SoCs support
□ Multicast routing rework
□ VFS path descriptors API
□ Dummynet support for pf
□ Ethernet support for pf
□ pf syncookie support
□ Microchip PolarFire SoC support
□ Racct (Resource Accounting) Bug Fixes and Improvements
□ OpenZFS RAIDZ Expansion update
□ Microchip Switchtec support
• Ports
□ Emacs Ports
□ FreeBSD Erlang Ecosystem Ports update
□ GCC in the Ports Tree (lang/gcc*)
□ KDE on FreeBSD
□ libglvnd on FreeBSD
□ FreeBSD in Science
• Documentation
□ FreeBSD Translations on Weblate
• Miscellaneous
• Third-Party Projects
□ FreshPorts
□ helloSystem
□ Containers & FreeBSD: Pot, Potluck & Potman

━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

FreeBSD Team Reports

Entries from the various official and semi-official teams, as found in the
Administration Page.

FreeBSD Foundation

Contact: Deb Goodkin <deb@FreeBSDFoundation.org>

The FreeBSD Foundation is a 501(c)(3) non-profit organization dedicated to
supporting and promoting the FreeBSD Project and community worldwide. Funding
comes from individual and corporate donations and is used to fund and manage
software development projects, conferences and developer summits, and provide
travel grants to FreeBSD contributors. The Foundation purchases and supports
hardware to improve and maintain FreeBSD infrastructure and provides resources
to improve security, quality assurance, and software engineering efforts;
publishes marketing material to promote, educate, and advocate for the FreeBSD
Project; facilitates collaboration between commercial vendors and FreeBSD
developers; and finally, represents the FreeBSD Project in executing contracts,
license agreements, and other legal arrangements that require a recognized
legal entity.

Here are some highlights of what we did to help FreeBSD last quarter:

COVID-19 Impact to the Foundation

Like most organizations, our team continued to work from home. Our temporary
ban on travel for staff members remains in effect, but continues to not affect
our output too much, since most conferences are still virtual. As the world
continues to open up we will re-evaluate the travel ban. We continued
supporting the community and Project through our regular channels.

Partnerships and Commercial User Support

We help facilitate collaboration between commercial users and FreeBSD
developers. We also meet with companies to discuss their needs and bring that
information back to the Project. Our temporary travel ban stayed in effect
during Q2, so we continued meeting with corporate users virtually. If things
look good for in-person meetings in the fall, then we’ll start incorporating
those into an in-person and virtual meeting mix.

Fundraising Efforts

First, we’d like to say thank you to everyone who has given us a financial
contribution this year! Last quarter we raised $70,410, which includes
donations from organizations like Verisign, VMware, and Stormshield, as well as
many individuals.

We depend on these donations to fund our work supporting FreeBSD. Late last
year we decided to put more funding into helping to improve FreeBSD. We hired a
Sr. Software Developer to work on arm64 and a Project Coordinator to manage our
projects and interface with the community. We also hired two of our part-time
software developers full-time. The purpose of this was to provide more
resources to step in to implement and improve major features in FreeBSD, review
patches and bug reports, implement bug fixes, and support the security efforts.
This ensures FreeBSD remains the innovative, secure, and reliable operating
system that you rely on.

You’ll find out how we used your donations for Q2 in our report, as well as
individual reports throughout this status report.

We are excited about our plans for 2021, which include more FreeBSD online
advocacy and training, operating system course content, and the software
development work mentioned above. While we are still in this pandemic, we’re
working hard to help connect folks within the community with more virtual
opportunities.

Please consider making a donation to help us continue and increase our support
for FreeBSD in 2021: https://www.freebsdfoundation.org/donate/.

We also have the Partnership Program, to provide more benefits for our larger
commercial donors. Find out more information and share with your companies!
https://www.freebsdfoundation.org/FreeBSD-foundation-partnership-program/

OS Improvements

During the second quarter Foundation staff and grant recipients committed 348
src tree changes, 19 ports tree changes, and 11 doc tree changes. This
represents 40% of src commits which identify a sponsor. For ports commits it’s
15%, and 18% of doc commits. Foundation staff and grant recipients also
contributed to a number of 3rd party projects. Two notable examples are the
LLVM project’s LLDB debugger and the Syzkaller code-coverage-guided system call
fuzzer.

You can read about a number of Foundation projects in individual quarterly
reports. Smaller projects and improvements include:

• Implement on-demand coredump generation by the kernel via ptrace
(PT_COREDUMP)

• General kernel debugging improvements

• Remove obsolete kernel mcount profiling

• Nullfs and tmpfs bug fixes

• libc cleanup and improvements

• rtld dlerror and thread local variable fixes (reported by Julia developers)

• kqueue and POSIX timer fixes

• UFS bug fixes

• Capsicum socket operation improvements

• hwpmc (hardware performance profiling) maintenance and CPU support

• Cirrus-CI boot smoke test

• sndstat(4) schema updates

• AMD PCI passthrough fixes in vmm(4), see: commit 1, commit 2 and review

• Virtio 1.0 modern support in bhyve(8)

As usual Foundation staff also supported the project with significant effort on
code reviews and general bug triage and fixes. Also, Ka Ho added an article
titled Use Language Servers for Development in the FreeBSD Src Tree.

Continuous Integration and Quality Assurance

The Foundation provides a full-time staff member and funds projects on
improving continuous integration, automated testing, and overall quality
assurance efforts for the FreeBSD project.

During the second quarter of 2021, work on pre-commit tests and building
release artifacts in the CI environment continued. A project using the netperf
cluster to do network-related CI jobs is being planned.

See the FreeBSD CI section of this report for completed work items and detailed
information.

Supporting FreeBSD Infrastructure

The Foundation provides hardware and support to improve the FreeBSD
infrastructure. Last quarter, we supported the test cluster at Sentex,
purchased a few needed parts for infrastructure in general, and started working
with the clusteradm team on a more efficient and improved hardware request/
purchase process.

FreeBSD Advocacy and Education

A large part of our efforts are dedicated to advocating for the Project. This
includes promoting work being done by others with FreeBSD; producing advocacy
literature to teach people about FreeBSD and help make the path to starting
using FreeBSD or contributing to the Project easier; and attending and getting
other FreeBSD contributors to volunteer to run FreeBSD events, staff FreeBSD
tables, and give FreeBSD presentations.

The FreeBSD Foundation sponsors many conferences, events, and summits around
the globe. These events can be BSD-related, open source, or technology events
geared towards underrepresented groups. We support the FreeBSD-focused events
to help provide a venue for sharing knowledge, to work together on projects,
and to facilitate collaboration between developers and commercial users. This
all helps provide a healthy ecosystem. We support the non-FreeBSD events to
promote and raise awareness of FreeBSD, to increase the use of FreeBSD in
different applications, and to recruit more contributors to the Project. While
we were still unable to attend in-person meetings due to Covid-19, we were able
to attend virtual events and help organize the June 2021 FreeBSD Developer
Summit. In addition to attending and planning virtual events, we are
continually working on new training initiatives and updating our selection of
how-to guides to facilitate getting more folks to try out FreeBSD.

Check out some of the advocacy and education work we did last quarter:

• Participated as an Industry Partner for USENIX LISA21

• Held two new FreeBSD Fridays: Introduction to BastilleBSD and How to Submit
a Patch to FreeBSD.

• Organized content and promoted FreeBSD Day on June 18-19, 2021

• Helped organize and run the June 2021 FreeBSD Developer Summit - videos are
now posted on the wiki.

• Presented at the The 16th Open Source China Open Source World Summit on
June 18

• New blog posts on the Linxulator work we funded and What’s new in FreeBSD
13.0

• New How To Guide on Git

• Continued to promote the FreeBSD Office Hours series Videos from the one
hour sessions can be found on the Project’s YouTube Channel. See the Office
Hours section of this report for more information.

• Committed to be a Silver Sponsor for EuroBSDcon

Keep up to date with our latest work in our newsletters: https://
www.freebsdfoundation.org/news-and-events/newsletter/

We help educate the world about FreeBSD by publishing the professionally
produced FreeBSD Journal. As we mentioned previously, the FreeBSD Journal is
now a free publication. Find out more and access the latest issues at https://
www.FreeBSDfoundation.org/journal/.

You can find out more about events we attended and upcoming events at https://
www.FreeBSDfoundation.org/news-and-events/.

Legal/FreeBSD IP

The Foundation owns the FreeBSD trademarks, and it is our responsibility to
protect them. We also provide legal support for the core team to investigate
questions that arise.

Go to https://www.FreeBSDfoundation.org to find out how we support FreeBSD and
how we can help you!

FreeBSD Release Engineering Team

Links:
FreeBSD 13.0-RELEASE schedule URL: https://www.freebsd.org/releases/13.0R/
schedule/
FreeBSD 13.0-RELEASE announcement URL: https://www.freebsd.org/releases/13.0R/
announce/
FreeBSD releases URL: https://download.freebsd.org/ftp/releases/ISO-IMAGES/
FreeBSD development snapshots URL: https://download.freebsd.org/ftp/snapshots/
ISO-IMAGES/

Contact: FreeBSD Release Engineering Team, <re@FreeBSD.org>

The FreeBSD Release Engineering Team is responsible for setting and publishing
release schedules for official project releases of FreeBSD, announcing code
freezes and maintaining the respective branches, among other things.

During the second quarter of 2021, the Release Engineering Team completed work
on the 13.0-RELEASE cycle, the first release from the stable/13 branch. During
the release cycle, one additional BETA build and two additional RC builds were
added to the schedule, however the release cycle went smoothly, overall.

Additionally throughout the quarter, several development snapshots builds were
released for the main, stable/13, and stable/12 branches. Development snapshot
builds for stable/11 will no longer be available.

Thank you to all that have helped test the 13.0 builds up until this point and
have reported issues. As always, we strive for quality over quantity.

Sponsor: Rubicon Communications, LLC ("Netgate") Sponsor: The FreeBSD
Foundation

Cluster Administration Team

Contact: Cluster Administration Team <clusteradm@FreeBSD.org>

Links:
Cluster Administration Team members URL: https://www.freebsd.org/administration
/#t-clusteradm

The FreeBSD Cluster Administration Team consists of the people responsible for
administering the machines that the Project relies on for its distributed work
and communications to be synchronised. In this quarter, the team has worked on
the following:

• Moved Phabricator (reviews.freebsd.org) to a faster machine

• Moved https://www.freebsd.org CGI backend to a faster machine

• Improved the network design of our primary cluster site

□ Removed public IPv4 connectivity from VLANs not hosting public-facing
services

□ Tidied up the pkgbuild and pkgexp VLANs where the production and
experimental package builders live.

□ Moved developer reference machines and universe building machines to a
different VLAN to better accommodate aarch64 and ppc64 machines

• Upgraded the machines running important internal FreeBSD.org services

□ DNS, Kerberos, LDAP, LetsEncrypt.org certbot, etc

□ FTP, Subversion, Git mirror seed

• Upgraded the developer reference machines and universe builders to a newer
FreeBSD 14-CURRENT

• Upgraded package building machines to newer versions of FreeBSD 14-CURRENT

• Assisted postmaster with migrating the mailing lists from Mailman to mlmmj

• Recycled a particularly troublesome PowerPC64 package building machine with
extreme prejudice (rest in pieces)

• Installed a new production PowerPC64 package builder

• Worked with Git migration working group for ports tree migration

• Ongoing day to day cluster management activity

□ Putting out fires

□ Babysitting pkgsync

Work in progress:

• Move pkg-master.nyi to new hardware

• Improve to the package building infrastructure

• Review the service jails and service administrators operation

• Setup powerpc pkgbuilder/ref/universal machines

• Search for more providers that can fit the requirements for a generic
mirrored layout or a tiny mirror

• Upgrading public-facing cluster services from 12.2-STABLE to 13.0-STABLE

• Installing a new cluster site in Japan

□ Full mirror site (ftp, pkg, git, doc, dns,…)

□ The network and machine resources for this mirror are generously
sponsored by the Cloud and SDN Laboratory at BroadBand Tower, Inc., one
of the Internet data center service providers in Tokyo, Japan, with
300+ Gbps international IP transit bandwidth

• Improvements to GeoDNS routing, particularly in Asia

• Working with doceng@ to improve https://www.freebsd.org and https://
docs.freebsd.org

• Improve the web service architecture

• Improve the cluster backup plan

Continuous Integration

Links:
FreeBSD Jenkins Instance URL: https://ci.FreeBSD.org
FreeBSD Hardware Testing Lab URL: https://ci.FreeBSD.org/hwlab
FreeBSD CI artifact archive URL: https://artifact.ci.FreeBSD.org
FreeBSD CI weekly report URL: https://hackmd.io/@FreeBSD-CI
FreeBSD Jenkins wiki URL: https://wiki.freebsd.org/Jenkins
Hosted CI wiki URL: https://wiki.freebsd.org/HostedCI
3rd Party Software CI URL: https://wiki.freebsd.org/3rdPartySoftwareCI
Tickets related to freebsd-testing@ URL: https://preview.tinyurl.com/y9maauwg
FreeBSD CI Repository URL: https://github.com/freebsd/freebsd-ci

Contact: Jenkins Admin <jenkins-admin@FreeBSD.org>
Contact: Li-Wen Hsu <lwhsu@FreeBSD.org>
Contact: freebsd-testing Mailing List
Contact: IRC #freebsd-ci channel on EFNet

The FreeBSD CI team maintains the continuous integration system of the FreeBSD
project. The CI system firstly checks the committed changes can be successfully
built, then performs various tests and analysis over the newly built results.
The artifacts from those builds are archived in the artifact server for further
testing and debugging needs. The CI team members examine the failing builds and
unstable tests and work with the experts in that area to fix the code or adjust
test infrastructure. The details of these efforts are available in the weekly
CI reports.

During the second quarter of 2021, we continued working with the contributors
and developers in the project to fulfil their testing needs and also keep
collaborating with external projects and companies to improve their products
and FreeBSD.

Important changes:

• The build environment of main (-CURRENT) and stable/13 branches are changed
to 13.0-RELEASE

Retired jobs:

• GCC 6 build for main on amd64

Work in progress and open tasks:

• Designing and implementing pre-commit CI building and testing

• Designing and implementing use of CI cluster to build release artifacts as
release engineering does

• Collecting and sorting CI tasks and ideas here

• Testing and merging pull requests in the FreeBSD-ci repo

• Reducing the procedures of CI/test environment setting up for contributors
and developers

• Setting up the CI stage environment and putting the experimental jobs on it

• Setting up public network access for the VM guest running tests

• Implementing using bare metal hardware to run test suites

• Adding drm ports building tests against -CURRENT

• Planning to run ztest and network stack tests

• Adding more external toolchain related jobs

• Improving the hardware lab to be more mature and adding more hardware

• Helping more software get FreeBSD support in its CI pipeline (Wiki pages:
3rdPartySoftwareCI, HostedCI)

• Working with hosted CI providers to have better FreeBSD support

Please see freebsd-testing@ related tickets for more WIP information, and don’t
hesitate to join the effort!

Sponsor: The FreeBSD Foundation

Ports Collection

Links:
About FreeBSD Ports URL:https://www.FreeBSD.org/ports/
Contributing to Ports URL: https://docs.freebsd.org/en/articles/contributing/
ports-contributing/
FreeBSD Ports Monitoring URL: http://portsmon.freebsd.org/
Ports Management Team URL: https://www.freebsd.org/portmgr/
Ports Tarball URL: http://ftp.freebsd.org/pub/FreeBSD/ports/ports/

Contact: René Ladan <portmgr-secretary@FreeBSD.org>
Contact: FreeBSD Ports Management Team <portmgr@FreeBSD.org>

The Ports Management Team is responsible for overseeing the overall direction
of the Ports Tree, building packages, and personnel matters. Below is what
happened in the last quarter.

According to FreshPorts, we currently enjoy a little over 44,200 ports in the
Ports Collection. These are accompanied by 2,532 PRs of which 526 are
unassigned. During the last quarter, there were 10,428 commits on main by 164
committers and 618 commits on 2021Q2 by 59 committers. Compared to 2021Q1, the
number of ports and PRs remained roughly the same and the main tree saw 10%
more commits this quarter.

During the last quarter, we welcomed Charlie Li (vishwin@) and Guangyuan Yang
(ygy@) and said goodbye to jlaffaye@, jpaetzel@, kmoore@, lifanov@, miwi@ and
shurd@.

On the infrastructure side, a new USES, ansible, was added so that
Ansible-related ports can more easily define plugins and modules. Several
default versions were updated:

• Default version of PYTHON and PYTHON3 switched to 3.8

• Default version of librsvg2 on PowerPC switched to rust

Regarding the license framework, the badly maintained LEGAL was removed in
favor of per-port LICENSEs and the BSD0CLAUSE license was added. In the options
framework, pre-defined shared version control OPTIONS and descriptions were
added.

Some notable port updates:

• Firefox 89.0.2

• Firefox-esr 78.11.0

• Chromium 91.0.4472.114

• Ruby 3.0.1

• Xorg server 1.20.11

Finally, antoine@ ran 18 exp-runs to test updates for cmake, expat2, KDE,
meson, poppler, rust, Xorg and the lapack family, to switch the default version
of Python to 3.8, and to remove the outdated qtchooser.

Documentation Engineering Team

Links:
FreeBSD Documentation Project URL: https://www.freebsd.org/docproj/
FreeBSD Documentation Project Primer for New Contributors URL: https://
docs.freebsd.org/en/books/fdp-primer/
Documentation Engineering Team URL: https://www.freebsd.org/administration/#
t-doceng

Contact: FreeBSD Doceng Team <doceng@FreeBSD.org>

The doceng@ team is a body to handle some of the meta-project issues associated
with the FreeBSD Documentation Project, more infomation on FreeBSD Doceng Team
Charter.

During the last quarter, we welcomed back Ceri Davies (ceri@) as a doc
committer. Sergio Carlavilla (carlavilla@) and Danilo G. Baio (dbaio@) joined
the doceng@ team.

A new article was included in the FreeBSD Documentation by Ka Ho Ng (khng@),
Use Language Servers for Development in the FreeBSD Src Tree.

Much work was done regarding the transition to Hugo/Asciidoctor, but there are
still pending issues in the FreeBSD Documentation Project after the migration.
We thank everyone who is helping to find and fixing them. The TODO items wiki
list was refreshed, and everyone is welcome to contribute to it.

Projects

Projects that span multiple categories, from the kernel and userspace to the
Ports Collection or external projects.

BSDDialog - TUI Widgets

Links:
BSDDialog Project URL: https://gitlab.com/alfix/bsddialog
Dialog Project URL: https://invisible-island.net/dialog
GPLinBase Wiki Page URL: https://wiki.freebsd.org/GPLinBase

Contact: Alfonso Sabato Siciliano <alfonso.siciliano@email.com>

The purpose of this project is to provide an utility and a library to build
scripts and tools with UI Widgets in a terminal. The project is inspired by
Dialog; however BSDDialog is released under the terms of the BSD-2-Clause
License while Dialog is licensed as LGPL so a link of the project has been
added to the "GPL Software in the Base System" wiki page.

The aim is to provide full compatibility with the dialog utility (the challenge
is to implement over seventy options and almost thirty widgets). The API
compatibility with the library is not a priority because libbsddialog should
meet FreeBSD’s needs, for example it provides the new mixedlist() function, it
can take as argument a set of: checklists, radiolists and separators, so it
could be useful for building a dialog4ports clone: easier to implement and not
depending on non-permissive dependencies.

BSDDialog is currently under development, no widget is really completed
(autosizing, resizing and scrolling are missing), nevertheless runnable
examples for the utility and the library are inside the example folders and
described in the README.

Finally, a curiosity: BSDDialog started from the MixerTUI code base, however
the original code has been almost completely rewritten.

Experimental installer

Links:
Installer repository URL: https://github.com/yangzhong-freebsd/lua-httpd
Live ISO containing installer URL: https://github.com/yangzhong-freebsd/ISO

Contact: Yang Zhong <yzhong@freebsdfoundation.org>

bsdinstall is FreeBSD’s current installer. It is a terminal application with an
unwieldy interface, and it asks some very obscure questions that are hard to
understand for ordinary users. So, the purpose of this project is to create a
graphical installer for FreeBSD that has a more streamlined interface, and to
improve other aspects of the FreeBSD installation process.

The experimental installer uses a web front-end: a web server runs locally from
the installation media, and the user configures their install by filling out
web forms on a browser also running on the installation media. A Web interface
is flexible and accessible, so it suits an installer well. This interface can
also support remote installs, where the server runs on the target and the
install is configured through some other machine, though I have not done much
work here.

The installer also aims to have a modular design, where the user configuration
options are written to an installation config file, that is then used for the
actual installation. While bsdinstall already supports scripted installations,
its config file format is very free-form. A more rigid config file design would
make it easier to write other installation front-ends in the future.

The installer is currently a rough proof-of-concept, but it can handle a basic
installation with limited configuration. Help with testing would be
appreciated; you can try the installer by downloading one of the releases in
the ISO repository. Also, please email me with any thoughts on the design of
the installer, or on useful features it should have.

Sponsor: The FreeBSD Foundation

Git Migration Working Group

Links:
Git for FreeBSD development wiki URL: https://wiki.freebsd.org/Git
Git transition wiki URL: https://wiki.freebsd.org/GitTransition
doc git repo web URL: https://cgit.FreeBSD.org/doc
ports git repo web URL: https://cgit.FreeBSD.org/ports
src (base system) git repo web URL: https://cgit.FreeBSD.org/src
Committers guide Git primer URL: https://docs.freebsd.org/en/articles/
committers-guide/#git-primer
Handbook Using Git appendix URL: https://docs.freebsd.org/en/books/handbook/
mirrors/#git
Game of Trees URL: http://gameoftrees.org/
gitup URL: https://github.com/johnmehr/gitup

Contact: Li-Wen Hsu <lwhsu@FreeBSD.org>
Contact: Warner Losh <imp@FreeBSD.org>
Contact: Ed Maste <emaste@FreeBSD.org>
Contact: Ulrich Spörlein <uqs@FreeBSD.org>
Contact: FreeBSD-git mailing list
Contact IRC #gitcvt channel on EFnet

The Git Working Group has been working on ports repository migration to Git,
this task started at the end of the March, beginning with a final Subversion
commit on March 31st to indicate that the conversion started. The whole
migration completed on April 6th. Since 2021Q2, the ports quarterly branch is
created in Git repository only. We continued working on portsnap and other
ports infrastructure to accommodate git.

We continued working on implementing and updating commit hooks. The work
including helped change FreeBSD 13.0 release process to use Git. And we are
sorting and making the infrastructure available to the public, as well as the
documents.

On June 8th, we worked with our ZFS developers to have better tracking of
upstream OpenZFS development. The vendor/openzfs branch was renamed to vendor/
openzfs/legacy. Two new branches were imported directly from upstream, vendor/
openzfs/master and vendor/openzfs/zfs-2.1-release, and merged to main and
stable/13. The details and the required action to correct the errors might
result for the people tracking the old branch is available at https://
lists.freebsd.org/archives/freebsd-current/2021-June/000153.html

The Git Working Group continues to track progress on two permissively-licensed
git compatible tools: Gitup and Game of Trees. Gitup is a small,
dependency-free tool to clone and update git repositories. It is used only to
keep a local tree up-to-date, and has no support for local commits.

Game of Trees is a version control client that is compatible with Git
repositories. It provides a user interface and workflow that is distinct from
that of Git. It is in no way intended to be a drop-in replacement for git, but
can be used to develop software maintained in a Git repository.

Gitup and Game of Trees are currently available as ports and packages. Future
work will evaluate them as candidates for the base system.

The core team began a new effort to investigate and evaluate new workflow
changes in the June 2021 DevSummit. In the third quarter of 2021 we expect to
complete the remaining migration tasks and create a new working group to help
with workflow refresh. We’ve wound up our regular meetings, and the remaining
migration tasks are being done by individuals (Li-Wen Hsu is mainly working on
this). The new working group(s) will have people that participated in this
working group as well as new people who will bring new perspectives to the
process.

Sponsor: The FreeBSD Foundation (in part)

LLDB Debugger Improvements

Links:
Moritz Systems Project Description URL: https://www.moritz.systems/blog/
lldb-freebsd-cpu-target-support-and-userland-debugging-improvements/
Progress Report 1 URL: https://www.moritz.systems/blog/
freebsd-remote-process-plugin-on-non-x86-architectures/
Progress Report 2 URL: https://www.moritz.systems/blog/
freebsd-legacy-process-plugin-removed/
Progress Report 3 URL: https://www.moritz.systems/blog/
lldb-support-for-fork-and-vfork/
Progress Report 4 URL: link:https://www.moritz.systems/blog/
lldb-core-dump-support-improvements/
Git Repository URL: https://github.com/moritz-systems/llvm-project

Contact: Kamil Rytarowski <kamil@moritz.systems>
Contact: Michał Górny <mgorny@moritz.systems>

The LLDB project builds on libraries provided by LLVM and Clang to provide a
great modern debugger. It uses the Clang ASTs and the expression parser, LLVM
JIT, LLVM disassembler, etc so that it provides an experience that “just
works”. It is also blazingly fast and more permissively licensed than GDB, the
GNU Debugger.

FreeBSD includes LLDB in the base system. At present, it has some limitations
in comparison with the GNU GDB debugger, and does not yet provide a complete
replacement. This project spanned from January 2021 to April 2021 and aimed to
improve the compatibility of LLDB with FreeBSD and extend it with new features.

The initial work was focused on finishing the port of the FreeBSD process
plugin to a new client-server model. After porting all previously supported
architectures we were able to remove the legacy plugin. Afterwards, we have
implemented support for tracing fork(2) and vfork(2) syscalls using a model
that is compatible with the follow-fork-mode setting from GDB. Finally, we have
worked on improving support for core dumps in LLDB. We have used the newly
introduced PT_COREDUMP ptrace(2) request to support creating a core dump of a
stopped program without crashing it. During our work, we have fixed many bugs
and improved the state of the test suite on amd64 and arm64 architectures.

The introduced changes are expected to be shipped with LLDB 13.0.

The overall experience of FreeBSD/LLDB developers and advanced users on this
rock solid Operating System reached the state known from other environments.
Furthermore, the FreeBSD-focused work also resulted in generic improvements,
enhancing the LLDB support for Linux and NetBSD.

Sponsor: The FreeBSD Foundation

Performance Monitoring Counters

Contact: Mitchell Horne <mhorne@FreeBSD.org>

This quarter, some improvements were made to the pmc(3) library and hwpmc(4)
kernel module.

Background: Traditionally, PMC event codes have been defined manually in the
sys/dev/hwpmc/pmc_events.h header and compiled statically into the kernel.
These definitions provide a translation between an event’s name and its
encoding and are leveraged by PMC tools like pmcstat(8) via the pmc(3) library.
In 2018, a new source of truth for event definitions was introduced for the
amd64 and i386 architectures, which is the pmu-events tables. These are a set
of json files, maintained by the Linux kernel, which are compiled directly into
libpmc and provide a more complete set of event definitions.

The main result of this work was porting the use of the pmu-events definitions
to arm64. Some minor refactoring to libpmc was performed, making porting to
other platforms in the future easier. A few small bugs were found and fixed
along the way; notably, the "instructions" alias for pmcstat(8) should work
again on Intel CPUs. The ability was added to query the legacy event tables
when pmu-events does not contain the desired event, restoring the ability to
use the pmc.soft(3) events on x86. Finally, event definitions for newer AMD
Zen3 CPUs were imported (credits: Greg V).

An larger update of the PMU event definitions is planned for Q3 of 2021. This
will bring us in sync with 3 years of updates for x86 CPUs and greatly increase
the number of events available on arm64 platforms.

Those who make regular or occasional use of FreeBSD’s PMC tools are encouraged
to test their typical workflow and report any regressions.

Sponsor: The FreeBSD Foundation

syzkaller on FreeBSD

Contact: Mark Johnston <markj@FreeBSD.org> Contact: Simran Kathpalia <
skathpalia3@gmail.com>

See the syzkaller entry in the 2019q1 quarterly report for an introduction to
syzkaller.

Steady progress has been made on fixing bugs found by syzkaller, both by
private instances and by the public syzbot instance. Bugs have been fixed in
TCP, ktrace(2), kqueue(2), POSIX timers, fork(2), crypto(4), and in the VFS and
UFS code. Work has also progressed on adding additional definition to
syzkaller, so unfortunately the size of backlog remains more or less steady.
Simran Kathpalia, a student participating in this year’s Google Summer Of Code
(GSOC) program for FreeBSD, has made excellent progress in extending syzkaller
to cover more of the FreeBSD kernel. She has added definitions for a number of
system calls and has also been working on automating the generation of such
definitions.

Some work has been done towards making it easier to easier to set up syzkaller
instances with minimal effort. For example, much of the work can be automated
using a BastilleBSD template, which uses a jail to simplify configuration.
While this example still requires some manual configuration and is not very
flexible, it is much simpler than setting up an instance manually, and helps
motivate some planned UI improvements and bug fixes for FreeBSD jails.

Future work includes:

• Further effort to try and reduce the size of the syzkaller backlog.

• Completion of some proof-of-concept work to extend syzkaller to fuzz the
Linuxulator subsystem.

• Additional system call and ioctl definitions.

• Automated fuzzing with kernel sanitizers enabled.

Sponsor: The FreeBSD Foundation

Kernel

Updates to kernel subsystems/features, driver support, filesystems, and more.

NXP DPAA driver

Contact: Jakub Klama <jakub@conclusive.pl>

The Conclusive Engineering team is working on support for the NXP Data Path
Acceleration Architecture (DPAA) present in the NXP QorIQ Layerscape ARMv8 SoCs
as well as PowerPC-based QorIQ SoCs.

This work is focused on providing a clean-room, native implementation of DPAA
driver and all of its associated components such as BMan, QMan, FMan and mEMAC
for ARM targets such as LS1046A.

First functional version of the driver with support for the 1Gbps and 10Gbps
MACs should be completed in the 2021 Q3 timeframe.

ENA FreeBSD Driver Update

Links:
ENA README URL: https://github.com/amzn/amzn-drivers/blob/master/kernel/fbsd/
ena/README

Contact: Michal Krawczyk <mk@semihalf.com>
Contact: Artur Rojek <ar@semihalf.com>
Contact: Marcin Wojtas <mw@semihalf.com>

ENA (Elastic Network Adapter) is the smart NIC available in the virtualized
environment of Amazon Web Services (AWS). The ENA driver supports multiple
transmit and receive queues and can handle up to 100 Gb/s of network traffic,
depending on the instance type on which it is used.

Completed since the last update:

• Update ENA driver to v2.4.0

• Update ENA man page

• Restructure the logging system

• Hide sysctl nodes for unused IO queues

• Add support for the large LLQ headers

• Update HAL version

Work in progress:

• Introduce full kernel RSS API support

• Allow reconfiguration of the RSS indirection table and hash key

• Prototype the driver port to the iflib framework

• MFC the ENA v2.4.0 driver to the FreeBSD 11/12/13-STABLE branches

• Support netmap on the c6gn/m6i AWS instance types

• Fix race between detach function and some of the procedural sysctl nodes

• Add new statistics counters

Sponsor: Amazon.com Inc

Graphics Driver Update from Linux 5.7

Links: WIP GitHub Repository URL: https://github.com/neelchauhan/drm-kmod/tree/
5.7-wip

Contact: Neel Chauhan <nc@FreeBSD.org>
Contact: Hans Petter Selasky <hselasky@FreeBSD.org>
Contact: Emmanuel Vadot <manu@FreeBSD.org>
Contact: Mark Linimon <linimon@FreeBSD.org>

We are attempting to update the drm-kmod graphics drivers to the Linux 5.7
code, based on the existing drm-kmod 5.5-wip branch.

Right now, the current version of drm-kmod do not support newer GPU such as
Intel’s Tiger Lake/Iris Xe, used on laptops such as the 2020 HP Spectre x360.
This prevents us from supporting accelerated graphics on newer hardware
containing Intel or AMD GPUs.

Some tasks have already been completed, including:

• amdgpu bring-up

• i915kms console bring-up

Some tasks need to be completed, including:

• i915kms Xorg bring-up (currently pagefaults on remap_io_sg() page address)

• amdgpu VA-API bring-up

• radeonkms bring-up

We welcome help for the incomplete tasks, especially from users wanting to use
newer hardware (or support FreeBSD-as-a-desktop in general).

Intel Wireless driver support

Links:
iwlwifi status FreeBSD wiki page URL: https://wiki.freebsd.org/WiFi/Iwlwifi

The Intel Wireless driver update project aims to bring support for newer
chipsets along with mac80211 LinuxKPI compat code. The dual-licensed Intel
driver code was ported in the past for the iwm(4) native driver and using the
LinuxKPI compat framework allows us to use the driver directly with only very
minor modifications. Multiple updates during development in the last year have
shown that pulling in newer versions can be done in under 1-2 hours.

After a break, part-time work resumed and the last LinuxKPI conflicts got
resolved and most of the other LinuxKPI changes were committed to FreeBSD main.

During the update process firmware crashes were unxpectedly encountered which
got solved and the 802.11 compat code improved. The iwlwifi driver and firmware
got updated from the iwlwifi-next git branch and the linux-firmware repository.

Integration into FreeBSD main is pending for a policy reason. For the latest
state of the development or code, please follow the referenced wiki page or the
freebsd-wireless mailing list.

I would love to thank everyone who has reviewed changes, did initial testing,
helped with drm-kmod, helped in various other ways, answered questions,
encouraged, or patiently waited for any results or running code.

Sponsor: The FreeBSD Foundation

Kernel Sanitizers

Contact: Mark Johnston <markj@FreeBSD.org>

Sanitizers are bug detection facilities which use a combination of
instrumentation inserted by the compiler (LLVM in this case) and runtime state
tracking to detect bugs in C code. They can automatically detect many types of
C programming bugs, such as use-after-frees and uses of uninitialized
variables, which may otherwise require substantial effort to identify. They are
particularly effective in combination with regression testing suites or fuzzing
tools such as syzkaller. Unlike tools such as Valgrind, software must be
recompiled to enable a given sanitizer, but sanitizers can be used in the
kernel. Kernels with sanitizers enabled incur a significant performance
overhead from the runtime, in both CPU utilization and memory usage.

Work has been ongoing to port a pair of kernel sanitizers from NetBSD to
FreeBSD. These are the Kernel Address SANitizer (KASAN) and Kernel Memory
SANitizer (KMSAN). The KASAN port is complete and has been committed to
FreeBSD’s development branch, and a small LLVM patch to enable KASAN and KMSAN
on FreeBSD has also been committed. KMSAN intercepts all memory accesses and
verifies that they are valid using some hidden state saved for each memory
allocation in the kernel; see kasan(9) for further details. It can be enabled
in amd64 kernels by compiling the GENERIC-KASAN kernel configuration. The
FreeBSD regression test suite currently passes with KASAN enabled.

The KMSAN port is currently in progress and is nearing completion. KMSAN
detects uses of uninitialized memory using the algorithms described in the
original MemorySanitizer paper. In particular, it can detect instances of
uninitialized kernel memory being copied out to userspace. Kernel subsystems
may additionally call into the KMSAN runtime to verify the state of a given
buffer. This can be used, for example, to add checks in the network stack for
uninitialized bytes in egress packets. A number of bugs have been found using
KMASN and the FreeBSD regression test suite; many have already been fixed
(search for "KMSAN" in src commit logs for examples), and patches have been
written for all others found so far.

Future work includes: * Finishing the KMSAN port and committing it to the
FreeBSD main branch. * Enabling CI jobs to run the test suite with KASAN and
KMSAN enabled. * Adding syzbot configurations with KASAN and KMSAN enabled, and
fixing bugs found it. * Reducing the scope of memory accesses which cannot be
validated using KASAN or KMSAN today; this consists primarily of making use of
the amd64 direct map optional in various parts of the kernel, and eliminating
uses of UMA_ZONE_NOFREE.

Sponsor: The FreeBSD Foundation

Linux compatibility layer update

Contact: Dmitry Chagin, <dchagin@FreeBSD.org>
Contact: Edward Tomasz Napierala, <trasz@FreeBSD.org>

The goal of this project is to improve FreeBSD’s ability to execute unmodified
Linux binaries. Current support status of specific Linux applications is being
tracked at the Linux app status Wiki page.

The 32-bit syscalls which accept a timespec parameter now have their 64-bit
timespec counterparts: clock_settime64(2), clock_gettime64(2),
clock_nanosleep_time64(2), clock_getres_time64(2), futex_time64(2),
pselect6_time64(2), rt_sigtimedwait_time64(2), utimensat_time64(2).

The O_PATH flag to open(2) system call is now supported, as well as the
AT_EMPTY_PATH flag to fstatat(2) (required for Qt5 applications in Ubuntu
Focal), fchownat(2), linkat(2), and utimensat(2).

The F_GETPIPE_SZ fcntl(2) is now supported.

The ppoll(2) system call was reworked to fix POLLRDHUP semantics.

The COMPAT_LINUX and COMPAT_LINUX32 kernel build options got removed; they were
confusing and not quite functional. This doesn’t affect the usual setups which
use Linuxulator loaded as a kernel module.

The FreeBSD kernel can now generate coredumps for Linux processes, on both
amd64 and arm64. The cores can be analyzed with Linux gdb(1).

There were a number of fixes to ptrace(2) implementation; as a result, the
Linux strace(1) doesn’t get confused with multithreaded programs.

The kernel version was bumped to 4.4.0, as required for new Arch Linux
binaries.

There was a number of fixes to Linuxulator on arm64, ranging from ELF
auxilliary vector (auxv) and Thread-Local Storage fixes to documentation
improvements. There is ongoing work to make Linuxulator on arm64 on par with
the amd64 one. There is also ongoing work on improving the vDSO functionality.

The sysutils/debootstrap port, and its corresponding debootstrap package, now
also work on arm64, not just x86. It is also much faster now. There have been
some further improvements to the startup scripts.

Sponsor: EPSRC

NXP LS1028A SoC support

Contact: Kornel Dulęba <mindal@semihalf.com>
Contact: Lukasz Hajec <lha@semihalf.com>
Contact: Marcin Wojtas <mw@semihalf.com>

The Semihalf team has been working on adding the FreeBSD support for the NXP
LS1028A SoC.

NXP LS1028A SoC is a dual-core 64-bit ARMv8 Cortex-A72 application processor
with high-speed peripherals such as 2 Time-Sensitive Networking-capable (TSN)
Ethernet controllers, quad-port TSN-enabled switch, PCIE, SD/MMC, USB3.0 and
others.

The newly introduced support includes:

• ENETC Ethernet and MDIO controllers drivers

• LS1028A support and other improvements in the FSL SDHCI driver

• LS1028A clockgen driver

• Generic PCI improvements:

□ Add ofw interface support to PCI - commits 28c4e511c23f and 40429103cd0

□ Clean up and add proper support for mapping dts nodes to PCI devices in
pci_host_generic_fdt - commits f0f7b0868a94 and ea52e815887b

TODO:

• Upstream quad-port TSN switch support

Sponsor: Alstom Group

Marvell ARM64 SoCs support

Contact: Zyta Szpak <zr@semihalf.com>
Contact: Kornel Dulęba <mindal@semihalf.com>
Contact: Marcin Wojtas <mw@semihalf.com>

The Semihalf team is working on improving the FreeBSD support for the Marvell
Octeon TX2 CN913x and Armada 7k/8k SoC families.

Marvell Armada 7k8k and Octeon TX2 CN913x SoC families are quad-core 64-bit
ARMv8 Cortex-A72 processors with high speed peripherals including 10 Gb
Ethernet, PCIe 3.0, SATA 3.0 and USB 3.0 for a wide range of networking,
storage, security and industrial applications.

Although the mentioned SoCs are mostly supported in FreeBSD HEAD, some pieces
required improvements.

Applied changes:

• Upstream PCIE improvements

□ PCIE Designware driver (pci_dw) fixes:

☆ Correct setting of outbound I/O ATU window (commit 57dbb3c25936)

☆ Allow mapping ATU windows bigger than 4GB (commit 243000b19f8b)

□ Generic improvements that enable proper user-space mapping and access
of the PCI BARs - commits f2f1ab39c040 and 1c1ead9b94a1

• SDHCI improvements

□ 64-bit DMA operation (commit 7d8700bc291b)

□ sdhci_xenon UHS support (commits 43e31350f8f6-4fa977f854e2)

TODO:

• Improve and merge ICU support rework (D28803)

Sponsor: Marvell

Multicast routing rework

Contact: Wojciech Macek <wma@semihalf.com>

The Semihalf team has been working on reworking the existing ip_mroute module.
The previous implementation was over 20 years old and didn’t use modern kernel
features.

A complete rework of locking mechanism was done to eliminate taking multiple
locks for a single job. Some portions of code were modified to use lockless
constructions, BW-meter API refactored to work in a single context and routing
hot path was identified and made to work efficiently. All these fixes helped
achieve over 5x speed boost in multicast routing.

The newly introduced rework includes:

• mroute: fix race condition during mrouter shutting down race condition

• mrouter: do not loopback packets unconditionally loopback

• ip_mroute: refactor bw_meter API bw_meter

• ip_mroute: rework ip_mroute locking

Sponsor: Stormshield

VFS path descriptors API

Contact: Konstantin Belousov <kib@FreeBSD.org>

Unix VFS API, from a very high point of view, provides two kinds of interfaces.
One is the path-based operations, typically manipulating metadata, like remove,
rename, change permissions and similar. The second is the file-descriptor
operations, typical are read, write, truncate, and other. The well-known open
(2) syscall returns a file descriptor from the path.

The file descriptor is a reliable handle for the file targeted by operations.
Whatever other action was performed on the file meanwhile, rename, even delete,
file descriptor still references the same file, and any operation done on it,
affects that file.

Until recently, there was no similar reliable way to take a handle on the file
path, and then reliably use it for the operations of the first kind. Now, Linux
introduced some facilities that make it possible, and FreeBSD tried to provide
a compatible API.

First, there is a way to get the file descriptor for a path. It is created with
the open(2) syscall, when the O_PATH flag is specified. The returned descriptor
cannot be used for normal file operations, it only designates a place in the
filesystem namespace. Attempts to e.g. read or write using it results in EBADF.
But the system also does not check the access rights on the target file while
opening. It is enough for the process to read the content of the directory with
the file name for open(O_PATH) to succeed.

To actually use such descriptor, there is a new flag AT_EMPTY_PATH, understood
by the \*at(2) set of syscalls. When supplied, and the path argument set to ""
(empty path), \*at(2) syscalls operate on the file designated by the dirfd
argument.

It is not too exciting for e.g. fstatat(2), where already available fstat(2)
operates on any kind of file descriptor. But consider linkat(2), which in
combination with AT_EMPTY_PATH now allows to re-link any opened file descriptor
to a named file (this requires root privilege). Other examples of available
operations are chownat(2), chmodat(2), and others; see man pages. When a file
descriptor and AT_EMPTY_PATH are supplied, the program knows for sure that the
corresponding operation was done on the specific file, not subject to a race
when our file was renamed, and some other file replaced it, providing a
different file on the same path. With O_PATH, we get the same access rules as
for normal chmod, not needing to have read or write permission for the file
content.

A way to translate the path descriptor to a normal file descriptor that can be
read from or written to is to use openat(2) with O_EMPTY_PATH flag. The syscall
checks the current access permissions and grants the requested mode by
returning a normal descriptor.

In Linux, there is no equivalent of the O_EMPTY_PATH flag. It seems, instead,
that its implementation of our equivalent of fdescfs opens the underlying file
on open(2) of /dev/fd/N name. This behavior is incompatible with the existing
FreeBSD fdescfs exposed operation, so it cannot be changed for the default
mount of fdescfs on /dev/fd. A new mount flag "nodup" was added for fdescfs
which emulates Linux for descriptors backed by vnodes. See the fdescfs(5) man
page for details.

The described new Unix VFS API is used by Samba. Adding it to FreeBSD should
allow our system to be still the first-grade platform for Samba, and was
requested by Samba porters.

Sponsor: The FreeBSD Foundation

Dummynet support for pf

Links:
dummynet in-progress tree URL: https://github.com/kprovost/freebsd-src/tree/
netgate/dummynet

Contact: Kristof Provost <kp@FreeBSD.org>

The pf firewall currently relies on ALTQ for traffic shaping. ALTQ is not
enabled in default kernel builds, and is not compatible with all network
drivers (only drivers which implement if_start()).

Work is in progress to add support for dummynet traffic shaping to pf. Dummynet
is already used by ipfw for traffic shaping.

As part of this work, automated tests are being added to dummynet, for both
ipfw and pf. This requires extending dummynet to add vnet support, which was
contributed by Tom Jones <thj@FreeBSD.org>.

While this work is incomplete feedback on architecture and functionality is
welcomed.

TODO:

• Additional test cases

• Debug failure of the IPv6 queue test case

• Fix panic on vnet shutdown if there are still IPv6 packets queued. (These
eventually get sent to ip6_input() with a now freed rcvif pointer. Badness
ensues.)

Sponsor: Rubicon Communications, LLC ("Netgate")

Ethernet support for pf

Links:
pf-link in-progress tree URL: https://github.com/kprovost/freebsd-src/tree/
netgate/pf-link

Work is ongoing to add basic support for Ethernet filtering to pf.

This will allow layer 2 addresses to be used to tag packets for subsequent
filtering or shaping in the existing pf code. The layer 2 code is strictly
stateless.

The intended use case for this is to improve pf’s capabilities in captive
portal setups (i.e. allow/deny internet access based on client MAC addresses).

TODO:

• (optional) anchor support

• move nvlist interface code into libpfctl

• audit nvlist code for bugs (several bugs were found in the recent nvlist
alternatives to existing ioctl calls)

• (optional) VLAN ID filtering

• (optional) MAC address table support

While this work is incomplete feedback on architecture and functionality is
welcomed.

Sponsor: Rubicon Communications, LLC ("Netgate")

pf syncookie support

Links:
pf syncookies in-progress tree URL: https://github.com/kprovost/freebsd-src/
tree/modirum/pf-syncookie-cleanup

Contact: Kristof Provost <kp@FreeBSD.org>

SYN cookies are a mechanism to resist SYN flood denial of service attacks. The
FreeBSD network stack has supported this since 2001. However, this does not
prevent such an attack from flooding the pf state table.

OpenBSD ported this mechanism from FreeBSD into their pf in 2018. This code is
now being ported to FreeBSD’s pf.

The work is still in progress, but the current version demonstrates the basic
capability. Next steps include additional testing, extra automated test cases
and implementing the adaptive mode.

The adaptive mode requires extra care in FreeBSD’s pf, above what was done in
OpenBSD, because of different locking strategies.

Sponsor: Modirum MDPay

Microchip PolarFire SoC support

Contact: Jakub Klama <jakub@conclusive.pl>
Contact: Wojciech Kloska <wojciech@conclusive.pl>

The Conclusive Engineering team is working on support for the Microchip
PolarFire FPGA SoC chip family. PolarFire SoC is based on five 64-bit hard
RISC-V cores, out of which four are equipped with MMU and capable of running
FreeBSD. The SoC also features an FPGA and various peripherals including
Gigabit Ethernet, PCIe and multi-function 12.3Gbps SERDES transceivers.

At the time of writing, the following tasks have been completed:

• Initial platform bring-up with SMP support

• Add support for creating "booti"-style images for RISC-V

• Ethernet support

Work in progress:

• Clock and reset domain support

• USB driver

• PCI Express driver

• I2C driver

• SPI driver

• GPIO driver

• eNVM driver

Racct (Resource Accounting) Bug Fixes and Improvements

Links:
racct PR for runj URL: https://github.com/samuelkarp/runj/pull/11

Contact: Cyril Zhang <cyril@freebsdfoundation.org>

racct is a resource accounting system in the FreeBSD kernel. It keeps track of
resource usage, such as memory use and CPU runtime. Additionally, it provides
an easy interface for limiting the resource usage of entities such as users
and, importantly, jails. For example, it may be of interest to set a limit on
the number of CPUs that a jail can use.

I have been discussing with other FreeBSD developers the prospect of adding
racct support as an experimental feature to an OCI-compliant container runtime,
runj by Samuel Karp. This would mimic Linux’s cgroups functionality in the OCI
specification, which allows Linux containers to have limits on memory, CPU
usage, etc. It also allows us to consider the possibility of adding
FreeBSD-specific configuration to the OCI specification. As part of this work,
I have been improving racct so that it is more functional for use with jails.

Improvements include:

• A new, more accurate scheme for calculating total CPU usage of all
processes in a jail

• Fixing a bug that incorrectly counted the runtime of all child processes in
the parent’s runtime

• Fixing a bug that incorrectly decremented the count of persistent
resources, such as shared memory, when a process exits

• Accounting for POSIX features like shared memory and semaphores, where
previously only SysV features were accounted for

• Adding tests

Feel free to get in touch.

Sponsor: The FreeBSD Foundation

OpenZFS RAIDZ Expansion update

Contact: Matthew Ahrens <matt@mahrens.org>

Links:
OpenZFS Pull Request URL: https://github.com/openzfs/zfs/pull/12225
video from 2021 FreeBSD Developer Summit URL: https://youtu.be/3SUKJye54aI?t=
6166
slides from 2021 FreeBSD Developer Summit URL: https://docs.google.com/
presentation/d/1FeQgEwChrtNQBHfWSNsPK3Y53O5BnPh3Cz5nRa5GAQY/view
news article from Ars Technica URL: https://arstechnica.com/gadgets/2021/06/
raidz-expansion-code-lands-in-openzfs-master/

Project

This feature allows disks to be added one at a time to a RAID-Z group,
expanding its capacity incrementally. This feature is especially useful for
small pools (typically with only one RAID-Z group), where there isn’t
sufficient hardware to add capacity by adding a whole new RAID-Z group
(typically doubling the number of disks).

FreeBSD’s ZFS implementation comes from the OpenZFS project. This work will be
integrated into the OpenZFS repo, with support for FreeBSD and Linux.

Status

The work is functionally complete, and a pull request has been opened.

Remaining Work

Remaining work includes some code cleanup, design documentation, addressing
test failures.

We also need to solicit code reviewers and respond to code review feedback.

Sponsor: The FreeBSD Foundation

Microchip Switchtec support

Contact: Jakub Klama <jakub@conclusive.pl>
Contact: Paweł Eichler <pawel@conclusive.pl>

The Conclusive Engineering team is working on support for the Microchip
Switchtec PCI Express storage switch family.

Switchtec devices are PCI Express Gen3/Gen4/Gen5 packet switches which offer
built-in NTB support for up to 48 NT partitions, extensive diagnostics and
programmable firmware which can be used to implement additional functionality
such as NVMe enclosure management.

At the time of writing, following features are completed:

• NTB support

• Firmware management and diagnostics

Work in progress:

• Event logging

• NVMe enclosure management

This work in co-sponsored by AGILESTORAGE Europe GmbH.

Ports

Changes affecting the Ports Collection, whether sweeping changes that touch
most of the tree, or individual ports themselves.

Emacs Ports

Links:
PR255706 URL: https://bugs.freebsd.org255706
commit bb995aaf6e25e33b028fa4b32321864b48f49055 URL: https://cgit.freebsd.org/
ports/commit/?id=bb995aaf6e25e33b028fa4b32321864b48f49055

Contact: Emacs Ports Team <emacs@FreeBSD.org>

The Emacs development port, editors/emacs-devel, continues to be updated twice
per month to synchronize with the HEAD of upstream’s master branch. A
noteworthy change from the first June 2021 update was the addition of a
NATIVECOMP port option. NATIVECOMP adds support for compiling Emacs lisp to
native code using libgccjit, which was first enabled in lang/gcc11-devel and is
now in lang/gcc11.

For more information about native compilation of Emacs lisp, see Gcc Emacs Wiki
.

FreeBSD Erlang Ecosystem Ports update

Links:
FreeBSD Erlang wiki URL: https://wiki.freebsd.org/Erlang
Erlang/OTP language URL: https://erlang.org/
Elixir language URL: https://elixir-lang.org/
Gleam language URL: https://gleam.run/

Contact: FreeBSD Erlang mailing list <erlang@FreeBSD.org>

The Erlang runtime system, commonly known as the BEAM, provides a runtime that
is used by a number of programming languages and applications in the FreeBSD
ports collection.

Notable changes in 2021 include:

• adding OTP24, including support on amd64 architecture for JIT compilation,
and dropping the previous NATIVE and HIPE options

• security fixes for the devel/rebar3 build tool

• adding a new language runtime for Gleam language

• more than 40 point release updates in the last quarter alone for the Erlang
runtimes

As the upstream Erlang OTP team have committed to supporting the 2 current
releases, more and more point updates are arriving for OTP22-24, but not for
the older Erlang runtime releases, which are now unlikely to get security and
bug fixes.

The Erlang team is planning to:

• deprecate in 2021Q3 any ports that are not compatible with OTP releases in
the last 2 years

• remove the deprecated runtimes in 2021Q4

• remove ports directly dependent on erlang- and elixir- languages, where
they are more commonly installed via mix and rebar3 tools, the standard
community build tool chain.

• update RabbitMQ to the next major release, which requires OTP23 minimum,
and introduces support for the JIT

• bump the main lang/erlang runtime to OTP24 because JIT is awesome

Additional testing and community contributions are welcomed, please reach out
on the mailing list, especially if you are able to help testing of specific
port updates.

GCC in the Ports Tree (lang/gcc*)

Links:
GCC Project URL: https://gcc.gnu.org
GCC 10 release series URL: https://gcc.gnu.org/gcc-10/

Contact: Gerald Pfeifer <gerald@pfeifer.com>

With the great help of linimon@ GCC 10 became the default version of GCC in the
Ports Collection bringing many improvements and fixes.

Looking one step ahead, GCC 11 is now available as a port and even for use as
GCC_DEFAULT via Mk/bsd.default-versions.mk .

Modern GCC ports like this now feature support for powerpcle, and most related
changes also made it it upstream.

On the infrastructure side, USE_GCC now allows for a build time-only
dependency, e.g. USE_GCC=yes:build .

And in case you are developing other ports, USE_GCC=any is a thing of the past
and USE_GCC no longer tries to use the old 4.2-based system compiler (/usr/bin/
gcc) even if present.

Finally, after some two decades of maintaining FreeBSD’s lang/gcc* ports, the
time has come to hand over the baton and largely step back. Please reach out if
you are want to help - we’d hate to simply drop maintainership and would be
happy to collaborate and transition.

KDE on FreeBSD

Links:
KDE FreeBSD URL: https://freebsd.kde.org/
KDE Community FreeBSD URL: https://community.kde.org/FreeBSD

Contact: Adriaan de Groot <kde@FreeBSD.org>

The KDE on FreeBSD project aims to package all of the software produced by the
KDE Community for the FreeBSD ports tree. The software includes a full desktop
environment called KDE Plasma, graphics applications, instant messengers, a
video-editing suite, as well as a tea timer and hundreds of other applications
that can be used on any FreeBSD machine.

The KDE team (kde@) is part of desktop@ and x11@ as well, building the software
stack to make FreeBSD beautiful and usable as a daily-driver graphics-based
desktop machine.

Organisational

The KDE-FreeBSD team has moved its primary communications channel (on IRC) in
with the rest of the FreeBSD world. You can now find us on Libera.Chat, in the
#freebsd-desktop channel.

Why the rename? We’ve been #kde-freebsd since 2003 or so. However, changes in
the IRC Network landscape have caused us to migrate to Libera.Chat and join up
with the rest of the desktop folk — we have a lot of shared concerns, after
all.

Big Picture

• The new KDE Gear release — all the KDE software products that are not
specifically KDE Frameworks or KDE Plasma, which release in a coordinated
fashion on a regular schedule — arrived in ports the day it was released.
Albert Astals Cid has a good blog post explaining the rationale behind KDE
Gear.

• Wayland support has arrived for KDE Plasma. It is now possible to run a KDE
Plasma Wayland environment on FreeBSD machines with a DRM-enabled graphics
card (Intel iGPU or AMDGPU).

• There is slow-but-steady progress in reducing the dependencies of KDE
Applications. Most FreeBSD ports are "batteries included" and that includes
all the build-tools to create the application, but really, an IDE should
not be a dependency of your photo-gallery application.

• X11 was made optional in some of the Qt ports; this opens up possibilities
such as "Wayland-Only" Qt, or "Framebuffer-Only". X11 remains enabled by
default.

• Qt6 is stuck in the "it builds, and parts of it run, but not enough" stage
and has not landed in ports; it will probably wait until after the next
quarterly is cut.

Highlights

• Poppler, CMake, polkit .. these underpinnings of the Free Desktop on
FreeBSD all received their regular updates.

• extproc/kdiff3 had some serious issues with 3-way-merges. This was reported
by Ed Maste, and turned out to have fixes upstream.

• x11/konsole, the terminal emulator, would not accept /bin/sh as a shell;
this is more of an issue on FreeBSD than on Linux where bash is a very
common choice. This was resolved upstream and landed in ports.

• The regular KDE Frameworks releases (monthly) and KDE Plasma releases
(monthly) and KDE Gear all were announced upstream and landed in FreeBSD
ports without a fuss. Upstream CI continues to be updated with the latest
FreeBSD ports so that CI reflects well what the KDE-on-FreeBSD team
delivers to FreeBSD users.

libglvnd on FreeBSD

Links:
libglvnd URL: https://gitlab.freedesktop.org/glvnd/libglvnd

Contact: Kevin Bowling <kbowling@FreeBSD.org>

libglvnd (GL Vendor-Neutral Dispatch) "is a vendor-neutral dispatch layer for
arbitrating OpenGL API calls between multiple vendors. It allows multiple
drivers from different vendors to coexist on the same filesystem, and
determines which vendor to dispatch each API call to at runtime. Both GLX and
EGL are supported, in any combination with OpenGL and OpenGL ES."

Making this the default GL implementation in FreeBSD Ports unblocks a number of
efforts and aligns us with future graphics stacks.

It cleans up the ability to have multiple GL implementations on one system or
image, regardless of what hardware is in use, by eliminating the need for
one-or-the-other hard links for these libraries. This can in turn be used by
Nvidia Optimus setups, or using your fancy PCIe interconnect to have multiple
vendor GPUs for any reason. It will support future Mesa releases where a
concurrent LTS (Long Term Support) branch install can be used to fill in
drivers that will be removed from newer Mesa releases.

Software like KDE and Firefox can use EGL contexts under X11 (with OpenGL) and
Wayland, and in the case of upcoming Firefox releases it may match the speed of
Google Chrome when doing so.

The library uses various implementation details to minimize any performance
impact from indirection, with platform specific support for aarch64, amd64,
armv7, i386, and powerpc64 (ELFv2).

Finally, it helps reduces over-linking with a new libOpenGL to provide OpenGL
without X11 for headless servers or kms or Wayland environments.

If there is any remaining fallout from this change please add kbowling@ to the
PR watchers.

Thank you to Jan Beich <jbeich@FreeBSD.org> for doing the ports infrastructure
work and Matt Turner (Gentoo/freedesktop) for helping me understand the
architecture and how to handle this change in a source-based distribution, as
well as merging a change in libGLU upstream to support building without X11
dependency.

FreeBSD in Science

Links:
Meta-port for Biostar Handbook tools URL: https://www.freshports.org/biology/
biostar-tools/
Simple, Portable Cluster Manager URL: https://acadix.biz/spcm.php
Desktop-installer URL: https://acadix.biz/desktop-installer.php

Contact: Jason Bacon <jwb@FreeBSD.org>
Contact: Yuri Victorovich <yuri@FreeBSD.org>

FreeBSD has long provided a solid foundation for scientific computing, with its
remarkable reliability, network and storage performance, and the FreeBSD ports
system which not only provides an extensive collection of scientific software,
but also facilitates optimized installation by allowing the user to easily
install from source with additional compiler flags.

Last quarter saw the introduction of sysutils/spcm (Simple, Portable Cluster
Manager), an integrated tool set for building and managing HPC (High
Performance Computing) clusters based on FreeBSD.

This quarter we saw rapid growth in the biology category, much of which
culminated in the completion of the biology/biostar-tools metaport. This
metaport installs virtually all of the software needed by bioinformatician in
training. It is now easier for bioinformatics students to install and run the
software they need on FreeBSD than on any other platform. Bioinformatics is the
analysis of biological data such as gene and protein sequences.

We also hit a milestone of 200 ports in the biology category, thanks to
contributions from 21 maintainers. Software installation has always been and
still is a major hurdle for many researchers. Far too often, researchers
attempt "cave man" installations, following primitive instructions from the
developers' site for manually running configure scripts and make. In many other
cases, they struggle with low-quality application containers or third-party on
package managers that are prone to failures and difficult to use. Problems
installing the software they need often cause major delays in their research
and often end in failure. The rapid growth of scientific software in the
FreeBSD ports collection, coupled with the introduction of SPCM and numerous
improvements to sysutils/desktop-installer, have removed many hurdles that
could delay or prevent scientific discovery.

Documentation

Noteworthy changes in the documentation tree, in manpages, or in external books
/documents.

FreeBSD Translations on Weblate

Links:
Translate FreeBSD on Weblate URL: https://wiki.freebsd.org/Doc/Translation/
Weblate
FreeBSD Weblate Instance URL: https://translate-dev.freebsd.org/

Contact: Danilo G. Baio <dbaio@FreeBSD.org>

The Weblate project Documentation (Books and Articles) is open to the public.

The Automatic Translation feature was executed in all languages, re-using
translations from the former project (Docbook).

There are still pending issues about the new translation process, and the
status can be seen on IdeaList#Translation.

Website translations weren’t released through Weblate yet. So we decided to
hold it until we have more details about the working group that will redesign
the Website and the Documentation Portal, to avoid re-work.

FreeBSD Website Revamp - WebApps working group

Contact: Sergio Carlavilla <carlavilla@FreeBSD.org>

Working group in charge of creating the new FreeBSD Documentation Portal; and
redesigning the FreeBSD main Website and its components. FreeBSD developers can
follow and join the working group on the FreeBSD Slack channel #wg-www21. The
work will be divided into four phases:

1. Redesign of the Documentation Portal

Create a new design, responsive and with global search. (Work in progress)

2. Redesign of the Manual Pages on Web

Scripts to generate the HTML pages using mandoc. (Not started)

3. Redesign of the Ports page on Web

Ports scripts to create an applications portal. (Not started)

4. Redesign of the FreeBSD main Website

New design, responsive and dark theme. (Not started)

Miscellaneous

Objects that defy categorization.

Third-Party Projects

Many projects build upon FreeBSD or incorporate components of FreeBSD into
their project. As these projects may be of interest to the broader FreeBSD
community, we sometimes include brief updates submitted by these projects in
our quarterly report. The FreeBSD project makes no representation as to the
accuracy or veracity of any claims in these submissions.

FreshPorts

Links:
FreshPorts URL:https://freshports.org/
FreshPorts blog URL: https://news.freshports.org/

Contact: Dan Langille <dvl@FreeBSD.org>

FreshPorts, and its sister site, FreshSource, have reported upon FreeBSD
commits for 20 years. They cover all commits, but its primary focus is ports.

FreshPorts tracks the commits and extracts data from the port Makefiles to
create a database of information useful to both port developers and port users.

For example, https://www.freshports.org/security/acme.sh/ shows the history of
this port, back to its creation in May 2017.

git

The transition from subversion to git went superbly. Now the work is
concentrating on branches. We are close to incorporating all branches (on src,
doc, and ports) into the website.

The website now queries the main repo every three minutes and pulls in updates.
Commit processing under git is faster and more reliable. Creating the XML
directly from git and not from parsing commit emails has its benefits.

Help wanted

Since the last report, Jethro Nederhof has been doing fantastic work bringing
the website up to HTML5 and into the modern world. Nothing dramatic, from what
the users see, as it has been mostly behind-the-scenes.

We can always use more helpers. The website mostly runs itself and requires
very little on-going maintenance (pkg upgrades, OS patche, etc). Most of the
work is designing new features, fixing bugs, identifying issues, and discussion
with users. Changes to the ports tree usually don’t affect the website much.

Tasks include:

• There is a long list of issues for the website.

• The git_proc_commit project also has a set of issues, mostly in Pyton, some
perhaps related to the website.

• Documentation outlining how the various projects fit together and how they
work is required.

• I have some subversion repos which should be converted to git and uploaded
to GitHub. This would allow people to work on the backend code (commit
ingress and processing).

• The FreshSource website could be updated to modern standards and the repo
converted to git.

Thank you

helloSystem

Links:
Documentation URL: https://hellosystem.github.io/

Contact: Simon Peter <probono@puredarwin.org>
Contact: #helloSystem on irc.libera.chat, mirrored to #helloSystem:matrix.org
on Matrix

What is helloSystem?

helloSystem is FreeBSD preconfigured as a desktop operating system with a focus
on simplicity, elegance, and usability. Its design follows the “Less, but
better” philosophy.

Q2 2021 Status

• Version 0.5.0 of helloSystem has been published. Installable Live ISO
images are available. Download and changelog are at https://github.com/
helloSystem/ISO/releases/tag/r0.5.0

• helloSystem was part of the Desktop panel at the FreeBSD Developer Summit
2021. A video is available at https://www.youtube.com/watch?v=3SUKJye54aI&t
=11410s

• Work has started towards 0.6.0. Thanks for contributed features and
bugfixes

□ Improved spatial file manager

□ Switch to KWin (if its dependencies can be reduced significantly for
use outside of KDE Plasma)

Experimental and release builds of the Live ISO are available at https://
github.com/helloSystem/ISO/releases.

Contributing

Help is wanted in a number of areas, especially in the areas of the FreeBSD
core OS and kernel, and Qt/C++.

Containers & FreeBSD: Pot, Potluck & Potman

Links:
Pot on github URL: https://github.com/pizzamig/pot
Potluck Repository & Project URL: https://potluck.honeyguide.net/
Potluck on github URL: https://github.com/hny-gd/potluck
Potman on github URL: https://github.com/grembo/potman

Contact: Luca Pizzamiglio (Pot) <pizzamig@freebsd.org>
Contact: Stephan Lichtenauer (Potluck) <sl@honeyguide.eu>
Contact: Michael Gmelin (Potman) <grembo@FreeBSD.org>

Pot is a jail management tool that also supports orchestration through Nomad.

In the last quarter, v0.12.0 was released and many new improvements have been
worked on since then. The main new feature under development at the moment is
the possibility to create layered pot images for e.g. quickly adding
applications in a build pipeline to a bigger base image which can then be
deployed via Ansible or Nomad.

Potluck aims to be to FreeBSD and Pot what Dockerhub is to Linux and Docker: A
repository of Pot flavours and complete images for usage with Pot and in many
cases Nomad.

Here, all images have been built for FreeBSD 13.0 for the first time, FreeBSD
11.4 support has been removed and 12.2 images have been upgraded to latest
package versions. Furthermore, new images like a PostgreSQL Patroni cluster
have been added or the Nomad and Consul images have been extended to support
clustering, encryption and a new Vault image.

Also, a PoC has been done which shows that Potluck images can potentially
easily be used with containerd and runj.

Potman aims to simplify building Pot images with Vagrant and VirtualBox based
on the Potluck approach, e.g. as part of a DevOps workflow for software
development including testing and publishing them to a repository.

Here, in the last quarter, usage and documentation has been improved.

Future plans include finishing the layered Pot images, further Potman workflow
features and more new and improved Potluck images.

As always, feedback and patches are welcome.