FreeBSD Quarterly Status Report - Fourth Quarter 2019
Lorenzo Salvadore
salvadore at freebsd.org
Mon Jan 27 20:11:33 UTC 2020
FreeBSD Project Quarterly Status Report - Fourth Quarter 2019
Here is the last quarterly status report for 2019. As you might
remember from last report, we changed our timeline: now we collect
reports the last month of each quarter and we edit and publish the full
document the next month. Thus, we cover here the period October 2019 -
December 2019.
If you thought that the FreeBSD community was less active in the
Christmas' quarter you will be glad to be proven wrong: a quick glance
at the summary will be sufficient to see that much work has been done
in the last months.
Have a nice read!
-- Lorenzo Salvadore
__________________________________________________________________
FreeBSD Team Reports
* FreeBSD Core Team
* FreeBSD Foundation
* FreeBSD Release Engineering Team
* Cluster Administration Team
* Continuous Integration
Projects
* IPSec Extended Sequence Number (ESN) support
* NFS Version 4.2 implementation
* DTS Update
* RockChip Support
* Creating virtual FreeBSD appliances from RE VMDK images
Kernel
* SoC audio framework and RK3399 audio drivers
* FreeBSD on Microsoft HyperV and Azure
* FreeBSD on EC2 ARM64
* ENA FreeBSD Driver Update
Architectures
* PowerPC on Clang
* NXP ARM64 SoC support
Userland Programs
* Linux compatibility layer update
Ports
* Ports Collection
* KDE on FreeBSD
* Java on FreeBSD
* Electron and VSCode
* Bastille
* Universal Packaging Tool (upt)
* Wine on FreeBSD
Third-Party Projects
* sysctlbyname-improved
* pot and the nomad pot driver
* 7 Days Challenge
* NomadBSD
__________________________________________________________________
FreeBSD Team Reports
Entries from the various official and semi-official teams, as found in
the Administration Page.
FreeBSD Core Team
Contact: FreeBSD Core Team <core at FreeBSD.org>
The FreeBSD Core Team is the governing body of FreeBSD.
* Julie Saravanos, the sister of Bruce D. Evans (bde), mailed core
with the sad news that Bruce passed away on 2019-12-18 at the age
of 68 years. Bruce was a deeply respected member of the community,
served on the Core team, and made over 5,000 commits. Bruce's
impact on our culture was so profound that new terminology was
spawned. This is an excerpt of a message from Poul-Henning Kamp to
Julie.
I don't know precisely when I first communicated with Bruce, it was
in the late 1980'ies via "UseNET", but I can say with certainty that
few people have inspired me more, or improved my programming more,
than Bruce he did over the next half of my life.
All large projects invent its own vocabulary and in FreeBSD two of
the neologisms are "Brucification", and "Brucified".
A "brucification" meant receiving a short, courteous note pointing
out a sometimes subtle deficiency, or an overlooked detail in a
source code change. Not necessarily a serious problem, possibly not
even a problem to anybody at all, but nonetheless something which
was wrong and ought to be fixed. It was not uncommon for the
critique to be considerably longer than the change in question.
If one ignored brucifications one ran the risk of being "brucified",
which meant receiving a long and painstakingly detailed list of
every single one of the errors, mistakes, typos, shortcomings, bad
decisions, questionable choices, style transgressions and general
sloppiness of thinking, often expressed with deadpan humor sharpened
to a near-fatal point.
The most frustrating thing was that Bruce would be perfectly
justified and correct. I can only recall one or two cases where I
were able to respond "Sorry Bruce, but you're wrong there..." - and
I confess that on those rare days I felt like I should cut a notch
in my keyboard.
The last email we received from Bruce is a good example of the depth
of knowledge and insight he provided for the project:
https://docs.freebsd.org/cgi/getmsg.cgi?fetch=1163414+0+archive/2019/svn-src-all/20191027.svn-src-all
* The 12.1 release was dedicated to another FreeBSD developer who
passed away in the fourth quarter of 2019, Kurt Lidl. The FreeBSD
Foundation has a memorial page to Kurt.
https://www.freebsdfoundation.org/blog/in-memory-of-kurt-lidl/
We send our condolences to both the families of Bruce and Kurt.
* Core has documented The Project's policy on support tiers.
https://www.freebsd.org/doc/en_US.ISO8859-1/articles/committers-guide/archs.html
* Core approved a source commit bit for James Clarke. Brooks Davis
(brooks) will mentor James and John Baldwin (jhb) will co-mentor.
* The Project's first Season of Docs ended with a negative result.
The work was not completed and contact could not be established
with the writer. No payment was made and the financing was set
aside for future work.
* Google Summer of Code completed. Information about the seven
accepted projects can be found on the wiki page.
https://wiki.freebsd.org/SummerOfCode2019Projects
* Adam Weinberger (admaw) was added to conduct at . Adam has
demonstrated competence, understanding, and fairness in personal
matters.
* Li-Wen Hsu (lwhsu) contacted Core after receiving a report from
concerned local community members about past updates to The
Project's internationalization policy. Lengthy discussions took
place to determine how to reaffirm that The Project maintains a
neutral position in political disputes. Updates were made to the
document and it was decided that any future changes would require
explicit Core approval.
https://www.freebsd.org/internal/i18n.html
* After nomination by Edward Napierała (trasz), core voted to grant
Daniel Ebdrup (debdrup) and Lorenzo Salvadore (salvadore)
membership in The Project. Both Daniel and Lorenzo have been
working on the quarterly reports for the past few quarters.
* The Core-initiated Git Transition Working Group continued to meet
over the last quarter of 2019. Their report is still forthcoming.
__________________________________________________________________
FreeBSD Foundation
Contact: Deb Goodkin <deb at FreeBSDFoundation.org>
The FreeBSD Foundation is a 501(c)(3) non-profit organization dedicated
to supporting and promoting the FreeBSD Project and community
worldwide. Funding comes from individual and corporate donations and is
used to fund and manage software development projects, conferences and
developer summits, and provide travel grants to FreeBSD contributors.
The Foundation purchases and supports hardware to improve and maintain
FreeBSD infrastructure and provides resources to improve security,
quality assurance, and release engineering efforts; publishes marketing
material to promote, educate, and advocate for the FreeBSD Project;
facilitates collaboration between commercial vendors and FreeBSD
developers; and finally, represents the FreeBSD Project in executing
contracts, license agreements, and other legal arrangements that
require a recognized legal entity.
Here are some highlights of what we did to help FreeBSD last quarter:
Partnerships and Commercial User Support
We help facilitate collaboration between commercial users and FreeBSD
developers. We also meet with companies to discuss their needs and
bring that information back to the Project. In Q4, Ed Maste and Deb
Goodkin met with a few commercial users in the US. It's not only
beneficial for the above, but it also helps us understand some of the
applications where FreeBSD is used. We were also able to meet with a
good number of commercial users at the Bay Area Vendor/Developer Summit
and Open Source Summit Europe. These venues provide an excellent
opportunity to meet with commercial and individual users and
contributors to FreeBSD.
Fundraising Efforts
In 2019, we focused on supporting a few key areas where the Project
needed the most help. The first area was software development. Whether
it was contracting FreeBSD developers to work on projects like wifi
support, to providing internal staff to quickly implement hardware
workarounds, we've stepped in to help keep FreeBSD innovative, secure,
and reliable. Software development includes supporting the tools and
infrastructure that make the development process go smoothly, and we're
on it with team members heading up the Continuous Integration efforts,
and actively involved in the clusteradmin and security teams.
Our advocacy efforts focused on recruiting new users and contributors
to the Project. We attended and participated in 38 conferences and
events in 21 countries. From giving FreeBSD presentations and workshops
to staffing tables, we were able to have 1:1 conversations with
thousands of attendees.
Our travels also provided opportunities to talk directly with FreeBSD
commercial and individual users, contributors, and future FreeBSD
users/contributors. We've seen an increase in use and interest in
FreeBSD from all of these organizations and individuals. These meetings
give us a chance to learn more about what organizations need and what
they and other individuals are working on. The information helps inform
the work we should fund.
In 2019, your donations helped us continue our efforts of supporting
critical areas of FreeBSD such as:
* Operating System Improvements: Providing staff to immediately
respond to urgent problems and implement new features and
functionality allowing for the innovation and stability you've come
to rely on.
* Improving and increasing test coverage, continuous integration, and
automated testing with a full-time software engineer to ensure you
receive the highest quality, secure, and reliable operating system.
* Security: Providing engineering resources to bolster the capacity
and responsiveness of the Security team providing you with peace of
mind when security issues arise.
* Growing the number of FreeBSD contributors and users from our
global FreeBSD outreach and advocacy efforts, including expanding
into regions such as China, India, Africa, and Singapore.
* Offering FreeBSD workshops and presentations at more conferences,
meetups, and universities around the world.
* Providing opportunities such as developer and vendor summits and
company visits to help facilitate collaboration between commercial
users and FreeBSD developers, as well as helping to get changes
pushed into the FreeBSD source tree, and creating a bigger and
healthier ecosystem.
We've accomplished a lot this year, but we are still only a small
501(c)3 organization focused on supporting FreeBSD and not a trade
organization like many other open source Foundations.
Please consider making a donation at
https://www.FreeBSDfoundation.org/donate/ to help us continue and
increase our support for FreeBSD.
We also have the Partnership Program, to provide more benefits for our
larger commercial donors. Find out more information at
https://www.FreeBSDfoundation.org/FreeBSD-foundation-partnership-program/
and share with your companies!
OS Improvements
The Foundation supports software development projects to improve
FreeBSD through our full time technical staff, contractors, and project
grant recipients. They maintain and improve critical kernel subsystems,
add new features and functionality, and fix bugs.
Between October and December there were 236 commits to the FreeBSD
source repository tagged with FreeBSD Foundation sponsorship. This is
about 10% of all commits during this period. Some of these projects
have their own entries in the quarterly report, and are not repeated
here, while others are briefly described below.
As usual, Foundation staff member Konstantin Belousov committed a large
number of UFS, NFS, tmpfs, VM system, and low-level Intel x86 bug fixes
and improvements. Kostik also committed improvements to the run-time
linker (rtld), and participated in very many code reviews, helping to
get changes from other developers integrated into the tree.
Following on from his work to improve debugging tools in the
Linuxulator environment, Edward Napierała integrated the Linux Test
Project (LTP) with FreeBSD's CI system, and committed a number of small
bug fixes to the Linuxulator itself.
Mark Johnston continued working on infrastructure for the Syzkaller
system call fuzzing tool, and committed fixes for many issues
identified by it. Mark committed improvements to RISC-V infrastructure,
the network stack, performance and locking, and x86 pmap.
Mark also added support for newer Intel WiFi chipsets to the iwm
driver, enabling WiFi support for the Lenovo X1 Carbon 7th generation,
and other contemporary laptops.
Ed Maste committed a number of improvements and cleanups in build
infrastructure, vt console fixes including issues with keyboard maps,
some blacklistd updates, documentation updates, and other small
changes. Ed also committed some work to prepare for the removal of GCC
4.2.1 from the FreeBSD source tree, currently planned for Q1 2020.
Continuous Integration and Quality Assurance
The Foundation provides a full-time staff member who is working on
improving our automated testing, continuous integration, and overall
quality assurance efforts.
During the fourth quarter of 2019, Foundation staff continued to
improve the project's CI infrastructure, worked with contributors to
fix the failing build and test cases. We worked with other teams in the
project for their testing needs and also worked with many external
projects and companies to improve their support of FreeBSD. We added
several new CI jobs and brought the FreeBSD Hardware Testing Lab
online.
We published 2019 in Review: CI and Testing Advancements on the
Foundation's blog.
See the FreeBSD CI section of this report for completed work items and
detailed information.
Supporting FreeBSD Infrastructure
The Foundation provides hardware and support to improve the FreeBSD
infrastructure. Last quarter, we continued supporting FreeBSD hardware
located around the world.
FreeBSD Advocacy and Education
A large part of our efforts are dedicated to advocating for the
Project. This includes promoting work being done by others with
FreeBSD; producing advocacy literature to teach people about FreeBSD
and help make the path to starting using FreeBSD or contributing to the
Project easier; and attending and helping other FreeBSD contributors
volunteer to run FreeBSD events, staff FreeBSD tables, and give FreeBSD
presentations.
The FreeBSD Foundation sponsors many conferences, events, and summits
around the globe. These events can be BSD-related, open source, or
technology events geared towards underrepresented groups. We support
the FreeBSD-focused events to help provide a venue for sharing
knowledge, to work together on projects, and to facilitate
collaboration between developers and commercial users. This all helps
provide a healthy ecosystem. We support the non-FreeBSD events to
promote and raise awareness of FreeBSD, to increase the use of FreeBSD
in different applications, and to recruit more contributors to the
Project.
Check out some of the advocacy and education work we did last quarter:
* Organized the 2019 Bay Area FreeBSD Vendor and Developers Summit in
Santa Clara, CA
* Presented at COSCON '19 in Shanghai, China
* Represented FreeBSD at All Things Open 2019, in Raleigh, North
Carolina
* Industry Partner Sponsor for LISA '19 in Portland, OR
* Silver Sponsor of OpenZFS in San Francisco, CA
* Gave a technical presentation at School of Mines in Golden, CO
* Presenting and representing FreeBSD at Seagl, in Seattle, WA
* Presented at Open Source Summit Europe in Lyon France
* Committed to sponsoring LinuxConfAu 2020, in Gold Coast, Australia
in addition to holding a FreeBSD Mini-Conf
* Accepted to present at the BSD Dev Room at FOSDEM '20, in Brussels,
Belgium
* Accepted to have a stand at FOSDEM '20, in Brussels, Belgium
* Committed to sponsoring FOSSASIA 2020, in Singapore
* Committed to hold FreeBSD Day at SCALE 18x, in Pasadena, CA
We continued producing FreeBSD advocacy material to help people promote
FreeBSD. Learn more about our efforts in 2019 to advocate for FreeBSD:
https://www.freebsdfoundation.org/blog/2019-in-review-advocacy/
Our Faces of FreeBSD series is back. Check out the latest post: Mahdi
Mokhtari.
https://www.freebsdfoundation.org/blog/faces-of-freebsd-2019-mahdi-mokhtari/
Read more about our conference adventures in the conference recaps and
trip reports in our monthly newsletters:
https://www.freebsdfoundation.org/news-and-events/newsletter/
We help educate the world about FreeBSD by publishing the
professionally produced FreeBSD Journal. As we mentioned previously,
the FreeBSD Journal is now a free publication. Find out more and access
the latest issues at https://www.FreeBSDfoundation.org/journal/.
You can find out more about events we attended and upcoming events at
https://www.FreeBSDfoundation.org/news-and-events/.
We have continued our work with a new website developer to help us
improve our website. Work has begun to make it easier for community
members to find information more easily and to make the site more
efficient.
Legal/FreeBSD IP
The Foundation owns the FreeBSD trademarks, and it is our
responsibility to protect them. We also provide legal support for the
core team to investigate questions that arise.
Go to http://www.FreeBSDfoundation.org to find out how we support
FreeBSD and how we can help you!
__________________________________________________________________
FreeBSD Release Engineering Team
Links
FreeBSD 12.1-RELEASE schedule
URL: https://www.freebsd.org/releases/12.1R/schedule.html
FreeBSD 12.1-RELEASE announcement
URL: https://www.freebsd.org/releases/12.1R/announce.html
FreeBSD development snapshots
URL: https://download.freebsd.org/ftp/snapshots/ISO-IMAGES/
Contact: FreeBSD Release Engineering Team <re at FreeBSD.org>
The FreeBSD Release Engineering Team is responsible for setting and
publishing release schedules for official project releases of FreeBSD,
announcing code freezes and maintaining the respective branches, among
other things.
The FreeBSD Release Engineering Team continued work on the
12.1-RELEASE, which started September 6th. This release cycle was the
first "freeze-less" release from the Subversion repository, and the
test bed for eliminating the requirement of a hard code freeze on
development branches.
The 12.1-RELEASE cycle concluded with the final build beginning
November 4th, preceded by three BETA builds and two RC builds. The RC3
build had been included in the original schedule, but had been decided
to not be required.
Additionally throughout the quarter, several development snapshots
builds were released for the head, stable/12, and stable/11 branches.
Much of this work was sponsored by Rubicon Communications, LLC
(netgate.com) and the FreeBSD Foundation.
__________________________________________________________________
Cluster Administration Team
Links
Cluster Administration Team members
URL: https://www.freebsd.org/administration.html#t-clusteradm
Contact: Cluster Administration Team <clusteradm at FreeBSD.org>
The FreeBSD Cluster Administration Team consists of the people
responsible for administering the machines that the Project relies on
for its distributed work and communications to be synchronised. In this
quarter, the team has worked on the following:
* Upgrade ref11-{amd64,i386}.freebsd.org to 11.3-STABLE r353313
* Ongoing systems administration work:
* Creating accounts for new committers.
* Backups of critical infrastructure.
* Keeping up with security updates in 3rd party software.
Work in progress:
* Review the service jails and service administrators operation.
* South Africa Mirror (JINX) in progress.
* NVME issues on PowerPC64 Power9 blocking dual socket machine from
being used as pkg builder.
* Drive upgrade test for pkg builders (SSDs) courtesy of the FreeBSD
Foundation.
* Boot issues with Aarch64 reference machines.
* New NYI.net sponsored colocation space in Chicago-land area.
* Setup new host for CI staging environment.
* Plan how to add new semi-official pkg mirrors
__________________________________________________________________
Continuous Integration
Links
FreeBSD Jenkins Instance
URL: https://ci.FreeBSD.org
FreeBSD Hardware Testing Lab
URL: https://ci.FreeBSD.org/hwlab
FreeBSD CI artifact archive
URL: https://artifact.ci.FreeBSD.org
FreeBSD CI weekly report
URL: https://hackmd.io/@FreeBSD-CI
freebsd-testing Mailing List
URL: https://lists.FreeBSD.org/mailman/listinfo/freebsd-testing
FreeBSD Jenkins wiki
URL: https://wiki.freebsd.org/Jenkins
Hosted CI wiki
URL: https://wiki.freebsd.org/HostedCI
3rd Party Software CI
URL: https://wiki.freebsd.org/3rdPartySoftwareCI
Tickets related to freebsd-testing@
URL: https://preview.tinyurl.com/y9maauwg
FreeBSD CI Repository
URL: https://github.com/freebsd/freebsd-ci
Contact: Jenkins Admin <jenkins-admin at FreeBSD.org>
Contact: Li-Wen Hsu <lwhsu at FreeBSD.org>
The FreeBSD CI team maintains continuous integration system and related
tasks for the FreeBSD project. The CI system regularly checks the
committed changes can be successfully built, then performs various
tests and analysis of the results. The results from build jobs are
archived in an artifact server, for the further testing and debugging
needs. The CI team members examine the failing builds and unstable
tests, and work with the experts in that area to fix the code or adjust
test infrastructure. The details are of these efforts are available in
the weekly CI reports.
During the fourth quarter of 2019, we worked with the contributors and
developers in the project for their testing needs and also worked with
many external projects and companies to improve their support of
FreeBSD. The FreeBSD Hardware Testing Lab is online in this quarter.
It's still in work in progress stage and we are merging the different
versions and will integrate more tightly to the main CI server. We are
also working on make this work more easierly to be reproduced.
Work in progress:
* Collecting and sorting CI tasks and ideas at
https://hackmd.io/bWCGgdDFTTK_FG0X7J1Vmg
* Setup the CI stage environment and put the experimental jobs on it
* Implementing automatic tests on bare metal hardware
* Adding drm ports building test against -CURRENT
* Testing and merging pull requests at
https://github.com/freebsd/freebsd-ci/pulls
* Planning for running ztest and network stack tests
* Helping more 3rd software get CI on FreeBSD through a hosted CI
solution
* Adding LTP test jobs.
* Adding non-x86 test jobs.
* Adding external toolchin related jobs.
Please see freebsd-testing@ related tickets for more WIP information.
This project was sponsored by The FreeBSD Foundation.
__________________________________________________________________
Projects
Projects that span multiple categories, from the kernel and userspace
to the Ports Collection or external projects.
IPSec Extended Sequence Number (ESN) support
Contact: Patryk Duda <pdk at semihalf.com>
Contact: Marcin Wojtas <mw at semihalf.com>
Extended Sequence Number (ESN) is IPSec extension defined in RFC4303
Section 2.2.1. It makes possible to implement high-speed IPSec
implementations where standard, 32-bit sequence number is not
sufficent. Key feature of the ESN is that only low order 32 bits of
sequence number are transmitted over the wire. High-order 32 bits are
maintained by sender and receiver. Additionally high-order bits are
included in the computation of Integrity Check Value (ICV) field.
Extended Sequence Number support contains following:
* Modification of existing anti-replay algorithm to fulfil ESN
requirements
* Trigger soft lifetime expiration at 80% of UINT32_MAX when ESN is
disabled
* Implement support for including ESN into ICV in cryptosoft engine
in both encrypt and authenticate mode (eg. AES-CBC and SHA256 HMAC)
and combined mode (eg. AES-GCM)
* Implement support for including ESN into ICV in AES-NI engine in
both encrypt and authenticate mode and combined mode
Remaining work:
* Upstream patches of the anti-replay algorithm
* Adjust implementation of crypto part after the reworked Open Crypto
Framework gets stable
This project was sponsored by Stormshield.
__________________________________________________________________
NFS Version 4.2 implementation
Contact: Rick Macklem <rmacklem at freebsd.org>
RFC-7862 describes a new minor revision to the NFS Version 4 protocol.
This project implements this new minor revision.
The NFS Version 4 Minorversion 2 protocol adds several optional
features to NFS, such as support for SEEK_DATA/SEEK_HOLE, file copying
done on the server that avoids data transfer over the wire and support
for posix_fallocate(), posix_fadvise(). Hopefully these features can
improve performance for certain applications.
This project has basically been completed. The code changes have now
all been committed to head/current and should be released in FreeBSD
13.
Testing by others would be appreciated. To do testing, an up to date
head/current system is required. Client mounts need the
"minorversion=2" mount option to enable this protocol. The NFS server
will have NFSv4.2 enabled by default.
__________________________________________________________________
DTS Update
Contact: Emmanuel Vadot <manu at FreeBSD.org>
DTS files (Device Tree Sources) were updated to be on par with Linux
5.4 for HEAD and 5.2 for the 12-STABLE branch. The DTS for the RISC-V
architecture are now imported as well.
__________________________________________________________________
RockChip Support
Contact: <freebsd-arm at FreeBSD.Org>
Contact: Emmanuel Vadot <manu at FreeBSD.Org>
Contact: Michal Meloun <mmel at FreeBSD.Org>
RockChip RK3399 now has USB3 support, some configuration such as device
mode are still not supported however host mode should work on any
board.
Support for SPI has been committed which enables ability to interact
with SPI flash if present.
All regulators for the RK808 PMIC (Power Management IC) have been
added.
All clocks are now supported which completes clock and reset
implementation, previously only clocks from devices with drivers were
supported.
The TS-ADC (Temperature Sensor ADC) is now supported, this adds the
ability to read temperature of the CPU and GPU via sysctl
hw.temperature .
Initial PCIe support has been committed and verified working on several
different boards. Known working devices are NVMe devices and PCIe cards
that doesn't utilize PCIe switching or bridge functionality.
Card Detection for SDCard on RK3328 and RK3399 is now supported. There
is still some problems if the board is using a GPIO for CD instead of
the internal detection mechanism.
__________________________________________________________________
Creating virtual FreeBSD appliances from RE VMDK images
Links
freebsd-mkova
URL: https://github.com/gonzoua/freebsd-mkova
Contact: Oleksandr Tymoshenko <gonzo at FreeBSD.org>
OVA is a file format for packaging and distributing virtual appliances:
pre-configured virtual machine images. Virtual appliance file contains
full VM information like the number of CPUs, amount of memory, list of
virtual devices, it also includes disk images. Applications like
VirtualBox or VMWare can import OVA files; this process can be easily
automated.
freebsd-mkova is a CLI tool to create OVA files using VMDK images
provided by FreeBSD RE. For now, only a limited set of attributes can
be specified: VM name, number of CPU, amount of memory, and disk size.
The tool also does only cursory sanity checks on the VMDK file format,
assuming it's a monolithic sparse file and that it has to be converted
to the stream-optimized format. The script can be extended to make
hardware configuration more flexible and VMDK parser more robust.
__________________________________________________________________
Kernel
Updates to kernel subsystems/features, driver support, filesystems, and
more.
SoC audio framework and RK3399 audio drivers
Links
rk3399_audio
URL: https://github.com/gonzoua/freebsd/tree/rk3399_audio
Contact: Oleksandr Tymoshenko <gonzo at FreeBSD.org>
Most modern SoCs and devboards have audio support in one form or
another, but it's one of the areas that are overlooked by FreeBSD
driver developers. The most common architecture for the audio pipeline
on a single-board computer consists of two DAIs (digital audio
interfaces): CPU and codec, connected by a serial bus.
CPU DAI is a SoC IP block that operates with samples: obtains them from
the driver for playback or provides them to the driver for recording
through FIFOs or DMA requests. Audio samples leave (or arrive at) the
SoC through a serial bus, usually I2S, that is connected to Codec DAI.
Codec DAI is an external (to the SoC) chip that packs one or more
DAC/ADC blocks along with mixers, amplifiers, and probably more
specialized devices like filters and/or sound effects. The analog part
of the codec is connected to microphones/headphones/speakers. On SBCs,
the codec usually communicates with SoC through two interfaces: data
path, over which audio samples travel, and a control interface that is
used to read/write chip registers and configure its behavior. The most
common choices for these are I2S and I2C buses, respectively.
For FDT-enabled devices, an audio pipeline is described as a virtual
DTB node that has links to the CPU and codec device(s), and which
specifies the data format, and clock details that both the CPU and the
codec chips would use. It also may have more than one CPU/codec pair.
Using Firefly-RK3399 as a test device, I was able to implement I2S
driver for RK3399 SoC (PIO mode, playback only), the driver for
Realtek's RT5640 chip (headphones playback only + mixer controls) and a
base outline of SoC audio framework. Some bits of rk_i2s and the
framework were ported from the NetBSD code developed by Jared McNeill.
On my WIP branch, I can play mp3 audio and control playback volume.
The primary missing functionalities at the moment are recording
support, multi-link audio cards, DMA support. The most critical among
these is DMA support. In the current implementation, all buffer
management is placed at the ausoc layer, which is not going to work for
DMA, because only the CPU DAI driver would know about the memory
constraints and access mechanisms. The current state of RK3399 support
does not allow to implement DMA transfers for rk_i2s easily, but I plan
to look into this right after adding recording support, which should
not be a lot of work.
__________________________________________________________________
FreeBSD on Microsoft HyperV and Azure
Links
FreeBSD on MicrosoftAzure wiki
URL: https://wiki.freebsd.org/MicrosoftAzure
FreeBSD on Microsoft HyperV
URL: https://wiki.freebsd.org/HyperV
Contact: FreeBSD Integration Services Team <bsdic at microsoft.com>
Contact: Wei Hu <whu at FreeBSD.org>
Contact: Li-Wen Hsu <lwhsu at FreeBSD.org>
Wei is working on HyperV Socket support for FreeBSD. HyperV Socket
provides a way for host and guest to communicate using common socket
interfaces without networking support. Some features in Azure require
HyperV Socket support in guest.
It is planned to commit the code by the end of February.
This project is sponsored by Microsoft. Details of HyperV Socket is
available at
https://docs.microsoft.com/en-us/virtualization/hyper-v-on-windows/user-guide/make-integration-service
Li-Wen and Wei are working on improving FreeBSD release on Azure.
During this quarter, Wei has published the 11.3-RELEASE on Azure.
Li-Wen is working on the FreeBSD release codes related to Azure for the
-CURRENT and 12-STABLE branches.
This project is sponsored by Microsoft and FreeBSD Foundation.
__________________________________________________________________
FreeBSD on EC2 ARM64
Links
FreeBSD/ARM 12 in AWS Marketplace
URL: https://aws.amazon.com/marketplace/pp/B081NF7BY7
FreeBSD/EC2 Patreon
URL: https://www.patreon.com/cperciva
M6G vs M5 buildworld cost/time performance
URL: https://twitter.com/cperciva/status/1206688489518985216
Contact: Colin Percival <cperciva at FreeBSD.org>
In November 2018, Amazon Web Services announced the first Elastic
Compute Cloud (EC2) instances built around the ARM64 platform. While
FreeBSD supported the ARM64 platform, running on this specific virtual
machines took some additional work, but by April 2019 the weekly
snapshot builds performed by the Release Engineering Team included
ARM64 AMIs for FreeBSD HEAD.
In November 2019 FreeBSD 12.1 was released, including the first
"RELEASE" FreeBSD EC2/ARM64 AMIs. A few weeks later, FreeBSD/ARM64 was
added as a new "product" to the AWS Marketplace.
At the re:Invent 2019 conference in December 2019, Amazon announced a
second family of ARM64 instances, known variously as "Graviton 2" and
"M6G". These are far more powerful than the first-generation ARM64 EC2
instances, and have a roughly 40% price/performance advantage over the
"M5" family of x86 EC2 instances; and existing FreeBSD 12.1 and HEAD
AMIs run "out of the box" on these instances.
Work is currently underway to improve kernel locking scalability on
these instances; with high levels of parallelism (e.g. buildworld -j64)
the G6M instances have approximately 1.5x higher sys:user ratios than
equally-sized M5 instances, suggesting that there is room for
improvement here.
Two issues have been recently identified, both likely relating to ACPI:
* EC2 "StopInstance" API calls, which translate to ACPI "power
button" notifications, do not trigger FreeBSD to shut down; this
results in a timeout from EC2 and a "hard poweroff".
* Hotplugging/unplugging EBS volumes, which normally operates via
ACPI device notifications, does not work.
Help from developers familiar with ARM64 and ACPI would be much
appreciated.
This project was sponsored by FreeBSD/EC2 Patreon.
__________________________________________________________________
ENA FreeBSD Driver Update
Links
ENA README
URL: https://github.com/amzn/amzn-drivers/blob/master/kernel/fbsd/ena/README
Contact: Michal Krawczyk <mk at semihalf.com>
Contact: Maciej Bielski <mba at semihalf.com>
Contact: Marcin Wojtas <mw at semihalf.com>
ENA (Elastic Network Adapter) is the smart NIC available in the
virtualized environment of Amazon Web Services (AWS). The ENA driver
supports multiple transmit and receive queues and can handle up to 100
Gb/s of network traffic, depending on the instance type on which it is
used.
Completed since the last update:
* Upstream of the driver v2.1.0 version, introducing:
* Netmap support
* Driver structure rework (split datapath code from initialization)
* Fix for keep-alive timeout due to prolonged reset
* Enable LLQ mode on arm64 instances by enabling memory mapped as WC
Work in progress::
* ENA v2.2.0 release, introducing new bug fixes, features and other
improvements
This project was sponsored by Amazon.com Inc.
__________________________________________________________________
Architectures
Updating platform-specific features and bringing in support for new
hardware platforms.
PowerPC on Clang
Contact: Justin Hibbits <jhibbits at freebsd.org>
Contact: Brandon Bergren <bdragon at freebsd.org>
Contact: Alfredo Dal'Ava Júnior <alfredo.junior at eldorado.org.br>
Shortly before the end of the year all 3 PowerPC targets (powerpc,
powerpc64, powerpcspe) switched to Clang as the base compiler. This was
an effort spanning nearly the full year, with several people involved.
32-bit PowerPC platforms (powerpc, powerpcspe) still require GNU ld,
but powerpc64 uses LLD as the base linker. The other two platforms will
migrate as soon as LLD is ready, which should be in the next several
months.
With the switch to Clang and LLD, powerpc64 also switched to ELFv2, a
modern ABI initially targeted for Linux powerpc64le (little endian),
but the ABI itself is endian agnostic; however, ELFv2 is binary
incompatible with ELFv1. FreeBSD is still big endian on all powerpc
targets.
__________________________________________________________________
NXP ARM64 SoC support
Contact: Marcin Wojtas <mw at semihalf.com>
Contact: Artur Rojek <ar at semihalf.com>
The Semihalf team initiated working on FreeBSD support for the NXP
LS1046A SoC
LS1046A are quad-core 64-bit ARMv8 Cortex-A72 processors with
integrated packet processing acceleration and high speed peripherals
including 10 Gb Ethernet, PCIe 3.0, SATA 3.0 and USB 3.0 for a wide
range of networking, storage, security and industrial applications.
Completed since the last update:
* QSPI
* Network performance improvements
Todo:
* Upstreaming of developed features. This work is expected to be
submitted/merged to HEAD in the Q1 of 2020.
This project was sponsored by Alstom Group.
__________________________________________________________________
Userland Programs
Changes affecting the base system and programs in it.
Linux compatibility layer update
Contact: Edward Tomasz Napierala <trasz at FreeBSD.org>
Linux binaries of Linux Test Projects tests are now part of the FreeBSD
Continuous Integration infrastructure. This makes it easy to track
progress in improving the Linux compatibility layer, and to detect
regressions.
There was a fair number of all kinds of improvements to the layer,
ranging from updated linux(4) man page, to a new linux rc script, which
now takes care of eg mounting Linux-specific filesystems or setting ELF
fallback brand, to new syscalls, to tiny improvements such as making ^T
work for Linux binaries.
From the user point of view, when running 13-CURRENT, Linux jails are
now in a mostly working state: you can SSH into a jail with CentOS 8
binaries, run screen(1), Emacs, Postgres, OpenJDK 11, use yum
upgrade... Of course there's still a bunch of things that need work:
* There is a patch from chuck@ that makes core dumps work for Linux
binaries; this will make debugging much easier
* There are pending reviews for patches that add extended attributes
support, fexecve(2) syscall, sendfile; they require wrapping up and
committing
* There are over 400 failing LTP tests. Some of them are false
positives, some are easy to fix bugs, some require adding new
system calls. Any help is welcome.
This project was sponsored by FreeBSD Foundation.
__________________________________________________________________
Ports
Changes affecting the Ports Collection, whether sweeping changes that
touch most of the tree, or individual ports themselves.
Ports Collection
Links
About FreeBSD Ports
URL: https://www.FreeBSD.org/ports/
Contributing to Ports
URL: https://www.freebsd.org/doc/en_US.ISO8859-1/articles/contributing/ports-contributing.html
FreeBSD Ports Monitoring
URL: http://portsmon.freebsd.org/index.html
Ports Management Team
URL: https://www.freebsd.org/portmgr/index.html
Contact: René Ladan <portmgr-secretary at FreeBSD.org>
Contact: FreeBSD Ports Management Team <portmgr at FreeBSD.org>
The Ports Management Team is responsible for overseeing the overall
direction of the Ports Tree, building packages, and personnel matters.
This entry shows what happened in the last quarter.
2019Q4 closed with a total of 38,200 ports and 2180 open PRs of which a
small 470 PRs are unassigned. Last quarter saw 7907 commits from 157
committers to the HEAD branch and 358 commits from 61 committers to the
2019Q4 branch. This seems to suggest a small increase in activity
compared to the quarter before.
During the last quarter, we welcomed Oleksii "Alex" Samorukov (samm@)
and Scott Long (scottl@, already a source committer) as new ports
committers. We also said goodbye to az@, brd@, dtekse@, eadler@, and
johans at .
The default versions of some ports changed: Lazarus is now at version
2.0.6, Samba at 4.10, and Python at 3.7. The web browsers received
their updates too: Chromium is now at version 78.0.3904.108, Firefox at
version 72.0 and its ESR counterpart at version 68.4.0. Finally, the Qt
stack got updated to version 5.13.2.
Some modernizations took place: the "palm" category was removed as well
as the virtual "ipv6" category. IPv6 support (next to IPv4) is now
considered the norm. Lastly, the CentOS 6 ports were removed after
their CentOS 7 counterparts were made the default in the previous
quarter.
As always, antoine@ was happy to take your exp-runs, this time a total
of 30, for various ports and framework updates, default version
updates, and the removal of OpenJDK 6 and OpenJRE 6.
__________________________________________________________________
KDE on FreeBSD
Links
KDE FreeBSD
URL: https://freebsd.kde.org/
KDE Community FreeBSD
URL: https://community.kde.org/FreeBSD
Contact: Adriaan de Groot <kde at FreeBSD.org>
The KDE on FreeBSD project packages the software produced by the KDE
Community for FreeBSD. The software includes a full desktop
environment, the art application https://kdenlive.org and hundreds of
other applications that can be used on any FreeBSD desktop machine.
The monthly releases of KDE Frameworks, bugfix-releases of KDE Plasma
Desktop and the quarterly feature release of KDE Plasma Desktop were
all landed in the ports tree shortly after upstream releases. There
were also monthly KDE Applications bugfix-releases which also landed in
a timely manner.
Digikam landed a new release thanks to Dima Panov. We hope this gets
rid of the instability caused by the previous release update from last
quarter.
The open bugs list grew to 32 this quarter with a handful of strange
build failures. We welcome detailed bug reports and patches. KDE
packaging updates are prepared in a copy of the ports repository on
GitHub and then merged in SVN. We welcome pull requests there as well.
__________________________________________________________________
Java on FreeBSD
Links
OpenJDK 11 repository at FreeBSD GitHub
URL: https://github.com/freebsd/openjdk-jdk11u
Contact: Greg Lewis <glewis at FreeBSD.org>
During Q4 the FreeBSD java porting effort features smaller updates than
those of the previous quarters. However, the following changes are
worth mentioning:
* Updated ports for OpenJDK 8u232, 11.0.5, and 13.0.1
* Removal of the EOL'ed Java 6, 9, and 10 ports
* Fixed remote debugging for Java 11+
* Fixed a problem with running external processes for Java 11+
This project was sponsored by FreeBSD Foundation.
__________________________________________________________________
Electron and VSCode
Links
Electron port
URL: https://github.com/tagattie/FreeBSD-Electron
VSCode port
URL: https://github.com/tagattie/FreeBSD-VSCode
Contact: Hiroki Tagato <tagattie at yandex.com>
Contact: Luca Pizzamiglio <pizzamig at FreeBSD.org>
Electron is a popular framework to build desktop application using
JavaScript, HTML and CSS. Few months ago, electronjs has been added to
the ports tree. Currently version 4.x and 6.x are supported.
In the last quarter, a popular application, the powerful VSCode editor,
has been added to the ports tree as well. VSCode is based on electron
6.x
atom, another popular editor, is still a work in progress and it's
based on electron 4.x
Many thanks to Hiroki, for the hard work, and to Antoine, for support
of the special poudriere configuration needed to build VSCode.
__________________________________________________________________
Bastille
Links
Bastille GitHub
URL: https://github.com/BastilleBSD/bastille
Bastille Templates
URL: https://gitlab.com/bastillebsd-templates
Bastille Website
URL: https://bastillebsd.org
Contact: Christer Edwards <christer.edwards at gmail.com>
What is Bastille?
Bastille is an open-source system for automating deployment and
management of containerized applications on FreeBSD.
Bastille uses FreeBSD jails as a container platform and adds template
automation to create a Docker-like collection of containerized
software. The template collection currently validates 30-40
applications from the ports tree, and is growing!
Templates take care of installing, configuring, enabling, and starting
the software, providing an automated way of building containerized
stacks.
Bastille is available in ports at sysutils/bastille.
Q4 2019 Status
In Q4 2019 Bastille published three releases (for a total of ten
releases in 2019). Highlights from these updates include:
* support for "thin" (shared base) and "thick" (unique base) jails
* support for INCLUDE and FSTAB in template system
* upgrade support for shared and unique base jails
* GitLab CI/CD testing for all official templates
* automatic template validation and CVE scan
* dedicated pf table for private IP jails
Bastille saw an increase in community contributions with six new GitHub
contributors. These people generously improved error checking, release
validation (sha256), firewall functionality, flexible networking and
initial support for resource limits!
We want to thank everyone that contributed to Bastille in 2019. Your
support has been amazing!
__________________________________________________________________
Universal Packaging Tool (upt)
Links
Upt repositories
URL: https://framagit.org/upt/
Upt itself
URL: https://framagit.org/upt/upt/
The FreeBSD backend
URL: https://framagit.org/upt/upt-freebsd
Contact: The upt mailing list <upt at framalistes.org>
Contact: <#upt-packaging>
The Universal Package Manager (upt) is a tool designed to easily port
software from common upstream package archives (such as
https://rubygems.org/) to various operating systems, including FreeBSD,
of course.
A lot of similar tools already exist: pytoport (which creates FreeBSD
ports for PyPI packages), gem2deb (which creates Debian packages from a
Ruby gem), and many others.
The main difference between these tools and upt is that the latter uses
a modular design, allowing it to handle packages from many sources and
support many different operating systems through plugins. You may try
upt by installing sysutils/py-upt, sysutils/py-upt-pypi and
sysutils/py-upt-freebsd. Suppose you would like to package "upt-cran",
which is hosted on PyPI. You could do it like so:
# upt package -f pypi -b freebsd -o /usr/ports/sysutils/ upt-cran
$ tree /usr/ports/sysutils/py-upt-cran /usr/ports/sysutils/py-upt-cran
|-- Makefile |-- distinfo `-- pkg-descr
$ cat sysutils/py-upt-cran/Makefile # $FreeBSD:
head/en_US.ISO8859-1/htdocs/news/status/report-2019-10-2019-12.xml
53827 2020-01-26 13:49:47Z trasz $
PORTNAME= upt-cran DISTVERSION= 0.1 CATEGORIES= sysutils python
MASTER_SITES= CHEESESHOP PKGNAMEPREFIX= ${PYTHON_PKGNAMEPREFIX}
MAINTAINER= python at FreeBSD.org COMMENT= CRAN frontend for upt
LICENSE= BSD3CLAUSE LICENSE_FILE= ${WRKSRC}/XXX
RUN_DEPENDS= ${PYTHON_PKGNAMEPREFIX}lxml>0:devel/py-lxml@${PY_FLAVOR} \
${PYTHON_PKGNAMEPREFIX}requests>0:www/py-requests@${PY_FLAVOR} \
${PYTHON_PKGNAMEPREFIX}upt>0:sysutils/py-upt@${PY_FLAVOR} TEST_DEPENDS=
${PYTHON_PKGNAMEPREFIX}requests-mock>0:www/py-requests-mock@${PY_FLAVOR
}
USES= python USE_PYTHON= autoplist distutils
.include <bsd.port.mk>
Note that the Rubygems and CPAN frontends are also available
(sysutils/py-upt-rubygems and sysutils/py-upt-cpan).
Bug reports and comments about this new tool are welcome.
__________________________________________________________________
Wine on FreeBSD
Links
Wine homepage
URL: https://www.winehq.org
Contact: Gerald Pfeifer <gerald at FreeBSD.org>
A lot has happened since our last quarterly report. The Wine 4 release
series has been in our tree for nearly a year and proven rather stable.
Both that port and wine-devel, which tracks bi-weekly development
releases, have seen regular adjustments to infrastructure changes and
small improvements, in particular also around non-default options.
Now we need help!
WoW64 (or Wine on Wine 64) allows running both 32-bit and 64-bit
Windows applications in one installation. A volunteer has proposed
* a general framework for lib32- companion libraries
https://reviews.freebsd.org/D16830
* an approach directly using our Wine ports
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=242625
to make this work and we do not have the expertise nor facilities to
properly review, test, and maintain those ourselves.
If you can facilitate getting (at least one of) these into the tree,
please help! And if you'd like to assume co-maintainership or sole
maintainership of emulators/wine*, that is an option, too.
__________________________________________________________________
Third-Party Projects
Many projects build upon FreeBSD or incorporate components of FreeBSD
into their project. As these projects may be of interest to the broader
FreeBSD community, we sometimes include brief updates submitted by
these projects in our quarterly report. The FreeBSD project makes no
representation as to the accuracy or veracity of any claims in these
submissions.
sysctlbyname-improved
Links
gitlab.com/alfix/sysctlbyname-improved
URL: https://gitlab.com/alfix/sysctlbyname-improved
Contact: Alfonso Sabato Siciliano <alfonso.siciliano at email.com>
The FreeBSD kernel maintains a Management Information Base (MIB) where
a component (object) represents a parameter of the system. The sysctl()
system call explores the MIB to find an object by its Object Identifier
(OID) and calls its handler to get or set the value of the parameter.
The sysctlbyname() syscall (or the old function) accepts the name of
the object (instead of its OID) to identify it. The purpose of this
project is to allow sysctlbyname() to handle:
* a CTLTYPE_NODE with a no-NULL handler, example
"kern.proc.pid.\<pid\>";
* an object with some level-name equals to the '\0' character,
example "security.jail.param.allow.mount.";
A sysctlbyname() clone is provided: sysctlbyname_improved(), the
implementation core is a new sysctl internal node to get the OID of a
node by its name eventually expanded with an input for its handler;
both, can be installed via _sysutils/sysctlbyname-improved-kmod_. The
internal node is also used by the sysctlmif_oidinputbyname() function
of the _devel/libsysctlmibinfo2_ userland library and can be handled by
the SYSCTLINFO_BYNAME macro of the sysctlinfo interface (described in
the previous quarterly status report).
__________________________________________________________________
pot and the nomad pot driver
Links
Nomad pot driver
URL: https://github.com/trivago/nomad-pot-driver
Pot project
URL: https://github.com/pizzamig/pot
minipot
URL: https://github.com/pizzamig/minipot
Contact: Luca Pizzamiglio <pizzamig at FreeBSD.org>
Contact: Esteban Barrios <esteban.barrios at trivago.com>
The pot utility added support to private bridges: a group of jail can
now use a dedicated bridge, instead of the public one, improving
isolation. Moreover, several small bugs have been found and fixed, and
support to pre/post start/stop hook script has been added.
The nomad pot driver received support for nomad restart without drain
and improved configuration stability.
A new port called minipot has been added: this port will install
configuration files and dependencies, converting a FreeBSD machine in a
single node cluster. It will install nomad, consul, pot, the nomad pot
driver and traefik, already configured and ready to use.
Experimental work has been done on a tool that allows to create and run
pot images (FreeBSD jails) on other operating systems (Linux and Mac),
adopting an approach similar to docker machine. We hope to make this
tool available soon.
Next steps:
* add dual IP stack support to pot
* add private bridge support to the nomad pot driver
* improve usability to create images
This project was sponsored by trivago N.V..
__________________________________________________________________
7 Days Challenge
Links
7 Days Challenge
URL: https://wiki.freebsd.org/MichaelCrilly/7dayschallenge
Contact: Michael Crilly <mike at opsfactory.com.au>
The 7 Days Challenge is an educational initiative to help people
onboard with FreeBSD more easily.
It will use a combination of tutorials, guides and how-tos to get users
engaged with FreeBSD quickly, target specific end goals the user might
have for FreeBSD, and more.
The primary objective is to demonstrate FreeBSD's capabilities as a
modern, relevant operating system in today's Cloud centric, automated
business models.
This project was sponsored by OpsFactory Pty Ltd (Australia).
__________________________________________________________________
NomadBSD
Links
NomadBSD Website
URL: https://www.nomadbsd.org/
NomadBSD Github
URL: https://www.github.com/NomadBSD/NomadBSD
NomadBSD Developer Mailing List
URL: https://www.freelists.org/list/nomadbsddevs
Contact: NomadBSD Team <info at NomadBSD.org>
NomadBSD is a persistent live system for USB flash drives, based on
FreeBSD. Together with automatic hardware detection and setup, it is
configured to be used as a desktop system that works out of the box,
but can also be used for data recovery, for educational purposes, or
testing FreeBSD's hardware compatibility.
After one release candidate the NomadBSD Team finished and released
NomadBSD 1.3 on December 7th. This release is based on FreeBSD 12.1,
fixed a lot of bugs and added new packages and features. Along those
features are the option to install NomadBSD on ZFS and the use of an
automatic configuration when running NomadBSD in VirtualBox.
New tools developed by the NomadBSD Team and added to version 1.3 are
nomadbsd-dmconfig to select a display manager theme, nomadbsd-adduser
which adds new user accounts and DSBBg to change the background image.
All these are using the Qt-Toolkit.
In Q4 we added two mirrors in France and Germany and would like to
thank nosheep.fr and fau.de for them.
We are looking for people to help the project. Help is much appreciated
in all areas:
* Translation of program interfaces
* Design artwork
* Programming new tools, extend existing ones
* Tests and Bug reports / UX and feature suggestions
* Mirrors outside of Europe
Open tasks:
* Support installation on disk partitions and add a partition editor
GUI.
* Complete disk encryption
* Add a user-friendly network manager
More information about the freebsd-hackers
mailing list