[FreeBSD-Announce] FreeBSD Quarterly Status Report - Third Quarter 2016
bjk at FreeBSD.org
Mon Nov 14 01:22:17 UTC 2016
-----BEGIN PGP SIGNED MESSAGE-----
FreeBSD Project Quarterly Status Report - 3rd Quarter 2016
As focused as we are on the present and what is happening now, it is
sometimes useful to take a fresh look at where we have come from, and
where we are going. This quarter, we had our newest doc committer
working to trace through the tangled history of many utilities, and we
also get a glimpse looking forward at what may come in FreeBSD 12.
Though 11.0-RELEASE was not finalized until after the period covered in
this report, we can still have some anticipatory excitement for the
features that will be coming in 12.0. The possibilities are
tantalizing: a base system with no GPL components, arm64 as a Tier-1
architecture, capsicum protection for common utilities, and the
CloudABI for custom software are just a few.
The work of the present is no less exciting, with 11.0 making its way
out just after the end of Q3, the new core coming into its own, and
much more that you'll have to read and find out.
Please submit status reports for the fourth quarter of 2016 by January 7.
FreeBSD Team Reports
* FreeBSD Release Engineering Team
* Ports Collection
* The FreeBSD Core Team
* The FreeBSD Foundation
* Capsicum Update
* ClonOS: New FreeBSD-Based Free/Open Hosting Platform
* CloudABI: Running Untrusted Programs Directly on top of FreeBSD
* Improvements to Non-Transparent Bridge Subsystem
* The Graphics Stack on FreeBSD
* Using lld, the LLVM Linker, to Link FreeBSD
* VirtualBox Shared Folders Filesystem
* ZFS Code Sync with Latest OpenZFS/Illumos
* evdev Support
* FreeBSD Driver for the Annapurna Labs ENA
* FreeBSD on Hyper-V and Azure
* Timekeeping Code Improvements
Google Summer of Code
* Google Summer of Code 2016
* Non-BSM to BSM Conversion Tools
* ptnet Driver and bhyve Device Model
* FreeBSD on Annapurna Labs Alpine
* FreeBSD on Marvell Armada38x
* UEFI Runtime Services
* KDE on FreeBSD
* LXQt on FreeBSD
* Xfce on FreeBSD
* Documenting the History of Utilities in /bin and /sbin
FreeBSD Team Reports
FreeBSD Release Engineering Team
FreeBSD 11.0-RELEASE schedule
Contact: FreeBSD Release Engineering Team <re at FreeBSD.org>
The FreeBSD Release Engineering Team is responsible for setting and
publishing release schedules for official project releases of FreeBSD,
announcing code freezes, and maintaining the respective branches, among
The FreeBSD Release Engineering Team continued the 11.0-RELEASE cycle
which was planned to be released in September, but as a result of
several last-minute issues, the final release announcement was delayed.
This project was sponsored by The FreeBSD Foundation.
FreeBSD Ports Website
How to Contribute
Ports Monitoring Website
Ports Management Team Website
Ports Management Team on Twitter
Ports Management Team on Facebook
Ports Management Team on Google+
Contact: René Ladan <portmgr-secretary at FreeBSD.org>
Contact: FreeBSD Ports Management Team <portmgr at FreeBSD.org>
The Ports Tree currently contains over 26,300 ports, with the PR count
around 2,150. Of these PRs, 516 are unassigned. The last quarter saw
5,295 commits by 117 active committers. Compared to the preceding
quarter, there is both a slight increase in the number of PRs and the
number of unassigned PRs, and a slight decrease in the number of
In the last quarter, four commits bits were taken in for safe keeping:
erwin, miwi, and sem left by their own request and jase was inactive
for more than 18 months. We welcomed two new committers: Tobias Berner
(tcberner) and Joseph Mingrone (jrm).
On the management side, erwin and miwi left portmgr. bapt also left
portmgr but is still the liaison for core.
On the infrastructure side, three new USES (grantlee, kde, linux) and
one new Keyword (javavm) were introduced. The default version of the
Linux ports is now CentOS 6, with the Fedora 10 ports scheduled for
removal at the end of the year. The license framework has been extended
with a NONE license to indicate that a port has no clearly defined
licensing terms. For those ports, no packages or distribution files are
distributed. Also, support for the complete set of Creative Commons
licenses has been added.
Some major user-visible ports were updated: Firefox to 49.0 and Firefox
Extended Service Release to 45.4.0; Chromium to 52.0.2743.116; the
default version of gcc to 4.8.5; and pkg itself to 1.8.7.
Behind the scenes, antoine ran 24 exp-runs to validate various package
updates, framework changes, and changes to the base system. bdrewery
added two new package building machines, supervised the package builds
for 11.0-RELEASE, and added support for building arm64 packages.
At EuroBSDcon, rene visited a presentation by Landry Breuil
<landry at openbsd.org> explaining how packages are built in the OpenBSD
world and explaining various design decisions.
1. If you have some spare time, please take up a PR for testing and
The FreeBSD Core Team
Contact: FreeBSD Core Team <core at FreeBSD.org>
The third quarter started with the handover to the ninth Core team as
it took office. With four members returning from the previous core
(Baptiste Daroussin, Ed Maste, George Neville-Neil and Hiroki Sato),
one returning member after a term away (John Baldwin), and four members
new to core (Allan Jude, Kris Moore, Benedict Reuschling and Benno
Rice), the new core team represents just about the ideal balance
between experience and fresh blood.
Beyond handing over all of the ongoing business, reviewing everything
on Core's agenda, and other routine changeover activities, the first
action of the new core was to respond to a query from Craig Rodrigues
concerning how hardware supplied to the project through donations to
the FreeBSD Foundation was being used.
The Foundation does keep records of what hardware has been supplied
over time and has some idea of the original purpose that hardware was
provisioned for, but does not track the current usage of the project's
hardware assets. Cluster administration keeps their own configuration
database, but this is not suitable for general publication and covers
much more than Foundation supplied equipment. After some discussion it
was decided that updated information about the current disposition of
Foundation supplied equipment should be incorporated in the
Foundation's annual report.
Ensuring that all of the FreeBSD code base is supplied under open and
unencumbered licensing terms and that we do not infringe on patent
terms or otherwise act counter to any legal requirements are some of
Core's primary concerns. During this quarter, there were three items of
* Importing Concurrency Kit. In consultation with the Foundation's
legal counsel, it was determined that importing selected parts of
concurrency kit is acceptable, and has been approved.
* The proposal to create a shadow GPLv3 toolchain repository was put
to the community. Ultimately the whole idea has been rendered
largely redundant by faster than anticipated progress on the
external toolchain ports and packages for those architectures where
LLVM is not yet sufficiently mature.
* Concerns were raised about handling GPL code in work in progress on
the linuxkpi shim. This issue is not related to the FreeBSD svn
repository but Core would like to stress that care must be taken to
avoid license infringement and plans to write a set of guidelines
for handling GPL code.
The item that has absorbed the largest portion of Core's attention this
quarter concerns the project's handling of security vulnerabilities in
bspatch(1), libarchive(3), freebsd-update(8) and portsnap(8). A partial
fix was applied in FreeBSD-SA-16:25.bspatch but this lacks fixes to
libarchive code that were not yet available from upstream.
SecTeam receives privileged early reports of many vulnerabilities and
consequently has a strict policy of not commenting publicly until an
advisory and patches have been published. Early access to information
about vulnerabilities is contingent on their ability to avoid premature
disclosure, and without such, they could not have security advisories
and patches ready to go immediately when a vulnerability is published.
However, in this case, vulnerabilities were already public and the lack
of any official response from the FreeBSD Project was leading to
concern amongst users and some critical press coverage. Core stepped in
and published a statement clarifying the situation and the particular
difficulties involved in securely modifying the mechanisms used to
deliver security patches. Core believes that prompt notification and
discussion of the implications and possible workarounds to any public
vulnerability should not wait on the availability of formal OS patches.
The OpenSSH project has deprecated DSA keys upstream. FreeBSD had kept
DSA keys enabled in the later 10.x releases for compatibility reasons,
but with the release of 11.0 the time has come to synchronize again
with upstream. Since there are numerous DSA keys in use in the FreeBSD
cluster, this necessitated an exercise to get replacement keys
installed. Core would like to thank David Wolfskill and the accounts
team for handling the surge in key changes with a great deal of aplomb.
During this quarter we welcomed Michael Zhilin, Imre Vadasz, Steve
Kiernan and Toomas Soome as new source committers. Over the same
period, we said farewell to Martin Wilke and Erwin Lansing who have
handed in their commit bits. We wish them well in their future
endeavours and hope to see them return as soon as they can.
The FreeBSD Foundation
FreeBSD Foundation Website
Contact: Deb Goodkin <deb at FreeBSDFoundation.org>
The FreeBSD Foundation is a 501(c)(3) non-profit organization dedicated
to supporting and promoting the FreeBSD Project and community
worldwide. Funding comes from individual and corporate donations and is
used to fund and manage software development projects, conferences and
developer summits, and provide travel grants to FreeBSD contributors.
The Foundation purchases hardware to improve and maintain FreeBSD
infrastructure and publishes FreeBSD white papers and marketing
material to promote, educate, and advocate for the FreeBSD Project. The
Foundation also represents the FreeBSD Project in executing contracts,
license agreements, and other legal arrangements that require a
recognized legal entity.
Here are some highlights of what we did to help FreeBSD last quarter:
Our work is 100% funded by your donations. Our spending budget for 2016
is $1,250,000 and we have raised $271,500 so far. Our Q1-Q3 financial
reports will be posted in early November. As you can see, we need your
donations to continue supporting FreeBSD at our current level. Please
consider making a donation at https://www.FreeBSDFoundation.org/donate/ .
The Foundation improves FreeBSD by funding software development
projects approved through our proposal submission process, and our
internal software developer staff members. Two Foundation-funded
projects continued last quarter: one project is to port NetBSD's
blacklistd daemon and related elements to FreeBSD, and the second is
phase two of the FreeBSD/arm64 port.
Foundation staff members were responsible for many changes over the
quarter. Kostik Belousov accomplished this work last quarter:
* Provided kernel support for EFI Runtime Services calls
* Implemented gettimeofday(2) purely in userspace for HPET timers
* Implemented fdatasync(2)
* Improved the locking of the time keeping code
* Made the sleepqueue code immune to rapid callout changes
* Made many stability fixes, the most important of which were UFS
issues and an i386 bug
* Improved the process management and ptrace code
Ed Maste, our Project Development Director, accomplished this work last
* Worked on FreeBSD/arm64 issues and Cavium ThunderX deployment
* Worked with upstream developers to test works in progress and
prepare LLD as the replacement linker in the FreeBSD base system
* Switched to using LLVM's libunwind in the base system
* Improved the reproducibility of builds in the FreeBSD base system
* Reviewed the blacklistd work that is in progress
* Attended BSDCam 2016, with a primary focus on toolchain discussions
* Participated in ongoing Capsicum calls, and helped with the
Capsicumization of several base system utilities
* Fixed a number of ELF Tool Chain issues, and integrated a new
upstream version into the FreeBSD base system
* Hosted biweekly graphics calls to coordinate work in progress by
funded and volunteer developers
* Implemented fixes for security issues in some FreeBSD update tools,
and coordinated their integration into the stable and release
George Neville-Neil continued hosting a bi-weekly Transport conference
call (notes at https://wiki.FreeBSD.org/TransportProtocols) and the
bi-weekly DTrace conference call (notes at
Foundation staff member Glen Barber worked with the Release Engineering
team to continue finalizing the 11.0-RELEASE cycle, which was delayed
to address several last-minute issues.
As part of the Cluster Administration team, Glen worked with the
amazing on-site staff at NYI to rack and install two Cavium ThunderX
machines, one of which is used for native package builds for the
FreeBSD/arm64 architecture, and the other of which is targeted to be
used as a reference machine in the FreeBSD infrastructure.
Getting Started with FreeBSD Project
We hired a summer intern, with no experience on FreeBSD, Linux, or any
command-line operating system, to figure out on his own how to install
and use FreeBSD. He wrote easy-to-follow how-to guides to help make the
new-user experience straightforward and positive. He submitted bug
reports and reported issues through the appropriate channels, and
worked with Glen Barber and Brad Davis to improve the new user
information on FreeBSD.org to make it easier for new people to get
started with FreeBSD. You can find his how-to guides at
https://www.FreeBSDFoundation.org/FreeBSD/how-to-guides/ and check out
his interview on BSDNow at
Supporting FreeBSD Infrastructure
We provide hardware and support for FreeBSD infrastructure. Last
quarter we purchased and brought up two 48-core Cavium ThunderX
machines to build FreeBSD package sets for the arm64 platform. We also
purchased more servers to help with continuous integration efforts.
FreeBSD Advocacy and Education
A large part of our efforts are dedicated to advocating for the
Project. This includes promoting work being done by others using
FreeBSD, producing advocacy literature to teach people about FreeBSD
and ease the path to starting out with FreeBSD, contributing to the
Project, and attending and getting other FreeBSD contributors to
volunteer to run FreeBSD events, staff FreeBSD tables, and give FreeBSD
We created new handouts to promote TeachBSD.org
(https://www.FreeBSDFoundation.org/wp-content/uploads/2016/08/TeachBSD_half_final.pdf) and the Google Summer of Code program
We published the July/August issue of the FreeBSD Journal:
We also published monthly newsletters to highlight work being done to
support FreeBSD, tell you about upcoming events, and provide other
information to keep you in the loop of what we are doing to support the
FreeBSD Project and community:
Conferences and Events
The FreeBSD Foundation sponsors many conferences, events, and summits
around the globe. These events can be BSD-related, open source, or
technology events geared towards underrepresented groups.
We support the FreeBSD-focused events to help provide a venue for
sharing knowledge, to work together on projects, and facilitate
collaboration between developers and commercial users. This all helps
provide a healthy ecosystem. We support the non-FreeBSD events to
promote and raise awareness about FreeBSD, to increase the use of
FreeBSD in different applications, and to recruit more contributors to
This quarter, we sponsored and/or attended the following events:
* Texas Linux Fest, July 8-9, 2016, Austin, TX
* The Eleventh HOPE, July 22-24, 2016, New York, NY
* BSDCam 2016, August 15-17, 2016, Cambridge, UK (sponsor, organizer,
and participated) (https://wiki.FreeBSD.org/DevSummit/201608)
* FOSSCON 2016, August 20, 2016, Philadelphia, PA
* womENcourage 2016, September 12-13, 2016, Linz, Austria (Silver
* SNIA Storage Developer Conference 2016, September 19-22, 2016,
Santa Clara, CA (Industry Partner Sponsor)
* EuroBSDcon 2016 and FreeBSD Developer Summit, September 22-25,
2016, Belgrade, Serbia (Silver Sponsor)
Our EuroBSDcon involvement included:
+ Held a Women in Tech BoF in partnership with ACM-W Europe
+ Benedict organized the EuroBSDcon Developer Summit
+ Deb gave a Foundation Update talk and Hiroki Sato and Benedict
Reuschling joined her for a Q&A session.
+ Kirk McKusick taught his two-day FreeBSD tutorial
+ George Neville-Neil taught a tutorial on Tracing FreeBSD for
DevOps and Developers
+ George also gave the Keynote talk, titled The Coming Decades
+ Phillip Paeps was one of the primary organizers for this
* OpenZFS Developer Summit 2016, September 26-27, 2016, San
Francisco, CA (Silver)
+ Justin Gibbs gave a talk on Fault Management
We sponsored three FreeBSD contributors to attend EuroBSDcon.
The Foundation owns the FreeBSD trademarks, and it is our
responsibility to protect them. We continued to review requests and
grant permission to use the trademarks.
FreeBSD Community Engagement
Anne Dickison, our Marketing Director, has been overseeing the efforts
to rewrite the Project's Code of Conduct to help make this a safe,
inclusive, and welcoming community.
Other Stuff We Did
We welcomed Kylie Liang and Philip Paeps to the Board of Directors.
More information and interviews can be found at:
George attended the ARM Partner Meeting in Cambridge.
Capsicum Wiki Page
Contact: Allan Jude <allanjude at FreeBSD.org>
Contact: Baptiste Daroussin <bapt at FreeBSD.org>
Contact: Conrad Meyer <cem at FreeBSD.org>
Contact: Ed Maste <emaste at FreeBSD.org>
Contact: Mariusz Zaborski <oshogbo at FreeBSD.org>
Several developers have undertaken a recent effort to sandbox
additional applications in the base system. This work is proceeding
nicely and one of the goals is to target basic utilities used in
security sensitive applications, like freebsd-update and portsnap.
This work higlighted two longstanding challenges in applying Capsicum.
First, there are a number of common constructs shared by many simple
programs, such as limiting capability rights on the stdio file
descriptors. To address this, a set of capsicum helper routines has
been added for these common cases.
Second, a common challenge occurs where applications need to open an
arbitrarily large number of files, possibly from various directories,
where preopening the file descriptors may not be suitable. Several
possible solutions for this are in discussion.
Recently Capsicumized utilities include:
Additional Capsicum changes are in review:
* b64decode, b64encode, uudecode, uuencode
An additional syscall (getdtablesize) and additional sysctls
(kern.proc.nfds, kern.hostname, etc.) are now permitted in capability
Capability rights are now propagated to child descriptors on accept(2).
Capsicum is now enabled in the 32-bit compatibility syscall layer.
Per-process (procctl) and global (sysctl) settings have been added to
aid in debugging while Capsicumizing existing applications. When
enabled, instead of returning ENOTCAPABLE or ECAPMODE for a system
call, the kernel will issue a SIGTRAP to generate a core dump or enter
This project was sponsored by Dell EMC Isilon, ScaleEngine Inc., and
The FreeBSD Foundation.
ClonOS: New FreeBSD-Based Free/Open Hosting Platform
Contact: Oleg Ginzburg <olevole at olevole.ru>
Currently, FreeBSD is well proven as a base for routers (pfSense,
OPNSense, BSDRP) and NAS (FreeNAS, zfsGuru, NAS4Free). However,
FreeBSD-based solutions are almost completely absent in the
virtualization area, and ClonOS is one of the attempts to change that.
ClonOS is a new free open-source FreeBSD-based platform for virtual
environment creation and management. In the core platform are:
* FreeBSD as the host OS
* CBSD (as a management tool)
* puppet (for configuration management)
* additional features such as go-micro services (obtaining VMs,
resizing disks, and so on)
1. We would like to see ClonOS in real-world use. In this regard we
are interested in finding more people and companies that use
FreeBSD in hosting tasks. In addition, it could be great to work
with the developers of existing NAS solutions (zfsGuru, NAS4Free).
CloudABI: Running Untrusted Programs Directly on top of FreeBSD
Official CloudABI Website
Using CloudABI on FreeBSD
Python for CloudABI
CloudABI on GitHub
Contact: Ed Schouten <ed at FreeBSD.org>
Contact: The CloudABI mailing list <cloudabi-devel at googlegroups.com>
CloudABI is a compact UNIX-like runtime environment inspired by
FreeBSD's Capsicum security framework. It allows you to safely run
potentially untrusted programs directly on top of FreeBSD, Linux and
macOS, without requiring the use of virtualisation, jails, etc. This
makes it a useful building block for cluster/cloud computing.
Over the last couple of months, several new libraries and applications
have been ported over to CloudABI, the most important addition being
Python 3.6. This means that you can now write strongly sandboxed apps
Support for different hardware platforms has also improved. In addition
to amd64 and arm64, we now support i686 and armv6. The release of LLVM
3.9 was important to us, as it has integrated all the necessary changes
to support the first three platforms. Full armv6 support is still
blocked on some issues with LLVM's linker, LLD.
This project was sponsored by Nuxi, the Netherlands.
1. Play around with CloudABI and let us know what you think of it!
Full support for amd64 and arm64 is part of FreeBSD 11.0. i686 and
armv6 support is only available on head, but will be merged to the
stable/11 branch in the future.
2. Interested in Python programming? Give our copy of Python a try and
share your experiences!
3. Do you maintain pieces of software that could benefit from strong
sandboxing? Try building them using the CloudABI cross compiler!
Improvements to Non-Transparent Bridge Subsystem
Contact: Alexander Motin <mav at FreeBSD.org>
Non-Transparent Bridges allow the creation of memory windows between
different systems using the regular PCIe links of CPUs as a transport.
During the last quarter, the NTB subsystem gained a significant set of
improvements and fixes:
* The code was modularized, utilizing FreeBSD's NewBus interfaces to
allow support for different hardware types with different drivers,
support for multiple NTB instances in a system, using the
ntb_transport module for consumers other than if_ntb, etc.
* Support for splitting NTB resources between different applications
was added, such as doing direct access to some range of remote
memory and to a virtual network interface between nodes at the same
* The virtual network interface driver gained support for many modern
features, such as multiple queues, new locking, etc.
* NTB performance and SMP scalability was improved.
* Multiple workarounds for hardware issues were added.
The code is committed to the FreeBSD head, stable/11 and stable/10
This project was sponsored by iXsystems, Inc.
1. Support for the next generation of Intel hardware.
2. Support for non-Intel hardware (AMD, PLX, etc.).
3. Support for I/OAT and other DMA offloads.
4. Creating a more efficient packet transport protocol.
5. Creating a greater variety of NTB applications.
The Graphics Stack on FreeBSD
Graphics Stack Roadmap and Supported Hardware Matrix
Ports Development Repository
DRM 4.7 Development Repository
GSoC 2016: Link /dev Entries to Sysctl Nodes
GSoC 2016: Redesign libdevq
Graphics Team Blog
Contact: FreeBSD Graphics Team <freebsd-x11 at FreeBSD.org>
Contact: Matthew Macy <mmacy at nextbsd.org>
Contact: Johannes Lundberg <yohanesu75 at me.com>
We are sad to report that both GSoc projects failed. The libdevq
project was abandoned as the student disappeared. The kernel project
was incomplete because the student could not work for personal reasons.
He plans to resume work and complete the task, even though GSoC 2016 is
X.org server version 1.18.4 and updates for the xf86-video-ati and
xf86-video-intel DDX drivers are ready for wider testing. A CFT will be
sent out shortly. These updates are required to use newer DRM versions.
The missing functionality from libdrm that is needed by the amdgpu
driver has been added. These changes will be committed to the ports
tree shortly after the xorg-server update.
DRM from Linux 4.8 was ported to the drm-next branch. This branch
should be used for radeon and amdgpu cards. The drm-next-4.7 branch
should be used for i915 cards due to instabilities in the intel driver
in the drm-next branch.
Johannes Lundberg has been working on getting the Wayland environment
running on FreeBSD. The Wayland ports are in a working state except for
the Weston compositor.
The current Weston port (from DragonFlyBSD) might be scrapped and a new
port created from scratch based on the upstream source code. With the
use of libinput, libudev-devd, and epoll-shim, the diff will not be
very large and will be easier to maintain.
Patches for wlc (another Wayland compositor) are being pushed upstream.
On the TODO list is refactoring the tty code into selectable backends
(linux, FreeBSD, etc), as recommended by the author of wlc. For now, it
is running on FreeBSD with patches in the ports tree.
Using lld, the LLVM Linker, to Link FreeBSD
LLD Wiki Page
Contact: Ed Maste <emaste at FreeBSD.org>
lld is the linker in the LLVM family of projects. It is a
high-performance linker that supports the ELF, COFF, and Mach-O object
formats. Where possible, lld maintains command-line and functional
compatibility with the existing GNU BFD ld and gold linkers. However,
the authors of lld are not constrained by strict compatibility where it
would hamper performance or desired functionality.
Compared to the GNU ld 2.17.50 currently in the base system, lld will
* AArch64 (arm64) support
* Link Time Optimization (LTO)
* New ABI support
* Other linker optimizations
* Much faster link times
* Maintained code base
The upstream lld project has now implemented nearly all of the
functionality required to link the amd64 FreeBSD base system, including
the kernel. The boot loader components and rescue utilities currently
do not build with lld.
This project was sponsored by The FreeBSD Foundation.
1. Merge lld to FreeBSD head as part of the Clang 3.9.0 import.
2. Request a ports exp-run with lld installed as /usr/bin/ld.
3. Fix building the boot loader with lld.
4. Fix building rescue with lld.
5. Test and iterate making lld fixes for additional architectures.
VirtualBox Shared Folders Filesystem
Contact: Li-Wen Hsu <lwhsu at FreeBSD.org>
Contact: Oleksandr Tymoshenko <gonzo at FreeBSD.org>
FreeBSD provides an API for guest operating systems to access shared
folders on the host so that the kernel driver can expose them to the
guest's userland. This project aims to add such functionality to the
VirtualBox Guest Additions driver.
Good progress was made over the last few months. Developers were able
to mount a filesystem in read-only mode and, with some limitations, in
read-write mode. The implementation still lacks some critical pieces,
but the roadmap is clear.
1. Finish the missing pieces.
2. Implement proper locking.
3. General clean-up and bugfixes.
ZFS Code Sync with Latest OpenZFS/Illumos
Contact: Alexander Motin <mav at FreeBSD.org>
Contact: Andriy Gapon <avg at FreeBSD.org>
The ZFS code base in FreeBSD regularly gets merges of new code, staying
in sync with the latest OpenZFS/Illumos sources. Among other things,
the latest merge included the following improvements:
* The ARC now mostly stores compressed data, the same as is stored on
disk, decompressing them on demand.
* The L2ARC now stores the same (compressed) data as the ARC without
recompression, and its RAM usage was further reduced.
* The largest size of indirect block possible has been increased from
16KB to 128KB, and speculative prefetching of indirect blocks is
* Improved ordering of space allocation.
* The SHA-512t256 and Skein hashing algorithms are now supported.
evdev WIP Repository
Original evdev Proposal
Contact: Vladimir Kondratiev <wulf at cicgroup.ru>
Contact: Oleksandr Tymoshenko <gonzo at FreeBSD.org>
evdev is a portable, API-compatible implementation of the Linux
/dev/input/eventX interface. It covers a wide variety of input devices
like keyboards, mice, and touchscreens (with multitouch support), and
support for it is implemented in a lot of existing userland components
like Qt, libinput, and tslib.
evdev support was started by Jakub Klama as a Google SoC 2014 project,
and later picked up and finished by Vladimir Kondratiev. General API
and evdev support bits for ukbd and ums were committed to head. Support
was also added for TI's AM33xx touchstreen controller (the popular
BeagleBone is based on the AM33xx) and the official touschreen for the
Raspberry Pi. Multitouch support for the Raspberry Pi was successfully
demonstarted using the latest Qt development branch.
1. Documentation. In particular, manual pages are needed for the KPI.
2. Support additional hardware.
3. Enable evdev support in existing ports, and add new evdev-dependent
FreeBSD Driver for the Annapurna Labs ENA
Amazon AWS Documentation of the ENA
Contact: Jan Medala <jan at semihalf.com>
Contact: Jakub Palider <jpa at semihalf.com>
The Elastic Network Adapter (ENA) is a 25G SmartNIC developed by
Annapurna Labs based on a custom ARMv8 chip. This is a high-performance
networking card that is available to AWS virtual machines. It
introduces enhancements in network utilization scalability on EC2
machines running various operating systems, in particular FreeBSD.
The goal of FreeBSD enablement is to provide top performance and a wide
range of monitoring and management features such as:
* multiple queue modes
* various offload functionality
* admin queue
* asynchronous notification
* robust hardware access
* scalable number of MSI-X interrupts
The current state offers stable driver operation with good performance
on machines running FreeBSD directly on the hardware.
This project was sponsored by Annapurna Labs -- an Amazon company.
1. Optimize performance for virtualized environments.
2. Prepare for submitting the driver as a Phabricator review.
FreeBSD on Hyper-V and Azure
FreeBSD Virtual Machines on Microsoft Hyper-V
Supported Linux and FreeBSD virtual machines for Hyper-V on Windows
Contact: Sepherosa Ziehau <sepherosa at gmail.com>
Contact: Hongjiang Zhang <honzhan at microsoft.com>
Contact: Dexuan Cui <decui at microsoft.com>
Contact: Kylie Liang <kyliel at microsoft.com>
This quarter, the Hyper-V storage driver was greatly improved: its
performance was increased by a factor of 1.2-2 by applying BUS_DMA and
UNMAP_IO, enlarging the request queue, and selecting the outgoing
channel with the LUN considered; TRIM/UNMAP was enabled; and some
critical bugs (PRs 209443, 211000, 212998) were fixed so that disk hot
add/remove and VHDX online resizing should work now.
The VMBus driver also received attention, with enhancements made for
the handling of device hot add/remove.
In the Hyper-V network driver, configurable RSS key and dynamic MTU
change are now supported.
FreeBSD images on Azure continue to be updated -- after publishing the
FreeBSD 10.3 VM image on the global Microsoft Azure in June, Microsoft
also published the VM image on the Microsoft Azure operated by 21Vianet
in China in September.
Patches have been developed to support PCIe pass-through (also known as
Discrete Device Assignment); this feature allows physical PCIe devices
to be passed through to FreeBSD VMs running on Hyper-V (Windows Server
2016), giving them near-native performance with low CPU utilization.
The patch to enable the feature will be posted for review soon.
This project was sponsored by Microsoft.
Timekeeping Code Improvements
Contact: Konstantin Belousov <kib at FreeBSD.org>
Work was done to properly lock the time-keeping code. The existing code
correctly interoperated with readers, both kernel- and user-space,
giving them lock-less access to the actual data ('timehands') needed to
derive the time of day from the timecounter hardware in the presence of
updaters. But updates of the timehands, which are performed by periodic
clock interrupts, the ntpd-driven sys_ntp_adjtime(2) syscall, the
settimeofday(2) syscall, pps sync, and possibly other sources, were not
coordinated. Moreso, the NTP code was locked by Giant, which did not
really serve any purpose.
As a result of the work, locking was applied to ensure that any
timehands adjustments are performed by a single mutator. An interesting
case is the parallel modification of the timehands from the top of the
kernel, for instance the settimeoday(2) syscall, and a simultaneous
clock tick event, where the syscall has already acquired the resources.
In this case, it is highly desirable to not block (spin) in the tick
handler, and the required adjustments are performed by the already
executing call from the top half. There, the typical trylock operation
is desired, which was surprisingly lacking in our spinlock
implementation. mtx_trylock_spin(9) was implemented and is used for
The userspace gettimeofday(2) implementation was enhanced to allow
syscall-less operation on machines that use HPET hardwire for
timecounters. The HPET algorithm coexists with older RDTSC-based code,
allowing dynamic switching of timecounters. HPET registers a page that
is mmap(2)-ed readonly by libc into userland application programs'
address space as needed. Measurements demonstrated modest improvements
in gettimeofday(2) performance, but, not unexpectedly, even the
syscall-less HPET timecounter is slower than invoking a syscall for
Some not strictly intertwined but related code is the time-bound sleep
implementation. Handling of races between callouts and the top-half
code that sets and processes the timeouts depended on the many fine
details of the callout_stop(9) KPI (kernel programming interface). In
particular, races or unpunctual KPI changes usually result in the
"catch-all" unkillable thread state with the "-" waitchain bug. The
sleepqueue timeout code was rewritten to stop depending on the KPI
details, which removed the source of recurring bugs, and also
surprisingly simplified the code.
This project was sponsored by The FreeBSD Foundation.
Google Summer of Code
Google Summer of Code 2016
GSoC 2016 Projects
GSoC Ideas page
Contact: Gavin Atkinson <gavin at FreeBSD.org>
Contact: Pedro Giffuni <pfg at FreeBSD.org>
As in all previous editions of the Google Summer of Code, FreeBSD was
an accepted organization, and we had the chance to mentor 15 projects.
Huge thanks to all our mentors for keeping the high quality standards
that make our community shine.
This year was rather unique in that we accepted for the first time
well-known members of the community that are not src committers to
co-mentor. We also accepted projects that have a different upstream
than FreeBSD. Both are clear signs that FreeBSD is growing and adapting
to the wider community.
This year we are also had administrative issues with Perforce and have
officially accepted the use of external repositories, in particular
github, as requested by students.
12 of 15 projects were successful, which we think is an excellent
result for a Google Summer of Code.
This project was sponsored by Google Inc, and The FreeBSD Foundation.
1. The world is changing and we need fresh project ideas. We need to
start looking for those ideas now.
2. The project ideas wiki page has been reset and we need to get it
populated before applying for the next Google Summer of Code.
Please help unleash the next stream of projects you want to see in
Non-BSM to BSM Conversion Tools
Pull Request With Consolidated Patch
Contact: Mateusz Piotrowski <0mp at FreeBSD.org>
This project was started during Google Summer of Code this year. The
aim was to create a library which can convert the audit trail files in
Linux Audit format or the format used by Windows to the BSM format used
by FreeBSD for its audit logs. Apart from that, I wanted to create a
simple command-line tool and extend auditdistd so that it is possible
to send non-BSM logs to it over a secure connection and save those
audit logs on disk, preferably in the BSM format.
So far, it is possible to reasonably convert some of the most common
Linux audit log events to BSM, but it still needs a lot of work.
Secondly, I was able to configure auditdistd to communicate with CentOS
over an insecure connection. Thirdly, the command-line tool is usable
but not perfect.
The present work focuses on configuring the secure TLS connection
between CentOS and auditdistd. I have already tried using rsyslogd but
was not able to make it work.
This project was sponsored by Google Summer of Code.
1. I need more examples of rare Linux Audit logs; please send me some
examples if you have any. It is much easier to improve the
conversion process with real-life examples of audit events as you
write the code to convert them.
2. Configure auditdistd to be able to communicate with some software
on CentOS over TLS in order to receive audit logs. I was not able
to come up with a simple solution for that.
3. Additional open tasks are listed on the Wiki page and in the TODO
file in the root directory of the project.
ptnet Driver and bhyve Device Model
FreeBSD Wiki Page for Project Overview
Contact: Vincenzo Maffione <v.maffione at gmail.com>
This project provides:
* A new driver (if_ptnet) for a paravirtualized network device,
modeled after the netmap API. The driver supports multi-queue
netmap ports, and it is able to work both in netmap mode and in
* The emulation of the ptnet device model as a module of the bhyve
The ptnet device and driver has been introduced to overcome the
performance limitations of TCP/IP networking between bhyve VMs. Prior
to this work, the most performant solution for VM-to-VM intra-host TCP
communication provided less than 2 Gbps TCP throughput. With ptnet, in
the same VM-to-VM TCP communication scenario, it is possible to obtain
up to 20 Gbps.
This project was sponsored by Google Summer of Code.
1. Share virtio-net header management code with the if_vtnet driver.
In the current code, about 100 lines of code have been copied and
pasted from if_vtnet.c.
FreeBSD on Annapurna Labs Alpine
Contact: Jan Medala <jan at semihalf.com>
Contact: Michal Stanek <mst at semihalf.com>
Contact: Wojciech Macek <wma at semihalf.com>
Alpine is a family of Platform-on-Chip devices, including multi-core
32-bit (first-gen Alpine) and 64-bit (Alpine V2) ARM CPUs, developed by
The primary focus areas of the Alpine platform are high-performance
networking, storage, and embedded applications. The network subsystem
features 10-, 25-, and 50-Gbit Ethernet controllers with support for
virtualization, load-balancing, hardware offload and other advanced
A basic patch set has already been committed to head including:
* PCIe Root Complex support
* Cache Coherency Unit driver
* North Bridge Service driver
* Updated Alpine HAL
* Extended MSI support in GICv2 and GICv3 code
Additional work, such as an MSI-X driver and full Ethernet support, is
currently undergoing community review on Phabricator.
The multi-user SMP system is stable and fully working, along with the
1G and 10G Ethernet links.
The interrupt management code has been adjusted to work with the new
INTRNG framework on both ARM32 and ARM64.
This project was sponsored by Annapurna Labs -- an Amazon company, and
FreeBSD on Marvell Armada38x
Contact: Marcin Wojtas <mw at semihalf.com>
Contact: Bartosz Szczepanek <bsz at semihalf.com>
FreeBSD includes support for the Marvell Armada38x platform, which has
been tested and improved in order to gain production quality. Most of
this effort has been invested in development and benchmarking of the
on-chip Gigabit Ethernet (NETA) functionality. Numerous bug fixes and
some new features have been introduced.
Work completed this quarter includes:
* NETA rework and improvements.
* Enable multi-port support in PCIe 2.0 driver (mv_pci_ctrl).
* Introduce an alternative, coherent, bus_dma for the armv7 arch.
* AHCI controller support.
* SDHCI controller support.
* Improve the e6000sw etherswitch driver.
* Fix Marvell bus configuration for numerous interfaces.
Along with support for new boards (SolidRun ClearFog and
DB-88F6285-AP), all changes will be submitted upstream.
This project was sponsored by Stormshield, and Semihalf.
1. Finalize NETA and prepare for submission.
2. Submit remaining fixes and drivers.
FreeBSD arm64 Wiki Entry
Using Crochet to Build FreeBSD Images
Contact: Andrew Turner <andrew at FreeBSD.org>
Contact: Jared McNeill <jmcneill at FreeBSD.org>
Transparent superpage support has been added. This allows FreeBSD to
create 2MiB blocks with a single pagetable and TLB entry. This shows a
small but significant improvement in the buildworld time on ThunderX
machines. Superpages have been enabled in head and merged to stable/11,
but they are disabled by default on stable/11 due to a lack of testing
Support for the pre-INTRNG interrupt framework has been removed. This
means that arm64 requires INTRNG to even build. This has allowed
various cleanups within the arm64 drivers that interact with the
The cortex Strings library from Linaro has been imported. The parts of
this that have been shown to be improvements over the previous C code
were attached to the libc build.
There is ongoing work to add ACPI support to the kernel. On ThunderX,
FreeBSD can get to the mountroot prompt, however, due to incomplete
ACPI tables the external PCIe support needed to support the netboot
setup in the test cluster is not functional.
Pine64 support has been committed to head. FreeBSD can now boot to
multiuser with SMP enabled. This includes support for clocks, secure ID
controller, USB Host controller, GPIOs, non-maskable interrupts, AXP81x
power management unit, cpu freqency and voltage scaling, MMC, UART,
gigabit networking, watchdog, and thermal sensors.
This project was sponsored by The FreeBSD Foundation, and ABT Systems
UEFI Runtime Services
Contact: Konstantin Belousov <kib at FreeBSD.org>
UEFI (Unified Extensible Firmware Interface) specifies two kinds of
services for use by operating systems. Boot Services are designed for
OS loaders to load and initialize kernels, while Runtime Services are
meant to be used by kernels during regular system operations. The boot
and runtime phases are explicitly separated. During boot, when loaders
are executed, the machine configuration is owned by UEFI. During
runtime, the kernel manages the configuration, but needs to inform the
firmware about any changes that are made.
The model of split boot/runtime configuration makes assumptions about
the OS architecture that do not quite apply to the existing FreeBSD
codebase. For instance, the firmware notification of the future runtime
configuration must be done while the loader is effectively still in
control. In technical terms, the SetVirtualAddressMap() call must be
made with the 1:1 physical:virtual mapping on amd64 systems, which for
FreeBSD means that the call can only be issued by the loader. But the
loader needs to know intimate details of the kernel address map to
provide the requested information. This creates a new, unfortunate,
coupling between loader and kernel.
Reading the publicly available information about the MS Windows boot
process explained the UEFI control transfer model. The Windows loader
constructs the address map for the kernel, and with such a division of
work the UEFI model is reasonable. The FreeBSD kernel constructs its
own address space, only relying on a minimal map constructed by the
loader, which is enough for the pmap subsystem to bootstrap itself and
then to perform machine initialization in common code.
Initial experiments with enabling runtime services were centered around
utilizing the direct address map (DMAP) on amd64, which currently
always exists and linearly maps at least the lower 4G of physical
addresses at some KVA location. It was supposed that the kernel would
export the DMAP details like linear base and guaranteed size for loader
from its ELF image, and provide the needed overflow map if the DMAP
cannot completely serve. Unfortunately, two show-stopper bugs were
discovered with this approach.
First, EDK-based firmware apparently requires that the runtime mapping
exists simultaneously with the physical mapping for the
SetVirtualAddressMap() call. Second, there were references from other
open-source projects mentioning that some firmware required the
presence of the physical mapping during the runtime call. Effectively,
this forces both kernel and loader to provide both mappings for all
With such restrictions, informing the firmware about the details of the
kernel address space only adds useless work. We could just as easily
establish the 1:1 physical mapping during runtime and get rid of
SetVirtualAddressMap() entirely. This approach was coded and the kernel
interface to access runtime services is based on it.
During development, particularly when trying to make the loader
modifications, it was quickly realized that there were no
fault-reporting facilities in loader.efi. Machine exceptions resulted
in a silent hang. Curiously, in such a situation the Intel firmware
outputs the error code over the serial port over 115200/8/1 settings,
regardless of UEFI console configuration, which was discovered by
accident. Unfortunately, the error code alone is not enough to diagnose
A primitive fault reporter was written for loader.efi on amd64, which
intercepts exceptions from the firmware IDT and dumps the machine state
to the loader console. Due to the complexity of the interception and
possible bugs which might do more harm than good there, the dumper is
only activated on explicit administrator action.
Note that the described work only provides the kernel interfaces to
make calling the EFI runtime services as easy as calling a regular C
function. User-visible feature development making use of the new
interfaces is being performed right now.
This project was sponsored by The FreeBSD Foundation.
KDE on FreeBSD
KDE on FreeBSD website
KDE ports staging area
KDE on FreeBSD wiki
KDE/FreeBSD mailing list
Development repository for integrating Plasma 5 and KDE Frameworks 5
Development repository for integrating Qt 5.7
Contact: KDE on FreeBSD Team <kde at FreeBSD.org>
The KDE on FreeBSD team focuses on packaging the KDE software and
making sure that the experience of KDE and Qt on FreeBSD is as good as
The following big updates were landed in the ports tree this quarter:
* Added a Qt5 option to multimedia/mlt.
* Added the devel/grantlee5 port and, with it, Uses/grantlee.mk.
* Added the multimedia/gstreamer1-qt5 port.
* Added the net-im/telepathy-qt5 port.
* CMake was updated to versions 3.6.1 and 3.6.2.
* An important fix was made to qmake, where the clang version was not
* Qt 5.6.1 was committed to ports.
* Phonon and its backend to were updated to 4.9.0 in preparation for
* Updated the net-im/telepathy-qt4 port to 0.9.7.
* Various LibreSSL related fixes by Matthew Rezny.
* bsd.kde4.mk has been replaced by Uses/kde.mk.
* www/webkit-qt5 was fixed to depend on the systems leveldb.
In our development repository, we have done this work:
* The plasma5 branch has been kept up to date with KDE's upstream and
contains ports for Frameworks 5.26.0, Plasma Desktop 5.8.0, and
Applications 16.08.1 (branches/plasma5).
LXQt on FreeBSD
FreeBSD LXQt Project
LXQt Development Repository
Contact: Olivier Duchateau <olivierd at FreeBSD.org>
Contact: Jesper Schmitz Mouridsen <jesper at schmitz.computer>
LXQt is the Qt port of and the upcoming version of LXDE, the
Lightweight Desktop Environment. It is the product of a merge between
the LXDE-Qt and Razor-qt projects.
The porting effort remains very much a work in progress: it requires
some components of Plasma 5, the new major KDE workspace.
The porting of the 0.11 branch is now complete, with new ports
(compared to the previous release). See our wiki page for a complete
list of applications.
We also have updates for:
* x11-toolkits/qtermwidget (0.7.0)
* x11/qterminal (0.7.0)
1. Improve FreeBSD support in sysutils/lxqt-admin, especially with
respect to user management.
2. Add additional panel plugins.
Xfce on FreeBSD
FreeBSD Xfce Project
FreeBSD Xfce Repository
Contact: FreeBSD Xfce Team <xfce at FreeBSD.org>
Xfce is a free software desktop environment for Unix and Unix-like
platforms such as FreeBSD. It aims to be fast and lightweight, while
still being visually appealing and easy to use.
During this quarter, the team has kept these applications up-to-date:
* audio/xfmpc (0.2.3)
* deskutils/xfce4-notifyd (0.3.2)
* deskutils/xfce4-volumed-pulse (0.2.2)
* devel/thunar-vcs-plugin (0.1.5)
* misc/xfce4-weather-plugin (0.8.8)
* sysutils/xfce4-settings (4.12.1)
* x11/xfce4-clipman-plugin (1.4.0)
* x11/xfce4-dashboard (0.6.0)
* x11/xfce4-goodies, the meta-port for the Xfce4 Goodies Project
* x11/xfce4-whiskermenu-plugin (1.6.0)
We also follow the unstable releases; the current unstable release
brings support for Gtk3 (available in our experimental repository) to:
* audio/xfce4-mpc-plugin (0.4.99)
* sysutils/garcon (0.5.0)
* sysutils/xfce4-battery-plugin (1.0.99)
* sysutils/xfce4-diskperf-plugin (2.5.99)
* sysutils/xfce4-fsguard-plugin (1.0.99)
* sysutils/xfce4-netload-plugin (1.2.99)
* sysutils/xfce4-systemload-plugin (1.1.99)
* www/xfce4-smartbookmark-plugin (0.4.99)
* x11/libexo (0.11.1)
* x11/libxfce4menu (4.13.1)
* x11/xfce4-dashboard (0.7.0)
* x11/xfce4-terminal (0.6.92)
* x11/xfce4-whiskermenu-plugin (2.0.1)
* x11-clocks/xfce4-datetime-plugin (0.6.99)
Currently the unstable releases work fine with our Gtk3 ports available
in the ports tree, but in the future support for 3.18 will be removed
in preference of 3.20.x.
1. Continue working on unstable releases.
Documenting the History of Utilities in /bin and /sbin
The igor Port
BSD Family Tree in Subversion
The UNIX Heritage Society
Cat-V Manual Library
Contact: Sevan Janiyan <sevan at FreeBSD.org>
For EuroBSDcon, I began looking into inconsistencies within components
inside our family of operating systems. My workflow consisted of
reading the documentation for a given utility and checking the history
in the revision control system for missing fixes or functionality in
the trees of NetBSD, FreeBSD, OpenBSD, and DragonFly BSD.
One thing which became obvious very quickly was the inconsistency
between operating systems about where and/or which version a utility
originated in, despite our common heritage.
I began working through the man pages in FreeBSD, verifying the details
in pages which already had a history section and making patches for
those which did not.
From there, changes were propogated out to NetBSD, OpenBSD, and
Dragonfly BSD where applicable (not all utilities originated from the
same source or implementation, for example).
This was a good exercise in:
* Becoming familiar with mandoc.
* Using tools such as the linting functionality in mandoc and the
igor documentation script.
* Becoming familiar with the locations where things are documented
and with external sources of historical information, such as the
BSD Family Tree included in the FreeBSD base system, and projects
like The UNIX Heritage Society and the manual library on cat-v.org
which hosts copies of manuals such as those shipped with Research
UNIX. These manuals are not commonly available elsewhere.
1. Cover the remaining manuals for userland utilities, and maybe
expand into library and syscall APIs, though I say that without
estimating the feasibility. The history of components originating
from a closed-source operating system is tricky to document, since
older versions are not always available.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2
-----END PGP SIGNATURE-----
More information about the freebsd-announce