2012: a BSD year in retrospective

BSD (Berkely System
Distribution)

was a research operating system based on the
original AT&T Unix, developed by the University of Berkeley, California. It has
been Open Source right from the beginning, and after the university lost
interest in developing it further, several community projects started up (the
very first ones were NetBSD and FreeBSD in the early nineties) to continue
developing BSD. Anyway, Linux was born roughly at the same time, but a pending
lawsuit about copyright infringements prevented the BSD projects to become as
successful as Linux (though you could argue about the exact reasons).

This led to Linux having great media coverage, while the BSD projects – though
still having fresh innovations and being actively developed – except for new
releases rarely make it to the press.

Community

As already covered on OSNews
there was a debate about increasing incompatibilities of third party tools
with BSD. This has always been an issue, especially for everything but FreeBSD,
which has some coverage by hardware vendors as it is often used in servers.
Though this was raised on an OpenBSD mailing list, developers are frustrated
everywhere. But to understand the reasons for the concern, it’s important to
understand the different philosophies of BSD and Linux developers. BSD guys are
conservative folks, and they have to be: Every decision they make will
immediately influence the whole operating system, as the whole BSD userland is
bundled together and developed by the same persons and teams. Thus they tend
to discuss more of the decisions part of any technical design.

With Linux, you don’t have this problem. The GNU/Linux world consists of many
independent software projects and distributions bundling them. A distributor
can easily decide not to adopt a special version of a tool, but rather stay with
an old one in order to preserve compatibility.

So the major issue in this case is not only that there are incompatibilities —
it is a general technical refusal of several implementations, but at the same
time being forced to implement things to be compatible with the rest of the
Open Source world (and especially desktop environments).

As an example, just browse a random BSD board, mailing list or chat, and search
for discussions about systemd. You’ll most probably find the worst trolling
and ranting about it, while the Linux world in most cases took it with a shrug.

DragonFlyBSD

DragonFlyBSD is a fork of FreeBSD after the core
developer of DragonFlyBSD, who was a FreeBSD developer at that time, didn’t like
the technical decisions taken in the project and decided to found his own one.

In 2012, DragonFlyBSD released two new versions: 3.0 on February 22nd, and 3.2
on November 2nd.

The ongoing work to improve SMP and clustering support resulted in exceptionally
good PostgreSQL benchmarks
also
beating Linux, but there were also several other important changes:

  • VFS accounting/vquota — DragonFlyBSD introduced a quota subsystem in their
    VFS (Virtual File System) layer, which is the subsystem used for accessing
    the real filesystem modules. Thus, they are able to have quotas on any
    filesystem, no matter if the filesystem supports it or not – but with the
    current implementation, those quotas are only stored until the machine is shut
    down.

  • TRIM support — DragonFlyBSD now supports the TRIM command to improve
    communication of the operating system with their solid state drives.

  • USB4BSD import — The USB4BSD USB stack originally written for FreeBSD was
    imported to DragonFly, allowing easier maintenance and a broad range of
    drivers, including XHCI (USB 3.0) support.

  • TrueCrypt/tcplay — TrueCrypt, a disk encryption tool, though being Closed
    Source, gained a wide distribution among computers due to its ease of use and
    cross-platform compatibility. DragonFlyBSD developers decided to write a
    compatible Open Source implementation. This implementation has already been
    ported to Linux and imported into some distributions, e.g. Fedora.

When it comes to filesystems, DragonFlyBSD is famous for its
HAMMER filesystem, which, informally
speaking, is something like a ZFS, but with DragonFlyBSD’s more powerful vfs
layer. In February 2012, with the conclusions and experiences gained from
HAMMER, the development of
HAMMER2 started.

While DragonFlyBSD currently uses pkgsrc, the package
system primarily used by NetBSD (but being cross-platform), there are
intentions to introduce a new
system
.
It is named DPorts, providing binary packages on top of FreeBSD’s packaging system,
ports. Though it is rather experimental for now, the final goal is to make binary
packages and the new pkg tool from FreeBSD the
default for coming releases. Users won’t be forced to compile packages anymore.

FreeBSD/PC-BSD

FreeBSD is by far the most active BSD when it comes to sheer numbers, having the
largest user and developer base (roughly 80% of the whole BSD community).
In 2012, a new minor version of FreeBSD 8 (8.3-RELEASE in April) was released,
as well as a new major version (9.0-RELEASE in January) and its first minor
release (9.1-RELEASE in December).

With 9.0-RELEASE, there came a long list of changes:

  • HASTHAST, the Highly Available STorage
    is an easy way to mirror block devices over the network and create high
    availability solutions with them. Essentially, it’s a RAID with
    synchronisation which is geared for networking.

  • Capsicum — The capabilities framework
    Capsicum has been
    added to FreeBSD but tagged experimental. It enables applications to use
    fine-tuned capabilities. There is a dedicated developer for Capsicum, being
    funded by Google (which is apprently interested in using it for Chrome).

  • Softupdates journaling — FFS now supports softupdates
    journaling
    ,
    enabling you to use softupdates
    in conjunction with
    journaling, which have
    been two competing technologies until now.

  • bsdinstall — The new
    bsdinstall(8)
    replaces the old installer which got a bit rusty in the end, not supporting
    many of the features FreeBSD introduced in the last years.
    bsdinstall provides more functionality and is more extendable by just being
    a bunch of sh scripts with dialog(1).

  • NFSv4 — FreeBSD 9.0 supports the fourth version of the network file system
    NFS. Though it has been specified in 2000 (and again in 2003), there were
    only few implementations so far.

  • Playstation 3 — FreeBSD now officially supports the Playstation
    3 game console. This might be a bit late, but the Playstation 3 has been
    useful for several number crunching applications due to its processor and its
    low price.

  • RACCT — A new Resource Accounting framework (RACCT) has been added:
    rctl(8)
    now supports limiting the resources on a per-jail, per-user, per-process or
    per-login-class model, but not percentual CPU time yet.

  • gpart — The ancient tools for labeling disks (fdisk, *label) have been
    dropped in favour of
    gpart(8),
    though they will remain in base for compatibility reasons. The new tool
    should mainly be used with GPT partitions, the old types (MBR and disklabel)
    are not endorsed anymore.

  • Clang/LLVM — In FreeBSD 9.0, the new compiler has been imported.
    It is not used for building 9.0, but in November 2012, all platforms were
    switched to use Clang by default, i.e. FreeBSD 10.0 will build with Clang
    instead of GCC by default.

With 9.1-RELEASE, there was a change indicating an interesting development:
ZFS changes from Illumos were pulled in. Maybe there is hope for an
Oracle-independent development of ZFS?

As many Open Source projects do, FreeBSD had a
fundraising campaign at the end
of the year, this time achieving a record result: About 750k USD were collected,
showing the commercial size of FreeBSD well in comparison to the other BSDs.

The money is spent for maintaining infrastructure and funding new projects, some
of which promise interesting results:

  • standard ARM platform — A standard ARM platform, the Genesi Efika MX
    SmartBook and SmartTop (ever heard of it?) with an ARM Cortex A8 SoC was
    determined, and developers got free devices to start development for.

  • NAND flash — The goal of this project is to improve the support for
    devices running on small NAND flash memory, i.e. a filesystem, a driver
    framework for controllers and memory chips and tools shall be developed.

  • growfs — The tool for enlarging filesystems shall be modified to be able
    to grow filesystems online. Until now, growfs is only capable of handling
    offline filesystems, i.e. unmounted filesystems.

For FreeBSD, there were two unpleasant events in 2012: First, the security
advisories
on
the
best
possible
date,
23rd of December (the day just before Christmas holidays, when most administrators
had already left). And second, there was another broad security issue this year:
a security breach, published
on November 17th. Administrators might have noticed that for about one week, there
were no portsnap updates to ports trees. The reason was that one week earlier,
the security team found an SSH key of a developer compromised as of 19th of
September, with access to two ports development machines.

Eventually, the security team announced they didn’t find anything security-relevant, but
still recommend not to trust binary package installations made in that time
frame. In December, everything was completely back online.

NetBSD

NetBSD is the oldest BSD derivate still under development. A few years ago, its
developers mostly focused on supporting many platforms and providing a good
abstraction to achieve this. Nowadays, the focus slightly shifted to provide a clean operating
system. Here, clean means that code should be clean, and new features should be
introduced thoughtfully, not just to be removed again after some releases or break
compatibility.

Three years after the last major NetBSD release (5.0), NetBSD 6.0 was finally
released on October 17th. The NetBSD 5 branch was updated to 5.2 on 3rd of
December and NetBSD 4.0 support was dropped five years after its initial release
in 2007.

Nearly three years after the last release, there are a lot of changes for 6.0:

  • apropos — During the Google Summer Of Code (GSoC) 2011, there was a
    project to rewrite
    apropos(1)
    to not only have a dumb full-text search, but a relevance-weighted search.
    This apropos is now included in NetBSD 6.0.
  • posix_spawn — POSIX finally decided to specify an interface to spawn a new
    process, named
    posix_spawn.
    In a GSoC project, this was implemented and is now part of NetBSD.

  • netpgp — Due to many large dependencies GnuPG has, and the fact that it is
    GPL licenced (remember: BSD folks don’t like that), there was a rewrite of
    PGP/GPG, named
    netpgp. The
    implementation is not yet as feature-rich as gpg, but it is being further
    developed and is already capable of doing basic encryption and signing.

  • v7fs, chfs — The ancient Unix 7th Edition file system has been
    reimplemented. The reason behind it is to have a very small filesystem
    alternative to FAT, and to be able to read old disks from PDP-11s.
    On the other hand, there is a new filesystem named CHFS which is intended to
    be used on small flash devices.

  • npf — In NetBSD 6.0, a new firewall solution named npf
    was introduced. It was written as a replacement to overcome the
    problems with the two which were already implemented (ipf and pf).

Recently, Antti Kantee (pooka@) provided two more nice applications of his
rump anykernel concept: He ran NetBSD
kernel modules on
Linux
and
even in a
browser
,
using JavaScript.

In July, there was an interesting post on a mailing list from a Google
developer
.
It addressed the bionic C library used in Android devices, which apparently uses
many parts from several sources.
The goal of the developer was to merge the code used by Android back with
upstream, eventually taking one single C library, and not several different
ones.
Though the discussion just ended without a result (at least without a public
result), it is still nice to see how many projects are actually using parts of
NetBSD’s code somehwere.

This is also the case for MINIX. On March 7th, MINIX
3.2.0
was released.
As the developers of MINIX are mainly researchers who want to use MINIX for
researching, and not for developing software, and their userland was fairly old,
they decided to use another one: NetBSD’s.
Thus, from MINIX 3.2.0 on, there are large parts of the ordinary NetBSD
userland as well as pkgsrc, the package management from NetBSD contained in
MINIX.

OpenBSD

OpenBSD is a fork of NetBSD after personal issues between the now-OpenBSD main
developer and the rest of the development team back in the nineties. Nowadays,
OpenBSD is famous for its focus on security and its radicalness in not
accepting Closed Source software, leading to the development of common tools like OpenSSH.

As every year, OpenBSD released two new versions (5.1 in May, 5.2 in November) – they have a fixed periodic release schedule.
OpenBSD releases always have a motto and an accompanying release song, sometimes
also with a story behind it. 5.1 had the generic motto Bug Busters, while 5.2
had the motto Aquarela Do Linux, referring to the problems Marc Espie pointed
out as mentioned before (see “Community”).

The major changes in 2012 were:

  • NAT64 — OpenBSD 5.1 introduced NAT64
    for their packet filter pf.

  • Books — The preorder of Absolute OpenBSD, 2nd Edition, a famous
    book about OpenBSD, started.

  • pthreads — Until then, pthreads in OpenBSD had been user-level only.
    This year, rthreads were introduced, mapping pthreads of a userland utility
    directly to kernel processes, improving SMP capability and speed, now making
    better usage of multicore machines.

In June 2012, the project Bitrig forked from OpenBSD.
Their philosophy, as opposed to OpenBSD, is mainly about being less
conservative. Its developers want to drop the main focus on security and
supporting old platforms, but focus on

  • using Clang as their default compiler (OpenBSD uses GCC),
  • providing better KVM (virtualisation) support,
  • providing filesystem journaling by porting NetBSD’s WAPBL,
  • supporting only x86 machines (i386, amd64),
  • improving OpenBSD’s SMP support.

Note

You should have in mind that I’m part of the FreeBSD and NetBSD community
myself, so this report might have been biased towards these two.

I also excluded all the general improvements operating systems go through over
time: improving performance, adding drivers, becoming more stable.

61 Comments

  1. 2013-02-14 10:31 pm
    • 2013-02-14 10:56 pm
    • 2013-02-15 10:17 am
      • 2013-02-15 2:39 pm
      • 2013-02-15 2:42 pm
        • 2013-02-16 12:22 am
    • 2013-02-16 4:06 am
    • 2013-02-20 12:09 am
  2. 2013-02-15 12:45 am
  3. 2013-02-15 1:27 am
    • 2013-02-18 9:51 am
      • 2013-02-18 7:13 pm
  4. 2013-02-15 1:33 am
    • 2013-02-18 4:33 pm
  5. 2013-02-15 2:24 am
    • 2013-02-15 1:39 pm
      • 2013-02-16 2:16 am
  6. 2013-02-15 3:30 am
    • 2013-02-15 5:13 am
      • 2013-02-15 7:35 pm
    • 2013-02-15 6:48 am
    • 2013-02-15 10:36 am
    • 2013-02-15 6:49 pm
      • 2013-02-15 7:33 pm
        • 2013-02-16 3:48 am
          • 2013-02-16 1:24 pm
          • 2013-02-17 1:36 am
          • 2013-02-18 4:07 pm
  7. 2013-02-15 2:46 pm
    • 2013-02-15 3:21 pm
    • 2013-02-15 3:52 pm
      • 2013-02-15 9:00 pm
        • 2013-02-15 10:28 pm
    • 2013-02-15 3:55 pm
      • 2013-02-15 6:30 pm
      • 2013-02-15 11:54 pm
        • 2013-02-18 2:54 pm
          • 2013-02-18 10:19 pm
  8. 2013-02-15 3:59 pm
  9. 2013-02-15 5:29 pm
  10. 2013-02-15 7:17 pm
  11. 2013-02-15 8:55 pm
  12. 2013-02-15 9:08 pm
    • 2013-02-16 3:43 am
    • 2013-02-16 7:47 am
      • 2013-02-16 7:17 pm
    • 2013-02-18 6:59 pm
  13. 2013-02-15 9:52 pm
    • 2013-02-15 10:50 pm
  14. 2013-02-16 8:44 am
    • 2013-02-16 10:46 am
      • 2013-02-17 4:30 am
        • 2013-02-17 11:17 pm
        • 2013-02-20 7:19 pm
  15. 2013-02-16 4:47 pm
  16. 2013-02-17 8:33 pm
  17. 2013-02-18 9:38 am
    • 2013-02-18 7:00 pm
  18. 2013-02-18 9:43 am
  19. 2013-02-19 3:38 am