BSD Now

OpenBSD 6.8 has been released, NetBSD 9.1 is out, OpenZFS devsummit report, BastilleBSD’s native container management for FreeBSD, cleaning up old tarsnap backups, Michael W. Lucas’ book sale, and more.

NOTES

Released Oct 18, 2020. (OpenBSD's 25th anniversary)

The NetBSD Project is pleased to announce NetBSD 9.1, the first update of the NetBSD 9 release branch. It represents a selected subset of fixes deemed important for security or stability reasons, as well as new features and enhancements.

As with most other conferences in the last six months, this year’s OpenZFS Developer’s Summit was a bit different than usual. Held via Zoom to accommodate for 2020’s new normal in terms of social engagements, the conference featured a mix of talks delivered live via webinars, and breakout sessions held as regular meetings. This helped recapture some of the “hallway track” that would be lost in an online conference.

Klara offers flexible Support Subscriptions for your ZFS and FreeBSD infrastructure, simply sign up for our monthly subscription! What's even better is that for the month of October we are giving away 3 months for free, for every yearly subscription, and one month free when you sign up for a 6-months subscription! Check it out on our website!

Some time ago, I had the requirement to use FreeBSD in a project, and soon the question came up if Docker and Kubernetes can be used.

I use Tarsnap for my critical data. Case in point, I use it to backup my Bacula database dump. I use Bacula to backup my hosts. The database in question keeps track of what was backed up, from what host, the file size, checksum, where that backup is now, and many other items. Losing this data is annoying but not a disaster. It can be recreated from the backup volumes, but that is time consuming. As it is, the file is dumped daily, and rsynced to multiple locations.

For those interested in such things, I recently posted my 60,000th tweet. This prodded me to try an experiment I’ve been pondering for a while.

We have an interview with Kyle Evans for you this week. We talk about his grep project, lua and flua in base, as well as bectl, being on the core team and a whole lot of other stuff.

NOTES

Wayland on BSD, My BSD sucks less than yours, Even on SSDs, ongoing activity can slow down ZFS scrubs drastically, OpenBSD on the Desktop, simple shell status bar for OpenBSD and cwm, and more.

NOTES

After I posted about the new default window manager in NetBSD I got a few questions, including "when is NetBSD switching from X11 to Wayland?", Wayland being X11's "new" rival. In this blog post, hopefully I can explain why we aren't yet!

This paper will look at some of the differences between the FreeBSD and OpenBSD operating systems. It is not intended to be solely technical but will also show the different "visions" and design decisions that rule the way things are implemented. It is expected to be a subjective view from two BSD developers and does not pretend to represent these projects in any way.

Video

Back in the days of our OmniOS fileservers, which used HDs (spinning rust) across iSCSI, we wound up changing kernel tunables to speed up ZFS scrubs and saw a significant improvement. When we migrated to our current Linux fileservers with SSDs, I didn't bother including these tunables (or the Linux equivalent), because I expected that SSDs were fast enough that it didn't matter. Indeed, our SSD pools generally scrub like lightning.

Let's install OpenBSD on a Lenovo Thinkpad X270. I used this computer for my computer science studies. It has both Arch Linux and Windows 10 installed as dual boot. Now that I'm no longer required to run Windows, I can ditch the dual boot and install an operating system of my choice.

These days, I try to use simple and stock software as much as possible on my OpenBSD laptop. I’ve been playing with cwm(1) for weeks and I was missing a status bar. After trying things like Tint2, Polybar etc, I discovered @gonzalo’s termbar. Thanks a lot!

DragonFly v5.8.3 released to address to issues

New Project: zedfs.com, TrueNAS CORE Ready for Deployment, IPC in FreeBSD 11: Performance Analysis, Unix Wildcards Gone Wild, Unix Wars, and more

NOTES

Have you ever had an idea that keeps coming back to you over and over again? For a week? For a month? I know that feeling. My new project was born from this feeling.

TrueNAS 12.0 RC1 was released yesterday and with it, TrueNAS CORE is ready for deployment. The merger of FreeNAS and TrueNAS into a unified software image can now begin its path into mainstream use. TrueNAS CORE is the new FreeNAS and is on schedule.

Interprocess communication, IPC, is one of the most fundamental functions of a modern operating system, playing an essential role in the fabric of contemporary applications. This report conducts an investigation in FreeBSD of the real world performance considerations behind two of the most common IPC mechanisms; pipes and sockets. A simple benchmark provides a fair sense of effective bandwidth for each, and analysis using DTrace, hardware performance counters and the operating system’s source code is presented. We note that pipes outperform sockets by 63% on average across all configurations, and further that the size of userspace transmission buffers has a profound effect on performance — larger buffers are beneficial up to a point (∼ 32-64 KiB) after which performance collapses as a result of devastating cache exhaustion. A deep scrutiny of the probe effects at play is also presented, justifying the validity of conclusions drawn from these experiments.

First of all, this article has nothing to do with modern hacking techniques like ASLR bypass, ROP exploits, 0day remote kernel exploits or Chrome's Chain-14-Different-Bugs-To-Get-There... Nope, nothing of the above. This article will cover one interesting old-school Unix hacking technique, that will still work nowadays in 2013.

Dozens of different operating systems have been developed over the years, but only Unix has grown in so many varieties. There are three main branches. Four factors have facilitated this growth...

The world’s first OpenZFS based live image, FreeBSD Subversion to Git Migration video, FreeBSD Instant-workstation 2020, testing the shutdown mechanism, login_ldap added to OpenBSD, and more

NOTES

FuryBSD is a tool to test drive stock FreeBSD desktop images in read write mode to see if it will work for you before installing. In order to provide the most reliable experience possible while preserving the integrity of the system the LiveCD now leverages ZFS, compression, replication, a memory file system, and reroot (pivot root).

FreeBSD moving to Git: Why? With luck, I'll be writing a few blogs on FreeBSD's move to git later this year. Today, we'll start with "why"?

A little over a year ago I published an instant-workstation script for FreeBSD. The idea is to have an installed FreeBSD system, then run a shell script that uses only base-system utilities and installs and configures a workstation setup for you.

Following on from my recent nut setup, this is the second in a series of three posts.

With this commit, Martijn van Duren (martijn@) added login_ldap(8) to -current

High Availability Router/Firewall Using OpenBSD, CARP, pfsync, and ifstated, Building the Development Version of Emacs on NetBSD, rc.d belongs in libexec, not etc, FreeBSD 11.3 EOL, OPNsense 20.7.1 Released, MidnightBSD 1.2.7 out, and more.

NOTES

I have been running OpenBSD on a Soekris net5501 for my router/firewall since early 2012. Because I run a multitude of services on this system (more on that later), the meager 500Mhz AMD Geode + 512MB SDRAM was starting to get a little sluggish while trying to do anything via the terminal. Despite the perceived performance hit during interactive SSH sessions, it still supported a full 100Mbit connection with NAT, so I wasn’t overly eager to change anything. Luckily though, my ISP increased the bandwidth available on my plan tier to 150Mbit+. Unfortunately, the Soekris only contained 4xVIA Rhine Fast Ethernet. So now, I was using a slow system and wasting money by not being able to fully utilize my connection.

I hadn’t really planned on installing a NetBSD VM (after doing all the other two BSDs), but then a NetBSD-related Emacs bug report arrived.

Let’s open with the controversy: the scripts that live under /etc/rc.d/ in FreeBSD, NetBSD, and OpenBSD are in the wrong place. They all should live in /libexec/rc.d/ because they are code, not configuration.

As of September 30, 2020, FreeBSD 11.3 will reach end-of-life and will no longer

Overall, the jump to HardenedBSD 12.1 is looking promising from our end. From the reported issues we still have more logging quirks to investigate and especially Netmap support (used in IPS and Sensei) is lacking in some areas that were previously working. Patches are being worked on already so we shall get there soon enough. Stay tuned.

MidnightBSD 1.2.7 is available via the FTP/HTTP and mirrors as well as github.

Modernizing the OpenBSD Console, OS roles have changed, FreeBSD Cluster with Pacemaker and Corosync, Wine in a 32-bit sandbox on 64-bit NetBSD, Find package which provides a file in OpenBSD, and more.

NOTES

At the beginning were text mode consoles. Traditionally, *BSD and Linux on i386 and amd64 used text mode consoles which by default provided 25 rows of 80 columns, the "80x25 mode". This mode uses a 8x16 font stored in the VGA BIOS (which can be slightly different across vendors).

Though I do wonder sometimes, with just a slight tweak to history, how things might have been different. In another dimension somewhere, I’m using the latest BeOS-powered PowerPC laptop, and a shiny new Palm smartphone. Both of these represented the pinnacle of UI design in the 1990s, and still in the 2020s have yet to be surpassed. People call me an Apple fanboy, but I’d drop all of it in a second for that gear.

I always missed ‘proper’ cluster software for FreeBSD systems. Recently I got to run several Pacemaker/Corosync based clusters on Linux systems. I thought how to make similar high availability solutions on FreeBSD and I was really shocked when I figured out that both Pacemaker and Corosync tools are available in the FreeBSD Ports and packages as net/pacemaker2 and net/corosync2 respectively.

"Mainline pkgsrc" can't do strange multi-arch Wine builds yet, so a 32-bit sandbox seems like a reasonable way to use 32-bit Wine on amd64 without resorting to running real Windows in NVMM. We'll see if this was a viable alternative to re-reviewing the multi-arch support in pkgsrc-wip...

There is one very handy package on OpenBSD named pkglocatedb which provides the command pkglocate.

A 35 Year Old Bug in Patch, Sandbox for FreeBSD, Changing from one dataset to another within a jail, You don’t need tmux or screen for ZFS, HardenedBSD August 2020 Status Report and Call for Donations, and more.

NOTES

Larry Wall posted patch 1.3 to mod.sources on May 8, 1985. A number of versions followed over the years. It's been a faithful alley for a long, long time. I've never had a problem with patch until I embarked on the 2.11BSD restoration project. In going over the logs very carefully, I've discovered a bug that bites this effort twice. It's quite interesting to use 27 year old patches to find this bug while restoring a 29 year old OS...

A sandbox is a software which artificially limits access to the specific resources on the target according to the assigned policy. The sandbox installs hooks to the kernel syscalls and other sub-systems in order to interrupt the events triggered by the application. From the application point of view, application working as usual, but when it wants to access, for instance, /dev/kmem the sandbox software decides against the assigned sandbox scheme whether to grant or deny access.

ZFS has a the ability to share itself within a jail. That gives the jail some autonomy, and I like that.

Back in January I mentioned how to add redundancy to a ZFS pool by adding a mirrored drive. Someone with a private account on Twitter asked me why FreeBSD—and NetBSD!—doesn’t ship with a tmux or screen equivilent in base in order to daemonise the process and let them run in the background.

This last month has largely been a quiet one. I've restarted work on porting five-year-old work from the Code Pointer Integrity (CPI) project into HardenedBSD. Chiefly, I've started forward-porting the libc and rtld bits from the CPI project and now need to look at llvm compiler/linker enhancements. We need to be able to apply SafeStack to shared objects, not just application binaries. This forward-porting work I'm doing is to support that effort.

Unix and things that run on Unix have been around for a long time now. In particular, GNU Readline was first released in 1989 (as was Bash), which is long enough ago for it (or lookalikes) to become pretty much pervasive, especially in Unix shells. Today it's easy to think of readline support as something that's always been there. But of course this isn't the case. Unix in its modern form dates from V7 in 1979 and 4.2 BSD in 1983, so a lot of Unix was developed before readline and was to some degree shaped by the lack of it.

OpenZFS with ZSTD lands in FreeBSD 13, LibreSSL doc status update, FreeBSD on SPARC64 (is dead), Bringing zpool checkpoints to a FreeBSD bootloader, and more

NOTES

More than six years ago, LibreSSL was forked from OpenSSL, and almost two years ago, i explained the status of LibreSSL documentation during EuroBSDCon 2018 in Bucuresti. So it seems providing an update might be in order.

’m coming pretty late to the party, because SPARC64 support in FreeBSD is apparently doomed: After the POWER platform made the switch to a LLVM/Clang-based toolchain, SPARC64 is one of the last ones that still uses the ancient GCC 4.2-based toolchain that the project wants to finally get rid off (it has already happened as I was writing this – looks like the firm plan was not so firm after all, since they killed it off early). And compared to the other platforms it has seen not too much love in recent times… SPARC64 being a great platform, I’d be quite sad to see it go. But before that happens let’s see what the current status is and what would need to be done if it were to survive, shall we?

Almost two years ago I wrote a blog post about checkpoints in ZFS. I didn’t hide that I was a big fan of them. That said, after those two years, I still feel that there are underappreciated features in the ZFS world, so I decided to do something about that.

FreeBSD USB Audio, Kyua: An introduction for NetBSD users, Keeping backup ZFS on Linux kernel modules around, CLI Tools 235x Faster than Hadoop, FreeBSD Laptop Battery Life Status Command, and more.

NOTES

I recently got a Behringer UMC22 sound card for video conferencing and DJing. This page documents what I’ve learned about using this sound card, and USB audio in general, on FreeBSD.

Kyua's current goal is to reimplement only the ATF tools while maintaining backwards compatibility with the tests written with the ATF libraries (i.e. with the NetBSD test suite).

I'm a long term user of ZFS on Linux and over pretty much all of the time I've used it, I've built it from the latest development version. Generally this means I update my ZoL build at the same time as I update my Fedora kernel, since a ZoL update requires a kernel reboot anyway. This is a little bit daring, of course, although the ZoL development version has generally been quite solid (and this way I get the latest features and improvements long before I otherwise would).

As I was browsing the web and catching up on some sites I visit periodically, I found a cool article from Tom Hayden about using Amazon Elastic Map Reduce (EMR) and mrjob in order to compute some statistics on win/loss ratios for chess games he downloaded from the millionbase archive, and generally have fun with EMR. Since the data volume was only about 1.75GB containing around 2 million chess games, I was skeptical of using Hadoop for the task, but I can understand his goal of learning and having fun with mrjob and EMR. Since the problem is basically just to look at the result lines of each file and aggregate the different results, it seems ideally suited to stream processing with shell commands. I tried this out, and for the same amount of data I was able to use my laptop to get the results in about 12 seconds (processing speed of about 270MB/sec), while the Hadoop processing took about 26 minutes (processing speed of about 1.14MB/sec).

I know how to find out battery life status using Linux operating system. How do I monitor battery status on a laptop running FreeBSD version 9.x/10.x/11.x/12.x?

BSD Beer