BSD Now

FreeBSD 11.4-RC 2 available, OpenBSD 6.7 on a PineBook Pro 64, How OpenZFS Keeps Your Data Safe, Bringing FreeBSD to EC2, FreeBSD 2020 Community Survey, and more.

NOTES

The second RC build of the 11.4-RELEASE release cycle is now available.

This document is work in progress and I'll update the date above once I change something. If you have something to add, remarks, etc please contact me. Preferably via Mastodon but other means of communication are also fine.

Veteran technology writer Jim Salter wrote an excellent guide on the ZFS file system’s features and performance that we absolutely had to share. There’s plenty of information in the article for ZFS newbies and advanced users alike. Be sure to check out the article over at Ars Technica to learn more about ZFS concepts including pools, vdevs, datasets, snapshots, and replication, just to name a few.

Colin is the founder of Tarsnap, a secure online backup service which combines the flexibility and scriptability of the standard UNIX "tar" utility with strong encryption, deduplication, and the reliability of Amazon S3 storage. Having started work on Tarsnap in 2006, Colin is among the first generation of users of Amazon Web Services, and has written dozens of articles about his experiences with AWS on his blog.

The FreeBSD Core Team invites you to complete the 2020 FreeBSD Community Survey. The purpose of this survey is to collect quantitative data from the public in order to help guide the project’s priorities and efforts. This is only the second time a survey has been conducted by the FreeBSD Project and your input is valued.

Sponsored By:

• Tarsnap Promo Code: bsdnow

Scheduling in NetBSD, ZFS vs. RAID on Ironwolf disks, OpenBSD on Microsoft Surface Go 2, FreeBSD for Linux sysadmins, FreeBSD on Lenovo T480, and more.

NOTES

In this blog, we will discuss about the 4.4BSD Thread scheduler one of the two schedulers in NetBSD and a few OS APIs that can be used to control the schedulers and get information while executing.

This has been a long while in the making—it's test results time. To truly understand the fundamentals of computer storage, it's important to explore the impact of various conventional RAID (Redundant Array of Inexpensive Disks) topologies on performance. It's also important to understand what ZFS is and how it works. But at some point, people (particularly computer enthusiasts on the Internet) want numbers.

I used OpenBSD on the original Surface Go back in 2018 and many things worked with the big exception of the internal Atheros WiFi. This meant I had to keep it tethered to a USB-C dock for Ethernet or use a small USB-A WiFi dongle plugged into a less-than-small USB-A-to-USB-C adapter.

If you’ve ever installed and explored another Linux distro (what Linux sysadmin hasn’t?!?), then exploring FreeBSD is going be somewhat similar with a few key differences.

Recently I replaced my 2014 MacBook Air with a Lenovo Thinkpad T480, on which I've installed FreeBSD, currently 12.1-RELEASE. This page documents my set-up along with various configuration tweaks and fixes.

Sponsored By:

• Tarsnap Promo Code: bsdnow

A brief introduction to randomness, logs grinding netatalk to a halt, NetBSD core team changes, Using qemu guest agent on OpenBSD kvm/qemu guests, WireGuard patchset for OpenBSD, FreeBSD 12.1 on a laptop, and more.

A brief introduction to randomness

But what if we want them to act unpredictably? This is very useful if we want to secure our private communications with randomized keys, or not let people cheat at video games, or if we're doing statistical simulations or similar.

I’ve heard it said the cobbler’s children walk barefoot. While posessing the qualities of a famed financial investment strategy, it speaks to how we generally put more effort into things for others than ourselves; at least in business.

Matt Thomas (matt@) has served on the NetBSD core team for over ten years, and has made many contributions, including ELF functionality, being the long-time VAX maintainer, gcc contributor, the generic pmap, and also networking functionality, and platform bring-up over the years. Matt has stepped down from the NetBSD core team, and we thank him for his many, extensive contributions.

In a post to the ports@ mailing list, Landry Breuil (landry@) shared some of his notes on using qemu guest agent on OpenBSD kvm/qemu guests.

A while ago I wanted to learn more about OpenBSD development. So I picked a project, in this case WireGuard, to develop a native client for. Over the last two years, with many different iterations, and working closely with the WireGuard's creator (Jason [Jason A. Donenfeld - Ed.], CC'd), it started to become a serious project eventually reaching parity with other official implementations. Finally, we are here and I think it is time for any further development to happen inside the src tree.

I’m using FreeBSD again on a laptop for some reasons so expect to read more about FreeBSD here. This tutorial explain how to get a graphical desktop using FreeBSD 12.1.

Backup and Restore on NetBSD, OpenBSD 6.7 available, Building a WireGuard Jail with FreeBSD's standard tools, who gets to chown things and quotas, influence TrueNAS CORE roadmap, and more.

Putting together the bits and pieces of a backup and restore concept, while not being rocket science, always seems to be a little bit ungrateful. Most Admin Handbooks handle this topic only within few pages. After replacing my old Mac Mini's OS by NetBSD, I tried to implement an automated backup, allowing me to handle it similarly to the time machine backups I've been using before. Suggestions on how to improve are always welcome.

The OpenBSD project produces and operating system which places focus on portability, standardisation, code correctness, proactive security and integrated cryptography. The project's latest release is OpenBSD 6.7 which introduces several new improvements to the cron scheduling daemon, improvements to the web server daemon, and the top command now offers scrollable output. These and many more changes can be found in the project's release announcement: "This is a partial list of new features and systems included in OpenBSD 6.7. For a comprehensive list, see the changelog leading to 6.7. General improvements and bugfixes: Reduced the minimum allowed number of chunks in a CONCAT volume from 2 to 1, increasing the number of volumes which can be created on a single disk with bioctl(8) from 7 to 15. This can be used to create more partitions than previously. Rewrote the cron(8) flag-parsing code to be getopt-like, allowing tight formations like -ns and flag repetition. Renamed the 'options' field in crontab(5) to 'flags'. Added crontab(5) -s flag to the command field, indicating that only a single instance of the job should run concurrently. Added cron(8) support for random time values using the ~ operator. Allowed cwm(1) configuration of window size based on percentage of the master window during horizontal and vertical tiling actions."

Recently, I had an opportunity to build a WireGuard jail on a FreeBSD 12.1 host.

One of the famous big splits between the BSD Unix world and the System V world is whether ordinary users can use chown (the command and the system call) to give away their own files. In System V derived Unixes you were generally allowed to; in BSD derived Unixes you weren't. Until I looked it up now to make sure, I thought that BSD changed this behavior from V7 and that V7 had an unrestricted chown. However, this turns out to be wrong; in V7 Unix, chown(2) was restricted to root only.

As many of you know, we’ve historically had three ticket types available in our tracker: Bugs, Features, and Improvements, which are all fairly self-explanatory. After some discussion internally, we’ve decided to implement a new type of ticket, a “Suggestion”. These will be replacing Feature and Improvement requests for the TrueNAS Community, simplifying things down to two options: Bugs and Suggestions. This change also introduces a slightly different workflow than before.

5x if_bridge Performance Improvement, How Unix Won, Understanding VLAN Configuration on FreeBSD, Using bhyve PCI passthrough on OmniOS, TrueNAS 11.3-U2 Available, and more.

With FreeBSD Foundation grant, Kristof Provost harnesses new parallel techniques to uncork performance bottleneck

+> Unix has won in every conceivable way. And in true mythic style, it contains the seeds of its own eclipse. This is my subjective historical narrative of how that happened.

I’m using the name “Unix” to include the entire family of operating systems descended from it, or that have been heavily influenced by it. That includes Linux, SunOS, Solaris, BSD, Mac OS X, and many, many others.

This blog post continues where the blog post A central log host with syslog-ng on FreeBSD left off. Open source solutions to check syslog log messages exist, such as Logcheck or Logwatch. Although these are not to difficult to implement and maintain, I still found these to much. So I went for my own home grown solution to check the syslog messages of the SoCruel.NU central log host. And the solution presented in this blog post works pretty well for me!

Until recently, I’ve never had a chance to use VLANs on FreeBSD hosts, though I sometimes configure them on ethernet switches.

Some hardware is not supported in illumos yet, but luckily there is bhyve which supports pci passthrough to any guest operating system. To continue with my OmniOS desktop on "modern" hardware I would love wifi support, so why not using a bhyve guest as router zone which provide the required drivers?

TrueNAS 11.3-U2.1 is generally available as of 4/22/2020. This update is based on FreeNAS 11.3-U2 which has had over 50k deployments and received excellent community and third party reviews. The Release Notes are available on the iXsystems.com website.

HardenedBSD April 2020 Status Report

Encrypted Crash Dumps in FreeBSD, Time on Unix, Improve ZVOL sync write performance with a taskq, central log host with syslog-ng, NetBSD Entropy overhaul, Setting Up NetBSD Kernel Dev Environment, and more.

Some time ago, I was describing how to configure networking crash dumps. In that post, I mentioned that there is also the possibility to encrypt crash dumps. Today we will look into this functionality. Initially, it was implemented during Google Summer of Code 2013 by my friend Konrad Witaszczyk, who made it available in FreeBSD 12. If you can understand Polish, you can also look into his presentation on BSD-PL on which he gave a comprehensive review of all kernel crash dumps features.

The main issue with crash dumps is that they may include sensitive information available in memory during a crash. They will contain all the data from the kernel and the userland, like passwords, private keys, etc. While dumping them, they are written to unencrypted storage, so if somebody took out the hard drive, they could access sensitive data. If you are sending a crash dump through the network, it may be captured by third parties. Locally the data are written directly to a dump device, skipping the GEOM subsystem. The purpose of that is to allow a kernel to write a crash dump even in case a panic occurs in the GEOM subsystem. It means that a crash dump cannot be automatically encrypted with GELI.

Time, a word that is entangled in everything in our lives, something we’re intimately familiar with. Keeping track of it is important for many activities we do.

Over millennia we’ve developed different ways to calculate it. Most prominently, we’ve relied on the position the sun appears to be at in the sky, what is called apparent solar time.

We’ve decided to split it as seasons pass, counting one full cycle of the 4 seasons as a year, a full rotation around the sun. We’ve also divided the passing of light to the lack thereof as days, a rotation of the earth on itself. Moving on to more precise clock divisions such as seconds, minutes, and hours, units that meant different things at different points in history. Ultimately, as travel got faster, the different ways of counting time that evolved in multiple places had to converge. People had to agree on what it all meant.

See the article for more

syslog-ng is the Swiss army knife of log management. You can collect logs from any source, process them in real time and deliver them to wide range of destinations. It allows you to flexibly collect, parse, classify, rewrite and correlate logs from across your infrastructure. This is why syslog-ng is the perfect solution for the central log host of my (mainly) FreeBSD based infrastructure.

This week I committed an overhaul of the kernel entropy system. Please let me know if you observe any snags! For the technical background, see the thread on tech-kern a few months ago: https://mail-index.NetBSD.org/tech-kern/2019/12/21/msg025876.html.

I used T_PAGEFLT’s blog post as a reference for setting my NetBSD kernel development environment since his website is down I’m putting down the steps here so it would be helpful for starters.

FuryBSD 2020Q2 Images Available, Technical reasons to choose FreeBSD over GNU/Linux, Ars technica reviews GhostBSD, “TLS Mastery” sponsorships open, BSD community show their various collections, a tale of OpenBSD secure memory allocator internals, learn to stop worrying and love SSDs, and more.

The Q2 2020 images are not a visible leap forward but a functional leap forward. Most effort was spent creating a better out of box experience for automatic Ethernet configuration, working WiFi, webcam, and improved hypervisor support.

Since I wrote my article "Why you should migrate everything from Linux to BSD" I have been wanting to write something about the technical reasons to choose FreeBSD over GNU/Linux and while I cannot possibly cover every single reason, I can write about some of the things that I consider worth noting.

When I began work on the FreeBSD 12.1-RELEASE review last week, it didn't take long to figure out that the desktop portion wasn't going very smoothly.

I think it's important for BSD-curious users to know of easier, gentler alternatives, so I did a little looking around and settled on GhostBSD for a follow-up review.

GhostBSD is based on TrueOS, which itself derives from FreeBSD Stable. It was originally a Canadian distro, but—like most successful distributions—it has transcended its country of origin and can now be considered worldwide. Significant GhostBSD development takes place now in Canada, Italy, Germany, and the United States.

My next book will be TLS Mastery, all about Transport Layer Encryption, Let’s Encrypt, OCSP, and so on.

This should be a shorter book, more like my DNSSEC or Tarsnap titles, or the first edition of Sudo Mastery. I would like a break from writing doorstops like the SNMP and jails books.

Do you have a nice collection, take a picture and send it in!

Hi there,

It's been a very long time I haven't written anything after my last OpenBSD blogs, that is,

OpenBSD Kernel Internals — Creation of process from user-space to kernel space.

OpenBSD: Introduction to execpromises in the pledge(2)

pledge(2): OpenBSD's defensive approach to OS Security

So, again I started reading OpenBSD source codes with debugger after reducing my sleep timings and managing to get some time after professional life. This time I have picked one of my favourite item from my wishlist to learn and share, that is, OpenBSD malloc(3), secure allocator

my home FreeNAS runs two pools for data. One RAIDZ2 with four spinning disk drives and one mirror with two SSDs. Toying with InfluxDB and Grafana in the last couple of days I found that I seem to have a constant write load of 1 Megabyte (!) per second on the SSDs. What the ...?

So I run three VMs on the SSDs in total. One with Windows 10, two with Ubuntu running Confluence, A wiki essentially, with files for attachments and MySQL as the backend database. Clearly the writes had to stop when the wikis were not used at all, just sitting idle, right?

Well even with a full query log and quite some experience in the operation of web applications I could not figure out what Confluence is doing (productively, no doubt) but trust me, it writes a couple of hundred kbytes to the database each second just sitting idle.

I've wanted to write about my infrastructure for a while, but I kept thinking, "I'll wait until after I've done $next_thing_on_my_todo." Of course this cycle never ends, so I decided to write about its state at the end of 2019. Maybe I'll write an update on it in a couple of moons; who knows?

Rethinking OpenBSD security, FreeBSD 2020 Q1 status report, the notion of progress and user interfaces, Comments about Thomas E. Dickey on NetBSD curses, making Unix a little more Plan9-like, Not-actually Linux distro review: FreeBSD, and more.

OpenBSD aims to be a secure operating system. In the past few months there were quite a few security errata, however. That’s not too unusual, but some of the recent ones were a bit special. One might even say bad. The OpenBSD approach to security has a few aspects, two of which might be avoiding errors and minimizing the risk of mistakes. Other people have other ideas about how to build secure systems. I think it’s worth examining whether the OpenBSD approach works, or if this is evidence that it’s doomed to failure.

Welcome, to the quarterly reports, of the future! Well, at least the first quarterly report from 2020. The new timeline, mentioned in the last few reports, still holds, which brings us to this report, which covers the period of January 2020 - March 2020.

One trait of modern Western culture is the notion of progress. A view claiming, at large, everything is getting better and better.

How should we think about progress? Both in general and regarding technology?

I was recently pointed at a web page on Thomas E. Dickeys site talking about NetBSD curses. It seems initially that the page was intended to be a pointer to some differences between ncurses and NetBSD curses and does appear to start off in this vein but it seems that the author has lost the plot as the document evolved and the tail end of it seems to be devolving into some sort of slanging match. I don't want to go through Mr. Dickey's document point by point, that would be tedious but I would like to pick out some of the things that I believe to be the most egregious. Please note that even though I am a NetBSD developer, the opinions below are my own and not the NetBSD projects.

I’m not really interested in defending anything. I tried out plan9port and liked it, but I have to live in Unix land. Here’s how I set that up.

A Warning

The suckless community, and some of the plan9 communities, are dominated by jackasses. I hope that’s strong enough wording to impress the severity. Don’t go into IRC for help. Stay off the suckless email list. The software is great, the people who write it are well-spoken and well-reasoned, but for some reason the fandom is horrible to everyone.

This month's Linux distro review isn't of a Linux distribution at all—instead, we're taking a look at FreeBSD, the original gangster of free Unix-like operating systems.

The first FreeBSD release was in 1993, but the operating system's roots go further back—considerably further back. FreeBSD started out in 1992 as a patch-release of Bill and Lynne Jolitz's 386BSD—but 386BSD itself came from the original Berkeley Software Distribution (BSD). BSD itself goes back to 1977—for reference, Linus Torvalds was only seven years old then.

Before we get started, I'd like to acknowledge something up front—our distro reviews include the desktop experience, and that is very much not FreeBSD's strength. FreeBSD is far, far better suited to running as a headless server than as a desktop! We're going to get a full desktop running on it anyway, because according to Lee Hutchinson, I hate myself—and also because we can't imagine readers wouldn't care about it.

FreeBSD does not provide a good desktop experience, to say the least. But if you're hankering for a BSD-based desktop, don't worry—we're already planning a followup review of GhostBSD, a desktop-focused BSD distribution.

Tales from a core file, Lenovo X260 BIOS Update with OpenBSD, the problem of Unix iowait and multi-CPU machines, Hugo workflow using FreeBSD Jails, Caddy, Restic; extending NetBSD-7 branch support, a tale of two hypervisor bugs, and more.

On the side, I’ve been wrapping up some improvements to the classic Unix stdio libraries in illumos. stdio contains the classic functions like fopen(), printf(), and the security nightmare gets(). While working on support for fmemopen() and friends I got to reacquaint myself with some of the joys of the stdio ABI and its history from 7th Edition Unix. With that in mind, let’s dive into this, history, and some mistakes not to repeat. While this is written from the perspective of the C programming language, aspects of it apply to many other languages.

My X260 only runs OpenBSD and has no CD driver. But I still need to upgrade its BIOS from time to time. And this is possible using the ISO BIOS image.

First off all, you need to download the “BIOS Update (Bootable CD)” from the Lenovo Support Website.

Various Unixes have had a 'iowait' statistic for a long time now (although I can't find a source for where it originated; it's not in 4.x BSD, so it may have come through System V and sar). The traditional and standard definition of iowait is that it's the amount of time the system was idle but had at least one process waiting on disk IO. Rather than count this time as 'idle' (as you would if you had a three-way division of CPU time between user, system, and idle), some Unixes evolved to count this as a new category, 'iowait'.

After hosting with Netlify for a few years, I decided to head back to self hosting. Theres a few reasons for that but the main reasoning was that I had more control over how things worked.

In this post, i’ll show you my workflow for deploying my Hugo generated site (www.jaredwolff.com). Instead of using what most people would go for, i’ll be doing all of this using a FreeBSD Jails based server. Plus i’ll show you some tricks i’ve learned over the years on bulk image resizing and more.

Let’s get to it.

Typically, some time after releasing a new NetBSD major version (such as NetBSD 9.0), we will announce the end-of-life of the N-2 branch, in this case NetBSD-7.

We've decided to hold off on doing that to ensure our users don't feel rushed to perform a major version update on any remote machines, possibly needing to reach the machine if anything goes wrong.

Security fixes will still be made to the NetBSD-7 branch.

We hope you're all safe. Stay home.

VM escape has become a popular topic of discussion over the last few years. A good amount of research on this topic has been published for various hypervisors like VMware, QEMU, VirtualBox, Xen and Hyper-V. Bhyve is a hypervisor for FreeBSD supporting hardware-assisted virtualization. This paper details the exploitation of two bugs in bhyve - FreeBSD-SA-16:32.bhyve (VGA emulation heap overflow) and CVE-2018-17160 (Firmware Configuration device bss buffer overflow) and some generic techniques which could be used for exploiting other bhyve bugs. Further, the paper also discusses sandbox escapes using PCI device passthrough, and Control-Flow Integrity bypasses in HardenedBSD 12-CURRENT

NetBSD 8.2 is available, NextCloud on OpenBSD, X11 screen locking, NetBSD and RISC OS running parallel, community feedback about switching to BSD, and more.

The third release in the NetBSD-8 is now available.

This release includes all the security fixes in NetBSD-8 up until this point, and other fixes deemed important for stability.

NextCloud and OpenBSD are complementary to one another. NextCloud is an awesome, secure and private alternative for proprietary platforms, whereas OpenBSD forms the most secure and solid foundation to serve it on. Setting it up in the best way isn’t hard, especially using this step by step tutorial.

Back when this tutorial was initially written, things were different. The OpenBSD port relied on PHP 5.6 and there were no package updates. But the port improved (hats off, Gonzalo!) and package updates were introduced to the -stable branch (hats off, Solene!).

A rewrite of this tutorial was long overdue. Right now, it is written for 6.6 -stable and will be updated once 6.7 is released. If you have any questions or desire some help, feel free to reach out.

For years I’ve been using XScreenSaver as a default, but I recently learned about xsecurelock and re-evaluated my screen-saving requirements

I have been experimenting with running two systems at the same time on the RK3399 SoC.