NetBSD 8.0 available, FreeBSD on Scaleway’s ARM64 VPS, encrypted backups with OpenBSD, Dragonfly server storage upgrade, zpool checkpoints, g2k18 hackathon reports, and more.

##Headlines

The NetBSD Project is pleased to announce NetBSD 8.0, the sixteenth major release of the NetBSD operating system.

This release brings stability improvements, hundreds of bug fixes, and many new features.

Some highlights of the NetBSD 8.0 release are:

USB stack rework, USB3 support added.

In-kernel audio mixer (audio_system(9)).

Reproducible builds (MKREPRO, see mk.conf(5)).

Full userland debug information (MKDEBUG, see mk.conf(5)) available. While most install media do not come with them (for size reasons), the debug and xdebug sets can be downloaded and extracted as needed later. They provide full symbol information for all base system and X binaries and libraries and allow better error reporting and (userland) crash analysis.

PaX MPROTECT (W^X) memory protection enforced by default on some architectures with fine-grained memory protection and suitable ELF formats: i386, amd64, evbarm, landisk.

PaX ASLR (Address Space Layout Randomization) enabled by default on: i386, amd64, evbarm, landisk, sparc64.

Position independent executables by default for userland on: i386, amd64, arm, m68k, mips, sh3, sparc64.

A new socket layer can(4) has been added for communication of devices on a CAN bus.

A special pseudo interface ipsecif(4) for route-based VPNs has been added.

Parts of the network stack have been made MP-safe. The kernel option NET_MPSAFE is required to enable this.

Hardening of the network stack in general.

Various WAPBL (the NetBSD file system “log” option) stability and performance improvements.

Specific to i386 and amd64 CPUs:

Meltdown mitigation: SVS (Separate Virtual Space), enabled by default.

SpectreV2 mitigation: retpoline (support in gcc), used by default for kernels. Other hardware mitigations are also available.

SpectreV4 mitigations available for Intel and AMD.

PopSS workaround: user access to debug registers is turned off by default.

Lazy FPU saving disabled on vulnerable Intel CPUs (“eagerfpu”).

SMAP support.

Improvement and hardening of the memory layout: W^X, fewer writable pages, better consistency, better performance.

(U)EFI bootloader.

Many evbarm kernels now use FDT (flat device tree) information (loadable at boot time from an external file) for device configuration, the number of kernels has decreased but the number of boards has vastly increased.

Lots of updates to 3rd party software included:

GCC 5.5 with support for Address Sanitizer and Undefined Behavior Sanitizer

GDB 7.12

GNU binutils 2.27

Clang/LLVM 3.8.1

OpenSSH 7.6

OpenSSL 1.0.2k

mdocml 1.14.1

acpica 20170303

ntp 4.2.8p11-o

dhcpcd 7.0.6

Lua 5.3.4

###Running FreeBSD on the ARM64 VPS from Scaleway

I’ve been thinking about this 6 since 2017, but only yesterday signed up for an account and played around with the ARM64 offering.

Now reconnect to ssh from a second terminal (note: rm the connection file if you use ControlPersist in ssh config), then exit the old session. Kill the old sshd process, restart or stop the rest of the stuff using the old disk:

Check that nothing is touching /oldroot:

There will probably be an old dbus-daemon, kill it.

(Look for the newest snapshot, don’t copy paste the July 19 link above if you’re reading this in the future. Actually maybe use a release instead of CURRENT…)

And reboot. (You might actually want to hard reboot here: for some reason on the first reboot from Linux, pressing the any-key to enter the prompt in the loader hangs the console for me.)

I didn’t have to go into the ESC menu and choose the local disk in the boot manager, it seems to boot from disk automatically.

Now we’re in the FreeBSD EFI loader.

Ignore the warning about comconsole not being a valid console.

(UPD: shouldn’t be necessary in the next snapshot)

Now it’s a regular installation process!

(In this example, I set up ZFS with a beadm-compatible layout which allows me to use Boot Environments.)

In the post-install chroot shell, fix some configs like so:

(Yeah, for some reason, the loader does not load zfs.ko’s dependency opensolaris.ko automatically here. idk what even. It does on my desktop and laptop.)

Now you can reboot into the installed system!!

Here’s how you can set up IPv6 (and root’s ssh key) auto configuration on boot:

And to fix incoming TCP connections, configure the DHCP client to change the broadcast address:

echo 'interface "vtnet0" { supersede broadcast-address 255.255.255.255; }' >> /etc/dhclient.conf

** Digital Ocean **

###Easy encrypted backups on OpenBSD with base tools

Today’s topic is “Encrypted backups” using only OpenBSD base tools. I am planning to write a bigger article later about backups but it’s a wide topic with a lot of software to cover and a lot of explanations about the differents uses cases, needs, issues an solutions. Here I will stick on explaining how to make reliable backups for an OpenBSD system (my laptop).

My process is to make a huge dump of level 0 and keep it on a remote server, then, once a week I make a level 1 backup which will contain everything changed since the last dump of level 0, and everyday I do a level 2 backup of my files. The level 2 will contain latest files and the files changing a lot, which are often the most interesting. The level 1 backup is important because it will offload a lot of changes for the level 2.

##News Roundup

Last month we did some storage upgrades, particularly of internet-facing machines for package and OS distribution. Yesterday we did a number of additional upgrades, described below. All using funds generously donated by everyone!

The main repository server received a 2TB SSD to replace the HDDs it was using before. This will improve access to a number of things maintained by this server, including the mail archives, and gives the main repo server more breathing room for repository expansion. Space was at a premium before. Now there’s plenty.

Monster, the quad socket opteron which we currently use as the database builder and repository that we export to our public grok service (grok.dragonflybsd.org) received a 512G SSD to add swap space for swapcache, to help cache the grok meta-data. It now has 600GB of swapcache configured. Over the next few weeks we will also be changing the grok updates to ping-pong between the two 4TB data drives it received in the last upgrade so we can do concurrent updates and web accesses without them tripping over each other performance-wise.

The main developer box, Leaf, received a 2TB SSD and we are currently in the midst of migrating all the developer accounts in /home and /build from its old HDDs to its new SSD. This machine serves developer repos, developer web stuff, our home page and wiki, etc, so those will become snappier as well.

Hard drives are becoming real dinosaurs. We still have a few left from the old days but in terms of active use the only HDDs we feel we really need to keep now are the ones we use for backups and grok data, owing to the amount of storage needed for those functions.

Five years ago when we received the blade server that now sits in the colo, we had a small 256G SSD for root on every blade, and everything else used HDDs. To make things operate smoothly, most of that 256G root SSD was assigned to swapcache (200G of it, in fact, in most cases). Even just 2 years ago replacing all those HDDs with SSDs, even just the ones being used to actively serve data and support developers, would have been cost prohibitive. But today it isn’t and the only HDDs we really need anywhere are for backups or certain very large bits of bulk data (aka the grok source repository and index). The way things are going, even the backup drives will probably become SSDs over the next two years.

###iX ad spot

###zpool checkpoints

In March, to FreeBSD landed a very interesting feature called ‘zpool checkpoints’. Before we jump straight into the topic, let’s take a step back and look at another ZFS feature called ‘snapshot’. Snapshot allows us to create an image of our single file systems. This gives us the option to modify data on the dataset without the fear of losing some data.

A very good example of how to use ZFS snapshot is during an upgrade of database schema. Let us consider a situation where we have a few scripts which change our schema. Sometimes we are unable to upgrade in one transaction (for example, when we attempt to alter a table and then update it in single transaction). If our database is on dataset, we can just snapshot it, and if something goes wrong, simply rollback the file system to its previous state.

The problem with snapshot is that it works only on a single dataset. If we added some dataset, we wouldn’t then be able to create the snapshot which would rollback that operation. The same with changing the attributes of a dataset. If we change the compression on the dataset, we cannot rollback it. We would need to change that manually.

Another interesting problem involves upgrading the whole operating system when we upgrade system with a new ZFS version. What if we start upgrading our dataset and our kernel begins to crash? (If you use FreeBSD, I doubt you will ever have had that experience but still…). If we rollback to the old kernel, there is a chance the dataset will stop working because the new kernel doesn’t know how to use the new features.

Zpool checkpoints is the solution to all those problems. Instead of taking a single snapshot of the dataset, we can now take a snapshot of the whole pool. That means we will not only rollback the data but also all the metadata. If we rewind to the checkpoint, all our ZFS properties will be rolled back; the upgrade will be rolledback, and even the creation/deletion of the dataset, and the snapshot, will be rolledback.

zpool checkpoint

zpool import -- rewind-to-checkpoint

zpool import --read-only=on --rewind-to-checkpoint

zpool checkpoint --discard or zpool checkpoint -d

For me, this feature is incredibly useful, especially when upgrading an operating system, or when I need to experiment with additional data sets. If you speak Polish, I have some additional information for you. During the first Polish BSD user group meeting, I had the opportunity to give a short talk about this feature. Here you find the video of that talk, and here is the slideshow.

I would like to offer my thanks to Serapheim Dimitropoulos for developing this feature, and for being so kind in sharing with me so many of its intricacies. If you are interested in knowing more about the technical details of this feature, you should check out Serapheim’s blog, and his video about checkpoints.

###g2k18 Reports

##Beastie Bits

Tarsnap

##Feedback/Questions