60 Seconds of WIP

I’m pushing to complete a first draft of OpenZFS Mastery in the next few weeks. Here’s a bit on ZFS and lightweight virtual machines.

FreeBSD and Linux have different approaches to lightweight virtualization. FreeBSD creates single namespaces that can be further subdivided, while Linux offers smaller namespaces that can be aggregated into larger entities. Neither is wrong.

You sponsoring OpenZFS Mastery would help me meet my bills.

Folks sometimes forget that OpenZFS Mastery has a co-author. Here’s a bit Allan Jude originally wrote.

ZFS has your back when dealing with these difficult users. The snapshot_limit and filesystem_limit properties allow you to restrict the number of snapshots or child filesystems that can be created under a dataset. Set these properties to one greater than the number of snapshots or datasets you want the user to create. If you set snapshot_limit to 10, the user can create nine snapshots. The tenth generates an error. The read-only filesystem_count and snapshot_count properties allow you to quickly see how many filesystems or snapshots exist, and compare that number to the limit. To contain the pure evil that is Lucas, here we restrain him to only two snapshots.

OpenZFS Mastery is looking for sponsors, and the Twisted Presents Kickstarter is open for a couple more days.

OpenZFS Mastery uses FreeBSD and Proxmox as reference platforms. Here’s a bit about why.

Proxmox is a Debian variant focused on virtualization, developed by Proxmox Server Solutions GmbH. It integrates ZFS as a primary filesystem and can install all the Debian packages. As we write this, Proxmox is the easiest way to get ZFS on Linux. While Proxmox is designed as a virtualization platform, it’s Debian plus other tools. You can install a desktop environment or any other Debian packages to tune the host as you wish.

If you want to use ZFS on another Linux, you will have struggles. Linux ZFS users often compromise to make their lives easier. Ubuntu can use ZFS for data disks, but relies on EXTFS for the root filesystem. Running root on ZFS in other Linuxes is an advanced skill. Some sysadmins get the root filesystem on ZFS, but live without boot environments. Upgrades might require initramfs chicanery or meticulous adherence to arcane unofficial procedures.

OpenZFS Mastery sponsorships are still open, and the Twisted Presents Kickstarter is live.

OpenZFS Mastery has reached chapter 9, on boot loaders.

The most tediously terrifying system administration task is an operating system upgrade. The new kernel might not boot the system, but that’s the least of your problems. What if a critical program requires an old version of a shared library? Maybe that new terminal mode is subtly incompatible with your application. Or perhaps the new linker strangles your mission-critical software. Sometimes problems aren’t immediately apparent but creeping damage becomes visible after a week or two, when reverting the upgrade becomes even more difficult. No matter what precautions you take or how much testing you perform, any upgrade can go bad.

Sysadmins have developed all sorts of tools and coping strategies to manage bad upgrades. Boot loaders help you quickly recover from bad kernels. Backups help you slowly recover from bad userlands. But none of these help you understand exactly what went wrong and duplicate the problem.

Unless you’re running ZFS.

OpenZFS Mastery sponsorships are still open.

There’s one story in Twisted Presents I haven’t read here, so that’s what you get.

This isn’t a fairytale, but: Once upon a time, we were all friends.

And it’s not going to end happily ever after. On this Christmas Eve, three tedious years after the disastrous college summer when Kurt betrayed us, Phil and I are finally making an ending.

Kurt thought so.

When will Twisted Presents launch? As soon as my life is in order to allow it.

I’ve just finished prepping the print version of Twisted Presents, so here’s a tidbit from one of the stories therein.

The first Hogswatch Eve after the accident, Merle had stopped on the way back from the hospital for a fifth of Makers’ Mark. He knew if he started drinking he’d never stop, but the next day was Hogswatch and he couldn’t think of any other way to survive the night. He woke the next morning with an empty bottle, an open sewer for a mouth, and a witch’s brew in his stomach.

But it was Hogswatch. He had to get to the clinic. Soon as he could. They were reading Night Watch. For when his voice gave out, he’d loaded a couple of movies onto a tablet. White Christmas and Gremlins. He hadn’t done much in the way of shopping, but he’d picked up a fruit and nut bar at the hospital gift shop a couple days ago. It was the first treat he’d ever bought her back in college, and all the feast he could handle.

The alternative was to lie here and cry. Again. Crying always made him feel worse.

I plan to launch Twisted Presents in time for you to get your books before Christmas. Because nothing says “Happy Holidays” like curses, firearms, and orcs.

I finished the OpenZFS Mastery chapter on volumes earlier this week.

Zvols support sparse volumes, also known as thin provisioning or overcommitting storage. A sparse volume is a volume that reserves less space than the volume actually needs. Sparse volumes permit allocating more space than the pool has available. You could, for example, create ten 1-exabyte sparse volumes on your 1 TiB dataset. So long as your volumes are never heavily used, nobody will notice that you’re overcommitted. Sparse volumes can be useful if you’re prepared to add more disks. Using a sparse volume means you can tell the Windows host that a disk is a specific large size, even if the underlying pool doesn’t have that much free space.

Sparse volumes are not generally recommended. If the pool fills up but the sparse volumes think they have space remaining, writes will fail with an “out of space” error. Guests will get errors they can’t hope to resolve. Windows in particular has low distress tolerance.

I’d appreciate your support keeping the rats fed as I finish writing this book.

I’m trying to finish up the OpenZFS Mastery chapter on disk space use today. Well, my part of it. I’m planning to dump–delegate, delegate–the section on deduplication to Allan. When you have a co-author expert, use them.

Zstd offers compression levels from 1 to 19. The default is zstd-3, a reasonable compromise between performance and efficiency. Writing data at higher compression levels require more CPU but might save more space. Zstd has a useful quirk where decompressing higher levels uses no more processing power than decompressing lower levels. If you have data that must be written once but read many times, increasing the compression level makes sense.

ZFS still supports gzip compression, but Zstd is always faster and achieves better results while using less processor and memory.

ZFS also offers the ZLE algorithm, for data with long runs of zeroes. Some file formats, such as GIFs and certain scientific data, include long runs of zeroes. If you have a dataset for such data, you might find ZLE compression useful.

How does one realistically tell if your data has long runs of zeroes? The simplest way I know is to pass it across a T1 with 23 channels set to b8zs but one channel stuck on AMI. Such a circuit cannot pass large all-zero packets. Sadly, this recommendation does not suit a book on filesystems.

I would appreciate your help keeping the lights on as I finish this book.

I’m writing about space usage OpenZFS Mastery.

ZFS offers tools to transparently increase the amount of data you can fit on a disk. Rather than requiring the system administrator to compress and decompress individual files, ZFS uses efficient compression at the filesystem level. While compressing logs during rotation makes the files look smaller in ls(1), it doesn’t change how many disk blocks those logs use. Snapshots and clones let you duplicate entire datasets with only a tiny amount of space. ZFS can also perform deduplication of files, improving disk usage at the cost of memory. We’ll see how to evaluate these options and determine when to use each.

If ZFS is so transparent, why is space usage complicated? Child datasets affect their parents. Those space-efficient clones can grow, and snapshots can prevent freeing space.

I’d appreciate your help keeping the lights on while I finish this book.

Probably jinxing myself by posting this, but: here’s a chunk from Twisted Presents.

Three nights ago had been the Longest Dark, the strongest night of the year. A night for the clan to come together and declare the year’s struggles, to mark the end of feuds and wars. Even living clanless and bereft, Mha had caught a whisper that her youngest daughter was in danger and had charged into battle. The magnificent fight had given Mha fresh cuts. She had defeated an elven wizard. Other orcs had honored her. December knew the truth, though. A daughter claimed by another clan no longer belonged to her mother, but she’d fought for her anyway. She’d had a flimsy cowhide of an excuse, but excuses weighed nothing against ice.

Soon she’d return to the barn. Back to her coal-fed stove and the sepulchral groom’s quarters where she rotted her days. For now, she bore December’s castigation.

Mha’s night-sight made the farmhouse’s lingering warmth a red rectangle. Humans had a holiday a few days after the Longest Dark, a holiday that Thorn’s neglectful nephew could not ignore. Thorn would return tomorrow but tonight, with nobody to feed the furnace, the snug farmhouse cooled.

Want to know when this will hit the world? Watch the Kickstarter.