BSD Now Video Feed

By Allan JudeTechnology

BSD Now Video Feed episodes

March 14, 2019Microkernel Failure | BSD Now 289
A kernel of failure, IPv6 fragmentation vulnerability in OpenBSD’s pf, a guide to the terminal, using a Yubikey for SSH public key authentication, FreeBSD desktop series, and more.
##Headlines
###A Kernel Of Failure -
How IBM bet big on the microkernel being the next big thing in operating systems back in the ’90s—and spent billions with little to show for it.
Today in Tedium: In the early 1990s, we had no idea where the computer industry was going, what the next generation would look like, or even what the driving factor would be. All the developers back then knew is that the operating systems available in server rooms or on desktop computers simply weren’t good enough, and that the next generation needed to be better—a lot better. This was easier said than done, but this problem for some reason seemed to rack the brains of one company more than any other: IBM. Throughout the decade, the company was associated with more overwrought thinking about operating systems than any other, with little to show for it in the end. The problem? It might have gotten caught up in kernel madness. Today’s Tedium explains IBM’s odd operating system fixation, and the belly flops it created.
###CVE-2019-5597IPv6 fragmentation vulnerability in OpenBSD Packet Filter
Packet Filter is OpenBSD’s service for filtering network traffic and performing Network Address Translation. Packet Filter is also capable of normalizing and conditioning TCP/IP traffic, as well as providing bandwidth control and packet prioritization.
Packet Filter has been a part of the GENERIC kernel since OpenBSD 5.0.Because other BSD variants import part of OpenBSD code, Packet Filter is also shipped with at least the following distributions that are affected in a lesser extent: FreeBSD, pfSense, OPNSense, Solaris.
Note that other distributions may also contain Packet Filter but due to the imported version they might not be vulnerable. This advisory covers the latest OpenBSD’s Packet Filter. For specific details about other distributions, please refer to the advisory of the affected product.
Kristof Provost, who maintains the port of pf in FreeBSD added a test for the vulnerability in FreeBSD head.
##News Roundup
###How I’m still not using GUIs in 2019: A guide to the terminal
TL;DR: Here are my dotfiles. Use them and have fun.
GUIs are bloatware. I’ve said it before. However, rather than just complaining about IDEs I’d like to provide an understandable guide to a much better alternative: the terminal.
IDE stands for Integrated Development Environment. This might be an accurate term, but when it comes to a real integrated development environment, the terminal is a lot better.
In this post, I’ll walk you through everything you need to start making your terminal a complete development environment: how to edit text efficiently, configure its appearance, run and combine a myriad of programs, and dynamically create, resize and close tabs and windows.
Don’t forget rule number one.
Whenever in doubt, read the manual.
###Using a Yubikey as smartcard for SSH public key authentication
SSH is an awesome tool. Logging into other machines securely is so pervasive to us sysadmins nowadays that few of us think about what’s going on underneath. Even more so once you start using the more advanced features such as the ssh-agent, agent-forwarding and ProxyJump. When doing so, care must be taken in order to not compromise one’s logins or ssh keys.
You might have heard of Yubikeys.
These are USB authentication devices that support several different modes: they can be used for OTP (One Time Password) authentication, they can store OpenPGP keys, be a 2-factor authentication token and they can act as a SmartCard.
In OpenBSD, you can use them for Login (with login_yubikey(8)) with OTP since 2012, and there are many descriptions available(1) how to set this up.
###The 18 Part FreeBSD Desktop Series by Vermaden
FreeBSD Desktop – Part 1 – Simplified Boot
FreeBSD Desktop – Part 2 – Install (FreeBSD 11)
FreeBSD Desktop – Part 2.1 – Install FreeBSD 12
FreeBSD Desktop – Part 3 – X11 Window System
FreeBSD Desktop – Part 4 – Key Components – Window Manager
FreeBSD Desktop – Part 5 – Key Components – Status Bar
FreeBSD Desktop – Part 6 – Key Components – Task Bar
FreeBSD Desktop – Part 7 – Key Components – Wallpaper Handling
FreeBSD Desktop – Part 8 – Key Components – Application Launcher
FreeBSD Desktop – Part 9 – Key Components – Keyboard/Mouse Shortcuts
FreeBSD Desktop – Part 10 – Key Components – Locking Solution
FreeBSD Desktop – Part 11 – Key Components – Blue Light Spectrum Suppress
FreeBSD Desktop – Part 12 – Configuration – Openbox
FreeBSD Desktop – Part 13 – Configuration – Dzen2
FreeBSD Desktop – Part 14 – Configuration – Tint2
FreeBSD Desktop – Part 15 – Configuration – Fonts & Frameworks
FreeBSD Desktop – Part 16 – Configuration – Pause Any Application
FreeBSD Desktop – Part 17 – Automount Removable Media
##Beastie Bits
Drist with persistent SSH
ARPANET: Celebrating 50 Years Since “LO”
Termtris - a tetris game for ANSI/VT220 terminals
Poor Man’s CI - Hosted CI for BSD with shell scripting and duct tape
Why I use the IBM Model M keyboard that is older than me?
A privilege separated and sandboxed IPv6 Stateless Address AutoConfiguration Daemon
Google-free Android Setup
BSD Users Stockholm Meetup #6
##Feedback/Questions
Sijmen - Hi, and a Sunday afternoon toy project
Clint - Tuning ZFS for NVME
James - Show question
Send questions, comments, show ideas/topics, or stories you want mentioned on the show to [email protected]
Your browser does not support the HTML5 video tag.
0 min
March 07, 2019Turing Complete Sed | BSD Now 288
Software will never fix Spectre-type bugs, a proof that sed is Turing complete, managed jails using Bastille, new version of netdata, using grep with /dev/null, using GMail with mutt, and more.
##Headlines
###Google: Software is never going to be able to fix Spectre-type bugs
Spectre is here to stay: An analysis of side-channels and speculative execution
Researchers from Google investigating the scope and impact of the Spectre attack have published a paper asserting that Spectre-like vulnerabilities are likely to be a continued feature of processors and, further, that software-based techniques for protecting against them will impose a high performance cost. And whatever the cost, the researchers continue, the software will be inadequate—some Spectre flaws don’t appear to have any effective software-based defense. As such, Spectre is going to be a continued feature of the computing landscape, with no straightforward resolution.
The discovery and development of the Meltdown and Spectre attacks was undoubtedly the big security story of 2018. First revealed last January, new variants and related discoveries were made throughout the rest of the year. Both attacks rely on discrepancies between the theoretical architectural behavior of a processor—the documented behavior that programmers depend on and write their programs against—and the real behavior of implementations.
Specifically, modern processors all perform speculative execution; they make assumptions about, for example, a value being read from memory or whether an if condition is true or false, and they allow their execution to run ahead based on these assumptions. If the assumptions are correct, the speculated results are kept; if it isn’t, the speculated results are discarded and the processor redoes the calculation. Speculative execution is not an architectural feature of the processor; it’s a feature of implementations, and so it’s supposed to be entirely invisible to running programs. When the processor discards the bad speculation, it should be as if the speculation never even happened.
###A proof that Unix utility sed is Turing complete
Many people are surprised when they hear that sed is Turing complete. How come a text filtering program is Turing complete, they wonder. Turns out sed is a tiny assembly language that has a comparison operation, a branching operation and a temporary buffer. These operations make sed Turing complete.
I first learned about this from Christophe Blaess. His proof is by construction – he wrote a Turing machine in sed (download turing.sed). As any programming language that can implement a Turing machine is Turing complete we must conclude that sed is also Turing complete.
Christophe offers his own introduction to Turing machines and a description of how his sed implementation works in his article Implementation of a Turing Machine as a sed Script.
Christophe isn’t the first person to realize that sed is almost a general purpose programming language. People have written tetris, sokoban and many other programs in sed. Take a look at these:
Tetris
Sokoban (game)
Calculator
##News Roundup
###Bastille helps you quickly create and manage FreeBSD Jails.
Bastille helps you quickly create and manage FreeBSD Jails.
Jails are extremely lightweight containers that provide a full-featured UNIX-like operating system inside. These containers can be used for software development, rapid testing, and secure production Internet services.
Bastille provides an interface to create, manage and destroy these secure virtualized environments.
Current version: 0.3.20190204-beta.
Shell Script Source here: https://github.com/BastilleBSD/bastille/blob/master/usr/local/bin/bastille
###netdata v1.12 released
Netdata is distributed, real-time, performance and health monitoring for systems and applications. It is a highly optimized monitoring agent you install on all your systems and containers.
Netdata provides unparalleled insights, in real-time, of everything happening on the systems it runs (including web servers, databases, applications), using highly interactive web dashboards. It can run autonomously, without any third party components, or it can be integrated to existing monitoring tool chains (Prometheus, Graphite, OpenTSDB, Kafka, Grafana, etc).
Netdata is fast and efficient, designed to permanently run on all systems (physical & virtual servers, containers, IoT devices), without disrupting their core function.
Patch release 1.12.1 contains 22 bug fixes and 8 improvements.
###Using grep with /dev/null, an old Unix trick
Every so often I will find myself writing a grep invocation like this:
find .... -exec grep /dev/null '{}' '+'
The peculiar presence of /dev/null here is an old Unix trick that is designed to force grep to always print out file names, even if your find only matches one file, by always insuring that grep has at least two files as arguments. You can wind up wanting to do the same thing with a direct use of grep if you’re not certain how many files your wildcard may match.
###USING GMAIL WITH MUTT
I recently switched to using mutt for email and while setting up mutt to use imap is pretty straightforward, this tutorial will also document some advanced concepts such as encrypting your account password and sending emails from a different From address.
This tutorial assumes that you have some familiarity with using mutt and have installed it with sidebar support (sudo apt-get install mutt-patched for the ubuntu folks) and are comfortable with editing your muttrc.
If you would just like to skip to the end, my mutt configuration file can be found here.
##Beastie Bits
An Extensive UNIX Timeline
Garbage.fm - OEF
brk() to sbrk()
Fred models, found again
Kafe: Can OS Kernels Forward Packets Fast Enough for Software Routers?
ARPANET: Celebrating 50 Years Since “LO”
##Feedback/Questions
Pablo - Topic suggestion: FreeBSD on a Laptop as daily driver
Ron - ZFS on the fly compression and seek
Dave - two zpool, or not two zpool, that is the question
Send questions, comments, show ideas/topics, or stories you want mentioned on the show to [email protected]
Your browser does not support the HTML5 video tag.
0 min
February 28, 2019rc.d in NetBSD | BSD Now 287
Design and Implementation of NetBSD’s rc.d system, first impressions of Project Trident 18.12, PXE booting a FreeBSD disk image, middle mouse button pasting, NetBSD gains hardware accelerated virtualization, and more.
##Headlines
###The Design and Implementation of the NetBSD rc.d system
Abstract
In this paper I cover the design and implementation of the rc.d system start-up mechanism in NetBSD 1.5, which replaced the monolithic /etc/rc start-up file inherited from 4.4BSD. Topics covered include a history of various UNIX start-up mechanisms (including NetBSD prior to 1.5), design considerations that evolved over six years of discussions, implementation details, an examination of the human issues that occurred during the design and implementation, as well as future directions for the system.
Introduction
NetBSD recently converted from the traditional 4.4BSD monolithic /etc/rc start-up script to an /etc/rc.d mechanism, where there is a separate script to manage each service or daemon, and these scripts are executed in a specific order at system boot.
This paper covers the motivation, design and implementation of the rc.d system; from the history of what NetBSD had before to the system that NetBSD 1.5 shipped with in December 2000, as well as future directions.
The changes were contentious and generated some of the liveliest discussions about any feature change ever made in NetBSD. Parts of those discussions will be covered to provide insight into some of the design and implementation decisions.
History
There is great diversity in the system start-up mechanisms used by various UNIX variants. A few of the more pertinent schemes are detailed below. As NetBSD is derived from 4.4BSD, it follows that a description of the latter’s method is relevant. Solaris’ start-up method is also detailed, as it is the most common System V UNIX variant.
###First impressions of Project Trident 18.12
Project Trident (hereafter referred to as Trident) is a desktop operating system based on TrueOS. Trident takes the rolling base platform of TrueOS, which is in turn based on FreeBSD’s development branch, and combines it with the Lumina desktop environment.
+Installing
The debut release of Trident is available as a 4.1GB download that can be burned to a disc or transferred to a USB thumb drive. Booting from the Trident media brings up a graphical interface and automatically launches the project’s system installer. Down the left side of the display there are buttons we can click to show hardware information and configuration options. These buttons let us know if our wireless card and video card are compatible with Trident and give us a chance to change our preferred language and keyboard layout. At the bottom of the screen we find buttons that will open a terminal or shutdown the computer.
Early impressions
Trident boots to a graphical login screen where we can sign into the Lumina desktop or a minimal Fluxbox session. Lumina, by default, uses Fluxbox as its window manager. The Lumina desktop places its panel along the bottom of the screen and an application menu sits in the bottom-left corner. On the desktop we find icons for opening the software manager, launching the Falkon web browser, running the VLC media player, opening the Control Panel and adjusting the Lumina theme.
The application menu has an unusual and compact layout. The menu shows just a search box and buttons for browsing applications, opening a file manager, accessing desktop settings and signing out. To see what applications are available we can click the Browse Applications entry, which opens a window in the menu where we can scroll through installed programs. This is a bit awkward since the display window is small and only shows a few items at a time.
Early on I found it is possible to swap out the default “Start menu” with an alternative “Application menu” through the Panels configuration tool. This alternative menu offers a classic tree-style application menu. I found the latter menu easier to navigate as it expands to show all the applications in a selected category.
Conclusions
I have a lot of mixed feelings and impressions when it comes to Trident. On the one hand, the operating system has some great technology under the hook. It has cutting edge packages from the FreeBSD ecosystem, we have easy access to ZFS, boot environments, and lots of open source packages. Hardware support, at least on my physical workstation, was solid and the Lumina desktop is flexible.
##News Roundup
###PXE booting of a FreeBSD disk image
I had to set up a regression and network performance lab. This lab will be managed by a Jenkins, but the first step is to understand how to boot a FreeBSD disk by PXE. This article explains a simple way of doing it.
For information, all these steps were done using 2 PC Engines APU2 (upgraded with latest BIOS for iPXE support), so it’s a headless (serial port only, this can be IPMI SoL with different hardware) .
THE BIG PICTURE
Before explaining all steps and command line, here is the full big picture of the final process.
###Why I like middle mouse button paste in xterm so much
In my entry about how touchpads are not mice, I mused that one of the things I should do on my laptop was insure that I had a keyboard binding for paste, since middle mouse button is one of the harder multi-finger gestures to land on a touchpad. Kurt Mosiejczuk recently left a comment there where they said:
Shift-Insert is a keyboard equivalent for paste that is in default xterm (at least OpenBSD xterm, and putty on Windows too). I use that most of the time now as it seems less… trigger-happy than right click paste.
This sparked some thoughts, because I can’t imagine giving up middle mouse paste if I have a real choice. I had earlier seen shift-insert mentioned in other commentary on my entry and so have tried a bit to use it on my laptop, and it hasn’t really felt great even there; on my desktops, it’s even less appealing (I tried shift-insert out there to confirm that it did work in my set of wacky X resources).
In thinking about why this is, I came to the obvious realization about why all of this is so. I like middle mouse button paste in normal usage because it’s so convenient, because almost all of the time my hand is already on the mouse. And the reason my hand is already on the mouse is because I’ve just used the mouse to shift focus to the window I want to paste into. Even on my laptop, my right hand is usually away from the keyboard as I move the mouse pointer on the touchpad, making shift-Insert at least somewhat awkward.
###NetBSD Gains Hardware Accelerated Virtualization
NetBSD Virtual Machine Monitor
NVMM provides hardware-accelerated virtualization support for NetBSD. It is made of an ~MI frontend, to which MD backends can be plugged. A virtualization API is shipped via libnvmm, that allows to easily create and manage virtual machines via NVMM. Two additional components are shipped as demonstrators, toyvirt and smallkern: the former is a toy virtualizer, that executes in a VM the 64bit ELF binary given as argument, the latter is an example of such binary.
##Beastie Bits
SoloBSD 19.02-STABLE
Project Trident 18.12-U5 available
“Sudo Mastery, Second Edition” and Cover Art
MKSANITIZER - bug detector software integration with the NetBSD userland
Darn kids nowadays… back in my day we drew rude symbols like normal people. {{top two comments}}
ShellCheck
finds bugs in your shell scripts.
Old School Sean - A history of UNIX
##Feedback/Questions
Ales - OpenBSD, FreeNAS, OpenZFS questions
Malcolm - Thoughts on Pgsql + ZFS thread?
Brad - Boot Environments in FreeBSD
Send questions, comments, show ideas/topics, or stories you want mentioned on the show to [email protected]
Your browser does not support the HTML5 video tag.
0 min
February 21, 2019Old Machine Revival | BSD Now 286
Adding glue to a desktop environment, flashing the BIOS on a PC Engine, revive a Cisco IDS into a capable OpenBSD computer, An OpenBSD WindowMaker desktop, RealTime data compression, the love for pipes, and more.
##Headlines
###Adding Glue To a Desktop Environment
In this article we will put some light on a lot of tools used in the world of Unix desktop environment customization, particularly regarding wmctrl, wmutils, xev, xtruss, xwininfo, xprop, xdotools, xdo, sxhkd, xbindkeys, speckeysd, xchainkeys, alttab, triggerhappy, gTile, gidmgr, keynav, and more. If those don’t make sense then this article will help. Let’s hope this can open your mind to new possibilities.
With that in mind we can wonder if what’s actually needed from a window manager, presentation and operation, can be split up and complemented with other tools. We can also start thinking laterally, the communication and interaction between the different components of the environment. We have the freedom to do so because the X protocol is transparent and components usually implement many standards for interfacing between windows. It’s like gluing parts together to create a desktop environment.
The tools we’ll talk about fall into one of those categories:
Debugging
Window manipulation
Simulation of interaction
Extended manipulation
Hotkey daemon
Layout manager
###Flashing the BIOS on the PC Engines APU4c4
I absolutely love the PC Engines APU devices. I use them for testing HardenedBSD experimental features in more constrained 64-bit environments and firewalls. Their USB and mSATA ports have a few quirks, and I bumped up against a major quirk that required flashing a different BIOS as a workaround. This article details the hacky way in which I went about doing that.
What prompted this article is that something in either the CAM or GEOM layer in FreeBSD 11.2 caused the mSATA to hang, preventing file writes. OPNsense 18.7 uses FreeBSD 11.1 whereas the recently-released OPNsense 19.1 uses HardenedBSD 11.2 (based on FreeBSD 11.2). I reached out to PC Engines directly, and they let me know that the issue is a known BIOS issue. Flashing the “legacy” BIOS series would provide me with a working system.
It also just so happens that a new “legacy” BIOS version was just released which turns on ECC mode for the RAM. So, I get a working OPNsense install AND ECC RAM! I’ll have one bird for dinner, the other for dessert.
Though I’m using an APU4, these instructions should work for the other APU devices. The BIOS ROM download URLs should be changed to reflect the device you’re targeting along with the BIOS version you wish to deploy.
SPECIAL NOTE: There be dragons! I’m primarily writing this article to document the procedure for my own purposes. My memory tends to be pretty faulty these days. So, if something goes wrong, please do not hold me responsible. You’re the one at the keyboard. ;)
VERY SPECIAL NOTE: We’ll use the mSATA drive for swap space, just in case. Should the swap space be used, it will destroy whatever is on the disk.
##News Roundup
###Revive a Cisco IDS into a capable OpenBSD computer!
Even though Cisco equipment is very capable, it tends to become End-of-Life before you can say “planned obsolescence”. Websites become bigger, bandwidths increase, and as a side effect of those “improvements”, routers, firewalls, and in this case, intrusion prevention systems get old quicker and quicker.
Apparently, this was also the case for the Cisco IDS-4215 Intrusion Detection Sensor that I was given a few months ago.
I’m not too proud to admit that at first, I didn’t care about the machine itself, but rather about the add-on PCI network card with 4 Fast Ethernet interfaces. The sensor has obviously seen better days, as it had a broken front panel and needed some cleaning, but upon a closer inspection under the hood (which is held closed by the 4 screws on top), this IDS consists of an embedded Celeron PC with two onboard Ethernet cards, a 2.5″ IDE hard disk, a CF card, and 2 PCI expansion slots (more on them later). Oh, and don’t forget the nasty server-grade fan, which pushed very little air for the noise it was making.
###An OpenBSD desktop using WindowMaker
Since I started using *N?X, I’ve regularly used WindowMaker. I’ve always liked the look and feel, the dock system and the dockapps. It may look a bit oldish nowadays. And that’s enough to try to change this. So here it is, a 2019 flavored WindowMaker Desktop, running on OpenBSD 6.4/amd64.
This configuration uses the Nord color-scheme, the Adapta-Nokto-Eta GTK theme and the Moblin Unofficial Icons icon set. I did remove applications icons. I just don’t need them on the bottom of the screen as I heavily use “F11” to pop-up the windows list. To be able to do that and keep the dockapps, I tweaked my ~/GNUstep/Defaults/WMWindowAttributes and created a ~/GNUstep/Library/WindowMaker/Themes/Nord.themed/style.
And here it is, the NeXT OpenBSD Desktop!
###RealTime Data Compression
In a previous episode, we’ve seen that it is possible to create opaque types. However, creation and destruction of such type must be delegated to some dedicated functions, which themselves rely on dynamic allocation mechanisms.
Sometimes, it can be convenient to bypass the heap, and all its malloc() / free() shenanigans. Pushing a structure onto the stack, or within thread-local storage, are natural capabilities offered by a normal struct. It can be desirable at times.
The previously described opaque type is so secret that it has no size, hence is not suitable for such scenario.
Fortunately, static opaque types are possible.
The main idea is to create a “shell type”, with a known size and an alignment, able to host the target (private) structure.
For safer maintenance, the shell type and the target structure must be kept in sync, by using typically a static assert. It will ensure that the shell type is always large enough to host the target structure. This check is important to automatically detect future evolution of the target structure.
###For the Love of Pipes
My top used shell command is |. This is called a pipe.
In brief, the | allows for the output of one program (on the left) to become the input of another program (on the right). It is a way of connecting two commands together.
According to doc.cat-v.org/unix/pipes/, the origin of pipes came long before Unix. Pipes can be traced back to this note from Doug McIlroy in 1964
##Beastie Bits
Installation Notes for NetBSD/i386 0.9
Porting Zig to NetBSD - a fun, speedy port
NNN - Tiny, lightning fast, feature-packed file manager Release v2.3
eta - A tool for monitoring progress and ETA of an arbitrary process
A FreeBSD User Tries Out…NetBSD 8.0
Faster vlan(4) forwarding?
FuguIta - OpenBSD 6.4 Live System
Adding Name-based hosting To Nginx on OpenBSD with Acme-Client
HOWTO set up QEMU with HAXM acceleration on NetBSD
README: gcc 7 switch coming to a port near you!
##BUG Calendar
ChiBUG, Chicago, USA: Tuesday, February 26th 18:00 at the Oak Park Library
CharmBUG, Baltimore, USA: Wednesday, February 27, 2019
19:30 at Columbia Ale House
NYC*BUG, New York, USA: Wednesday, March 6, 2019 18:45 at Suspenders
KnoxBUG, Knoxville, USA: Monday, February 25, 2019 - 18:00 at iX Systems offices
BSDPL, Warsaw, Poland: February 28, 2019 18:15 - 21:00 at Wheel Systems Office
##Feedback/Questions
Sam - Customizing OpenBSD ports source code
Frank - Rivalry Linux & BSD
Zach - mysql/mariadb tuning
Send questions, comments, show ideas/topics, or stories you want mentioned on the show to [email protected]
0 min
February 14, 2019BSD Strategy | BSD Now 285
Strategic thinking to keep FreeBSD relevant, reflecting on the soul of a new machine, 10GbE Benchmarks On Nine Linux Distros and FreeBSD, NetBSD integrating LLVM sanitizers in base, FreeNAS 11.2 distrowatch review, and more.
##Headlines
###Strategic thinking, or what I think what we need to do to keep FreeBSD relevant
Since I participate in the FreeBSD project there are from time to time some voices which say FreeBSD is dead, Linux is the way to go. Most of the time those voices are trolls, or people which do not really know what FreeBSD has to offer. Sometimes those voices wear blinders, they only see their own little world (were Linux just works fine) and do not see the big picture (like e.g. competition stimulates business, …) or even dare to look what FreeBSD has to offer.
Sometimes those voices raise a valid concern, and it is up to the FreeBSD project to filter out what would be beneficial. Recently there were some mails on the FreeBSD lists in the sense of “What about going into direction X?”. Some people just had the opinion that we should stay where we are. In my opinion this is similarly bad to blindly saying FreeBSD is dead and following the masses. It would mean stagnation. We should not hold people back in exploring new / different directions. Someone wants to write a kernel module in (a subset of) C++ or in Rust… well, go ahead, give it a try, we can put it into the Ports Collection and let people get experience with it.
This discussion on the mailinglists also triggered some kind of “where do we see us in the next years” / strategic thinking reflection. What I present here, is my very own opinion about things we in the FreeBSD project should look at, to stay relevant in the long term. To be able to put that into scope, I need to clarify what “relevant” means in this case.
FreeBSD is currently used by companies like Netflix, NetApp, Cisco, Juniper, and many others as a base for products or services. It is also used by end‐users as a work‐horse (e.g. mailservers, webservers, …). Staying relevant means in this context, to provide something which the user base is interested in to use and which makes it more easy / fast for the user base to deliver whatever they want or need to deliver than with another kind of system. And this in terms of time to market of a solution (time to deliver a service like a web‐/mail‐/whatever‐server or product), and in terms of performance (which not only means speed, but also security and reliability and …) of the solution.
I have categorized the list of items I think are important into (new) code/features, docs, polishing and project infrastructure. Links in the following usually point to documentation/HOWTOs/experiences for/with FreeBSD, and not to the canonical entry points of the projects or technologies. In a few cases the links point to an explanation in the wikipedia or to the website of the topic in question.
###Reflecting on The Soul of a New Machine
Long ago as an undergraduate, I found myself back home on a break from school, bored and with eyes wandering idly across a family bookshelf. At school, I had started to find a calling in computing systems, and now in the den, an old book suddenly caught my eye: Tracy Kidder’s The Soul of a New Machine. Taking it off the shelf, the book grabbed me from its first descriptions of Tom West, captivating me with the epic tale of the development of the Eagle at Data General. I — like so many before and after me — found the book to be life changing: by telling the stories of the people behind the machine, the book showed the creative passion among engineers that might otherwise appear anodyne, inspiring me to chart a course that might one day allow me to make a similar mark.
Since reading it over two decades ago, I have recommended The Soul of a Machine at essentially every opportunity, believing that it is a part of computing’s literary foundation — that it should be considered our Odyssey. Recently, I suggested it as beach reading to Jess Frazelle, and apparently with perfect timing: when I saw the book at the top of her vacation pile, I knew a fuse had been lit. I was delighted (though not at all surprised) to see Jess livetweet her admiration of the book, starting with the compelling prose, the lucid technical explanations and the visceral anecdotes — but then moving on to the deeper technical inspiration she found in the book. And as she reached the book’s crescendo, Jess felt its full power, causing her to reflect on the nature of engineering motivation.
Excited to see the effect of the book on Jess, I experienced a kind of reflected recommendation: I was inspired to (re-)read my own recommendation! Shortly after I started reading, I began to realize that (contrary to what I had been telling myself over the years!) I had not re-read the book in full since that first reading so many years ago. Rather, over the years I had merely revisited those sections that I remembered fondly. On the one hand, these sections are singular: the saga of engineers debugging a nasty I-cache data corruption issue; the young engineer who implements the simulator in an impossibly short amount of time because no one wanted to tell him that he was being impossibly ambitious; the engineer who, frustrated with a nanosecond-scale timing problem in the ALU that he designed, moved to a commune in Vermont, claiming a desire to deal with “no unit of time shorter than a season”. But by limiting myself to these passages, I was succumbing to the selection bias of my much younger self; re-reading the book now from start to finish has given new parts depth and meaning. Aspects that were more abstract to me as an undergraduate — from the organizational rivalries and absurdities of the industry to the complexities of West’s character and the tribulations of the team down the stretch — are now deeply evocative of concrete episodes of my own career.
See Article for rest…
##News Roundup
###Out-Of-The-Box 10GbE Network Benchmarks On Nine Linux Distributions Plus FreeBSD 12
Last week I started running some fresh 10GbE Linux networking performance benchmarks across a few different Linux distributions. That testing has now been extended to cover nine Linux distributions plus FreeBSD 12.0 to compare the out-of-the-box networking performance.
Tested this round alongside FreeBSD 12.0 was Antergos 19.1, CentOS 7, Clear Linux, Debian 9.6, Fedora Server 29, openSUSE Leap 15.0, openSUSE Tumbleweed, Ubuntu 18.04.1 LTS, and Ubuntu 18.10.
All of the tests were done with a Tyan S7106 1U server featuring two Intel Xeon Gold 6138 CPUs, 96GB of DDR4 system memory, and Samsung 970 EVO SSD. For the 10GbE connectivity on this server was an add-in HP NC523SFP PCIe adapter providing two 10Gb SPF+ ports using a QLogic 8214 controller.
Originally the plan as well was to include Windows Server 2016/2019. Unfortunately the QLogic driver download site was malfunctioning since Cavium’s acquisition of the company and the other Windows Server 2016 driver options not panning out and there not being a Windows Server 2019 option. So sadly that Windows testing was thwarted so I since started testing over with a Mellanox Connectx-2 10GbE NIC, which is well supported on Windows Server and so that testing is ongoing for the next article of Windows vs. Linux 10 Gigabit network performance plus some “tuned” Linux networking results too.
###Integration of the LLVM sanitizers with the NetBSD base system
Over the past month I’ve merged the LLVM compiler-rt sanitizers (LLVM svn r350590) with the base system. I’ve also managed to get a functional set of Makefile rules to build all of them, namely:
ASan, UBSan, TSan, MSan, libFuzzer, SafeStack, XRay.
In all supported variations and modes that are supported by the original LLVM compiler-rt package.
###Distrowatch FreeNAS 11.2 review
The project’s latest release is FreeNAS 11.2 and, at first, I nearly overlooked the new version because it appeared to be a minor point release. However, a lot of work went into the new version and 11.2 offers a lot of changes when compared next to 11.1, “including a major revamp of the web interface, support for self-encrypting drives, and new, backwards-compatible REST and WebSocket APIs. This update also introduces iocage for improved plugins and jails management and simplified plugin development.”
##Beastie Bits
Instructions for installing rEFInd to dual boot a computer with FreeBSD and windows (and possibly other OSes as well).
NetBSD desktop pt.6: “vi(1) editor, tmux and unicode $TERM”
Unix flowers
FreeBSD upgrade procedure using GPT
Pull-based Backups using OpenBSD base*
Developing WireGuard for NetBSD
OpenZFS User Conference, April 18-19, Norwalk CT
KnoxBug Feb 25th
##Feedback/Questions
Jake - C Programming
Farhan - Explanation of rtadvd
Nelson - Bug Bounties on Open-Source Software
Send questions, comments, show ideas/topics, or stories you want mentioned on the show to [email protected]
0 min
February 07, 2019FOSDEM 2019 | BSD Now 284
We recap FOSDEM 2019, FreeBSD Foundation January update, OPNsense 19.1 released, the hardware-assisted virtualization challenge, ZFS and GPL terror, ClonOS 19.01-RELEASE, and more.
##Headlines
###FOSDEM 2019 Recap
Allan and I were at FOSDEM 2019 in Brussels, Belgium over the weekend.
On the Friday before, we held a FreeBSD Devsummit in a hotel conference room, with 25 people attending. We talked about various topics of interest to the project. You can find the notes on the wiki page.
Saturday was the first day of FOSDEM. The FreeBSD Project had a table next to the Illumos Project again. A lot of people visited our table, asked questions, or just said “Hi, I watch BSDNow.tv every week”. We handed out a lot of stickers, pens, swag, and flyers. There was also a full day BSD devroom, with a variety of talks that were well attended.
In the main conference track, Allan held a talk explaining how the ZFS ARC works. A lot of people attended the talk and had more questions afterwards. Another well attended talk was by Jonathan Looney about Netflix and FreeBSD.
Sunday was another day in the same format, but no bsd devroom. A lot of people visited our table, developers and users alike. A lot of meeting and greeting went on.
Overall, FOSDEM was a great success with FreeBSD showing a lot of presence. Thanks to all the people who attended and talked to us. Special thanks to the people who helped out at the FreeBSD table and Rodrigo Osorio for running the BSD devroom again.
###FreeBSD Foundation Update, January 2019
Dear FreeBSD Community Member,
Happy New Year! It’s always exciting starting the new year with ambitious plans to support FreeBSD in new and existing areas. We achieved our fundraising goal for 2018, so we plan on funding a lot of work this year! Though it’s the new year, this newsletter highlights some of the work we accomplished in December. We also put together a list of technologies and features we are considering supporting, and are looking for feedback on what users want to help inform our 2019 development plans. Our advocacy and education efforts are in full swing as we prepare for upcoming conferences including FOSDEM, SANOG33, and SCaLE.
Finally, we created a year-end video to talk about the work we did in 2018. That in itself was an endeavor, so please take a few minutes to watch it! We’re working on improving the methods we use to inform the community on the work we are doing to support the Project, and are always open to feedback. Now, sit back, grab a refreshing beverage, and enjoy our newsletter!
Happy reading!!
Deb
###OPNsense 19.1 released
For more than four years now, OPNsense is driving innovation through modularising and hardening the open source firewall, with simple and reliable firmware upgrades, multi-language support, HardenedBSD security, fast adoption of upstream software updates as well as clear and stable 2-Clause BSD licensing.
The 19.1 release, nicknamed “Inspiring Iguana”, consists of a total of 620 individual changes since 18.7 came out 6 months ago, spread out over 12 intermediate releases including the recent release candidates. That is the average of 2 stable releases per month, security updates and important bug fixes included! If we had to pick a few highlights it would be: The firewall alias API is finally in place. The migration to HardenedBSD 11.2 has been completed. 2FA now works with a remote LDAP / local TOTP combination. And the OpenVPN client export was rewritten for full API support as well.
These are the most prominent changes since version 18.7:
fully functional firewall alias API
PIE firewall shaper support
firewall NAT rule logging support
2FA via LDAP-TOTP combination
WPAD / PAC and parent proxy support in the web proxy
P12 certificate export with custom passwords
Dpinger is now the default gateway monitor
ET Pro Telemetry edition plugin[2]
extended IPv6 DUID support
Dnsmasq DNSSEC support
OpenVPN client export API
Realtek NIC driver version 1.95
HardenedBSD 11.2, LibreSSL 2.7
Unbound 1.8, Suricata 4.1
Phalcon 3.4, Perl 5.28
firmware health check extended to cover all OS files, HTTPS mirror default
updates are browser cache-safe regarding CSS and JavaScript assets
collapsible side bar menu in the default theme
language updates for Chinese, Czech, French, German, Japanese, Portuguese and Russian
API backup export, Bind, Hardware widget, Nginx, Ntopng, VnStat and Dnscrypt-proxy plugins
Here are the full changes against version 19.1-RC2:
ipsec: add firewall interface as soon as phase 1 is enabled
ipsec: phase 1 selection GUI JavaScript compatibility fix
monit: widget improvements and bug fix (contributed by Frank Brendel)
ui: fix regression in single host or network subnet select in static pages
plugins: os-frr 1.7 updates OSFP outbound rules (contributed by Fabian Franz)
plugins: os-telegraf 1.7.4 fixes packet filter input
plugins: os-theme-rebellion 1.8.2 adds image colour invert
plugins: os-vnstat 1.1[3]
plugins: os-zabbix-agent now uses Zabbix version 4.0
src: revert mmc_calculate_clock() as HS200/HS400 support breaks legacy support
src: update sqlite3-3.20.0 to sqlite3-3.26.0[4]
src: import tzdata 2018h, 2018i[5]
src: avoid unsynchronized updates to kn_status[6]
ports: ca_root_nss 3.42
ports: dhcp6c 20190128 prevent rawops double-free (contributed by Team Rebellion)
ports: sudo patch to fix listpw=never[7]
##News Roundup
###The hardware-assisted virtualization challenge
Over two years ago, I made a pledge to use NetBSD as my sole OS and only operating system, and to resist booting into any other OS until I had implemented hardware-accelerated virtualization in the NetBSD kernel (the equivalent of Linux’ KVM, or Hyper-V).
Today, I am here to report: Mission Accomplished!
It’s been a long road, but we now have hardware-accelerated virtualization in the kernel! And while I had only initially planned to get Oracle VirtualBox working, I have with the help of the Intel HAXM engine (the same backend used for virtualization in Android Studio) and a qemu frontend, successfully managed to boot a range of mainstream operating systems.
###ZFS and GPL terror: How much freedom is there in Linux?
ZFS – the undesirable guest
ZFS is todays most advanced filesystem. It originated on the Solaris operating system and thanks to Sun’s decision to open it up, we have it available on quite a number of Unix-like operating systems. That’s just great! Great for everyone.
For everyone? Nope. There are people out there who don’t like ZFS. Which is totally fine, they don’t need to use it after all. But worse: There are people who actively hate ZFS and think that others should not use it. Ok, it’s nothing new that some random guys on the net are acting like assholes, trying to tell you what you must not do, right? Whoever has been online for more than a couple of days probably already got used to it. Unfortunately its still worse: One such spoilsport is Greg Kroah-Hartman, Linux guru and informal second-in-command after Linus Torvalds.
There have been some attempts to defend the stance of this kernel developer. One was to point at the fact that the “ZFS on Linux” (ZoL) port uses two kernel functions, __kernel_fpu_begin() and __kernel_fpu_end(), which have been deprecated for a very long time and that it makes sense to finally get rid of them since nothing in-kernel uses it anymore. Nobody is going to argue against that. The problem becomes clear by looking at the bigger picture, though:
The need for functions doing just what the old ones did has of course not vanished. The functions have been replaced with other ones. And those ones are deliberately made GPL-only. Yes, that’s right: There’s no technical reason whatsoever! It’s purely ideology – and it’s a terrible one.
###ClonOS 19.01-RELEASE
ClonOS is a turnkey Open Source platform based on FreeBSD and the CBSD framework. ClonOS offers a complete web UI for easily controlling, deploying and managing FreeBSD jails containers and Bhyve/Xen hyperviser virtual environments.
ClonOS is currently the only platform available which allow both Xen and Bhyve hypervisor to coexist on the same host. Being a FreeBSD base platform, ClonOS ability to create and manage jails allows you to run FreeBSD applications without losing performance.
Features:
easy management via web UI interface
live Bhyve migration [coming soon, roadmap]
Bhyve management (create, delete VM)
Xen management (create, delete VM) [coming soon, roadmap]
connection to the “physical” guest console via VNC from the browser or directly
Real time system monitoring
access to load statistics through SQLite3 and beanstalkd
support for ZFS features (cloning, snapshots)
import/export of virtual environments
public repository with virtual machine templates
puppet-based helpers for configuring popular services
ClonOS is a free open-source FreeBSD-based platform for virtual environments creation and management. In the core:
FreeBSD OS as hoster platform
bhyve(8) as hypervisor engine
Xen as hypervisor engine
vale(4) as Virtual Ethernet Switch
jail(8) as container engine
CBSD Project as management tools
Puppet as configuration management
##Beastie Bits
Florian Obser on unwind(8)
A low tech SMS gateway for fun and no profit
Netflix and FreeBSD : Using Open Source to Deliver Streaming Video
powerd++ 0.4.0 release
Is it time to rewrite the operating system in Rust?
Small change, big effect
Swedish BSD Meetup, Feb 19, 2019
Polish BSD User Group Meetup, Feb 21, 2019
##Feedback/Questions
Casey - Cool new Digital Ocean Feature
Morgan - Jail w/differnet version of FreeBSD
Brad - FreeBSD Installer
Send questions, comments, show ideas/topics, or stories you want mentioned on the show to [email protected]
0 min
January 31, 2019Graphical Interface-View | BSD Now 283
We’re at FOSDEM 2019 this week having fun. We’d never leave you in a lurch, so we have recorded an interview with Niclas Zeising of the FreeBSD graphics team for you. Enjoy.
##Interview - Niclas Zeising - [email protected] / @niclaszeising
Interview topic: FreeBSD Graphics Stack
BR: Welcome Niclas. Since this is your first time on BSDNow, can you tell us a bit about yourself and how you started with Unix/BSD?
AJ: What made you start working in the FreeBSD graphics stack?
BR: What is the current status with the FreeBSD graphics stack?
AJ: What challenges do you face in the FreeBSD graphics stack?
BR: How many people are working in the graphics team and what kind of help do you need there?
AJ: You’re also involved in FreeBSD ports and held a poudriere tutorial at last years EuroBSDcon. What kind of feedback did you get and will you give that tutorial again?
BR: You’ve been organizing the Stockholm BSD user group meeting. Can you tell us a bit about that, what’s involved, how is it structured?
AJ: What conferences do you go to where people could talk to you?
BR: Is there anything else you’d like to mention before we let you go?
##Feedback/Questions
Casey - TrueOS
Troels - zfs send vs zfs send -R
matclarke - Orphaned packages
Send questions, comments, show ideas/topics, or stories you want mentioned on the show to [email protected]
0 min
January 24, 2019Open the Rsync | BSD Now 282
Project Trident 18.12 released, Spotifyd on NetBSD, OPNsense 18.7.10 is available, Ultra EPYC AMD Powered Sun Ultra 24 Workstation, OpenRsync, LLD porting to NetBSD, and more.
##Headlines
###AsiaBSDCon 2019 Call for Papers
You have until Jan 30th to submit
Full paper requirement is relaxed a bit this year (this year ONLY!) due to the short submission window. You don’t need all 10-12 pages, but it is still preferred.
Send a message to [email protected] with your proposal. Could be either for a talk or a tutorial.
Two days of tutorials/devsummit and two days of conference during Sakura season in Tokyo, Japan
The conference is also looking for sponsors
If accepted, flight and hotel is paid for by the conference
###Project Trident 18.12 Released
Twitter account if you want to keep up on project news
Screenshots
Project Trident Community Telegram Channel
DistroWatch Page
LinuxActionNews Review
RoboNuggie’s in depth review
###Building Spotifyd on NetBSD
These are the steps I went through to build and run Spotifyd (this commit at the time of writing) on NetBSD AMD64. It’s a Spotify Connect client so it means I still need to control Spotify from another device (typically my phone), but the audio is played through my desktop… which is where my speakers and headphones are plugged in - it means I don’t have to unplug stuff and re-plug into my phone, work laptop, etc. This is 100% a “good enough for now solution” for me; I have had a quick play with the Go based microcontroller from spotcontrol and that allows a completely NetBSD only experience (although it is just an example application so doesn’t provide many features - great as a basis to build on though).
##News Roundup
###OPNsense 18.7.10 released
2019 means 19.1 is almost here. In the meantime accept this small
incremental update with goodies such as Suricata 4.1, custom passwords
for P12 certificate export as well as fresh fixes in the FreeBSD base.
A lot of cleanups went into this update to make sure there will be a
smooth transition to 19.1-RC for you early birds. We expect RC1 in 1-2
weeks and the final 19.1 on January 29.
###Introducing the Ultra EPYC AMD Powered Sun Ultra 24 Workstation
A few weeks ago, I got an itch to build a workstation with AMD EPYC. There are a few constraints. First, I needed a higher-clock part. Second, I knew the whole build would be focused more on being an ultra high-end workstation rather than simply utilizing gaming components. With that, I decided it was time to hit on a bit of nostalgia for our readers. Mainly, I wanted to do an homage to Sun Microsystems. Sun made the server gear that the industry ran on for years, and as a fun fact, if you go behind the 1 Hacker Way sign at Facebook’s campus, they left the Sun Microsystems logo. Seeing that made me wonder if we could do an ultimate AMD EPYC build in a Sun Microsystems workstation.
###OpenRsync
This is a clean-room implementation of rsync with a BSD (ISC) license. It is designed to be compatible with a modern rsync (3.1.3 is used for testing). It currently compiles and runs only on OpenBSD.
This project is still very new and very fast-moving.
It’s not ready for wide-spread testing. Or even narrow-spread beyond getting all of the bits to work. It’s not ready for strong attention. Or really any attention but by careful programming.
Many have asked about portability. We’re just not there yet, folks. But don’t worry, the system is easily portable. The hard part for porters is matching OpenBSD’s pledge and unveil.
###The first report on LLD porting
LLD is the link editor (linker) component of Clang toolchain. Its main advantage over GNU ld is much lower memory footprint, and linking speed. It is of specific interest to me since currently 8 GiB of memory are insufficient to link LLVM statically (which is the upstream default).
The first goal of LLD porting is to ensure that LLD can produce working NetBSD executables, and be used to build LLVM itself. Then, it is desirable to look into trying to build additional NetBSD components, and eventually into replacing /usr/bin/ld entirely with lld.
In this report, I would like to shortly summarize the issues I have found so far trying to use LLD on NetBSD.
###Ring in the new
It’s the second week of 2019 already, which means I’m curious what Nate is going to do with his series This week in usability … reset the numbering from week 1? That series is a great read, to keep up with all the little things that change in KDE source each week — aside from the release notes.
For the big ticket items of KDE on FreeBSD, you should read this blog instead.
In ports this week (mostly KDE, some unrelated):
KDE Plasma has been updated to the latest release, 5.14.5.
KDE Applications 18.12.1 were released today, so we’re right on top of them.
Marble was fixed for FreeBSD-running-on-Power9.
Musescore caught up on 18 months of releases.
Phonon updated to 4.10.1, along with its backends.
And in development, Qt WebEngine 5.12 has been prepared in the incongruously-named plasma-5.13 branch in Area51; that does contain all the latest bits described above, as well.
##Beastie Bits
NomadBSD 1.2-RC1 Released
ZFS - The First Enterprise Blockchain
Powersaving with DragonFly laptop
NetBSD reaches 100% reproducable builds
Potential Bhyve Web Interface?
LibGDX proof of concept on OpenBSD - Video
LiteCLI is a user-friendly CommandLine client for SQLite database
In honor of Donald Knuth’s 81 birthday Stanford uploaded 111 lectures on Youtube
Portland BSD Pizza Night - 2018-01-31 19:00 - Sweet Heart Pizza
Stockholm BSD February meetup
Polish BSD User Group: Jan 25 18:15 - 21:00
AsiaBSDcon 2019 CfP
##Feedback/Questions
Greg - VLANs and jails
Tara - ZFS on removable disks
Casey - Interview with Kirk McKusick
Send questions, comments, show ideas/topics, or stories you want mentioned on the show to [email protected]
0 min
January 17, 2019 EPYC Server Battle | BSD Now 281
SCP client vulnerabilities, BSDs vs Linux benchmarks on a Tyan EPYC Server, fame for the Unix inventors, Die IPv4, GhostBSD 18.12 released, Unix in pictures, and more.
0 min
January 10, 2019 FOSS Clothing | BSD Now 280
A EULA in FOSS clothing, NetBSD with more LLVM support, Thoughts on FreeBSD 12.0, FreeBSD Performance against Windows and Linux on Xeon, Microsoft shipping NetBSD, and more.
0 min

About BSD Now Video Feed

From the publisher's feed

BSD Now Video Feed

Share BSD Now Video Feed

Sign up to save your podcasts

BSD Now Video Feed

BSD Now Video Feed episodes

About BSD Now Video Feed