BSD Now

Arcan and OpenBSD, running OpenBSD 6.3 on RPI 3, why C is not a low-level language, HardenedBSD switching back to OpenSSL, how the Internet was almost broken, EuroBSDcon CfP is out, and the BSDCan 2018 schedule is available.

Let me preface this by saying that this is a (very) long and medium-rare technical article about the security considerations and minutiae of porting (most of) the Arcan ecosystem to work under OpenBSD. The main point of this article is not so much flirting with the OpenBSD crowd or adding further noise to software engineering topics, but to go through the special considerations that had to be taken, as notes to anyone else that decides to go down this overgrown and lonesome trail, or are curious about some less than obvious differences between how these things “work” on Linux vs. other parts of the world.

A disclaimer is also that most of this have been discovered by experimentation and combining bits and pieces scattered in everything from Xorg code to man pages, there may be smarter ways to solve some of the problems mentioned – this is just the best I could find within the time allotted. I’d be happy to be corrected, in patch/pull request form that is 😉

Each section will start with a short rant-like explanation of how it works in Linux, and what the translation to OpenBSD involved or, in the cases that are still partly or fully missing, will require. The topics that will be covered this time are:

Installing the OpenBSD on raspberry pi 3 is very easy and well documented which almost convinced me of not writing about it, but still I felt like it may help somebody new to the project (But again I really recommend reading the document if you are interested and have the time).

Note: I'm always running snapshots and recommend anybody to do it as well. But the snapshots links will change to the next version every 6 month, so I changed the links to the 6.3 version to keep the blog post valid over times. If you're familiar to the OpenBSD flavors, feel free to use the snapshots links instead.

Due to the lack of driver, the OpenBSD can not boot directly from the SD Card yet, So we'll need an USB Stick for the installtion target aside the SD Card for the U-Boot and installer. Also, a Serial Console connection is required. I Used a PL2303 USB to Serial (TTL) adapter connected to my Laptop via USB port and connected to the Raspberry via TX, RX and GND pins.

iXsystems

Every month or so, someone will ask me what happened to Larrabee and why it failed so badly. And I then try to explain to them that not only didn't it fail, it was a pretty huge success. And they are understandably very puzzled by this, because in the public consciousness Larrabee was like the Itanic and the SPU rolled into one, wasn't it? Well, not quite. So rather than explain it in person a whole bunch more times, I thought I should write it down.

This is not a history, and I'm going to skip a TON of details for brevity. One day I'll write the whole story down, because it's a pretty decent escapade with lots of fun characters. But not today. Today you just get the very start and the very end.

When I say "Larrabee" I mean all of Knights, all of MIC, all of Xeon Phi, all of the "Isle" cards - they're all exactly the same chip and the same people and the same software effort. Marketing seemed to dream up a new codeword every week, but there was only ever three chips:

That's it. There were some other codenames I've forgotten over the years, but they're all of one of the above chips. Behind all the marketing smoke and mirrors there were only three chips ever made (so far), and only four planned in total (we had a thing called LRB3 planned between KNC and KNL for a while). All of them are "Larrabee", whether they do graphics or not.

When Larrabee was originally conceived back in about 2005, it was called "SMAC", and its original goals were, from most to least important:

That ordering is important - in terms of engineering and focus, Larrabee was never primarily a graphics card. If Intel had wanted a kick-ass graphics card, they already had a very good graphics team begging to be allowed to build a nice big fat hot discrete GPU - and the Gen architecture is such that they'd build a great one, too. But Intel management didn't want one, and still doesn't. But if we were going to build Larrabee anyway, they wanted us to cover that market as well.

... the design of Larrabee was of a CPU with a very wide SIMD unit, designed above all to be a real grown-up CPU - coherent caches, well-ordered memory rules, good memory protection, true multitasking, real threads, runs Linux/FreeBSD, etc. Larrabee, in the form of KNC, went on to become the fastest supercomputer in the world for a couple of years, and it's still making a ton of money for Intel in the HPC market that it was designed for, fighting very nicely against the GPUs and other custom architectures. Its successor, KNL, is just being released right now (mid 2016) and should do very nicely in that space too. Remember - KNC is literally the same chip as LRB2. It has texture samplers and a video out port sitting on the die. They don't test them or turn them on or expose them to software, but they're still there - it's still a graphics-capable part.

But it's still actually running FreeBSD on that card, and under FreeBSD it's just running an x86 program called DirectXGfx (248 threads of it).

In the wake of the recent Meltdown and Spectre vulnerabilities, it's worth spending some time looking at root causes. Both of these vulnerabilities involved processors speculatively executing instructions past some kind of access check and allowing the attacker to observe the results via a side channel. The features that led to these vulnerabilities, along with several others, were added to let C programmers continue to believe they were programming in a low-level language, when this hasn't been the case for decades.

Processor vendors are not alone in this. Those of us working on C/C++ compilers have also participated.

Computer science pioneer Alan Perlis defined low-level languages this way: "A programming language is low level when its programs require attention to the irrelevant."

While, yes, this definition applies to C, it does not capture what people desire in a low-level language. Various attributes cause people to regard a language as low-level. Think of programming languages as belonging on a continuum, with assembly at one end and the interface to the Starship Enterprise's computer at the other. Low-level languages are "close to the metal," whereas high-level languages are closer to how humans think.

For a language to be "close to the metal," it must provide an abstract machine that maps easily to the abstractions exposed by the target platform. It's easy to argue that C was a low-level language for the PDP-11. They both described a model in which programs executed sequentially, in which memory was a flat space, and even the pre- and post-increment operators cleanly lined up with the PDP-11 addressing modes.

Fast PDP-11 Emulators

The root cause of the Spectre and Meltdown vulnerabilities was that processor architects were trying to build not just fast processors, but fast processors that expose the same abstract machine as a PDP-11. This is essential because it allows C programmers to continue in the belief that their language is close to the underlying hardware.

C code provides a mostly serial abstract machine (until C11, an entirely serial machine if nonstandard vendor extensions were excluded). Creating a new thread is a library operation known to be expensive, so processors wishing to keep their execution units busy running C code rely on ILP (instruction-level parallelism). They inspect adjacent operations and issue independent ones in parallel. This adds a significant amount of complexity (and power consumption) to allow programmers to write mostly sequential code. In contrast, GPUs achieve very high performance without any of this logic, at the expense of requiring explicitly parallel programs.

The quest for high ILP was the direct cause of Spectre and Meltdown. A modern Intel processor has up to 180 instructions in flight at a time (in stark contrast to a sequential C abstract machine, which expects each operation to complete before the next one begins). A typical heuristic for C code is that there is a branch, on average, every seven instructions. If you wish to keep such a pipeline full from a single thread, then you must guess the targets of the next 25 branches. This, again, adds complexity; it also means that an incorrect guess results in work being done and then discarded, which is not ideal for power consumption. This discarded work has visible side effects, which the Spectre and Meltdown attacks could exploit.

On a modern high-end core, the register rename engine is one of the largest consumers of die area and power. To make matters worse, it cannot be turned off or power gated while any instructions are running, which makes it inconvenient in a dark silicon era when transistors are cheap but powered transistors are an expensive resource. This unit is conspicuously absent on GPUs, where parallelism again comes from multiple threads rather than trying to extract instruction-level parallelism from intrinsically scalar code. If instructions do not have dependencies that need to be reordered, then register renaming is not necessary.

Consider another core part of the C abstract machine's memory model: flat memory. This hasn't been true for more than two decades. A modern processor often has three levels of cache in between registers and main memory, which attempt to hide latency.

The cache is, as its name implies, hidden from the programmer and so is not visible to C. Efficient use of the cache is one of the most important ways of making code run quickly on a modern processor, yet this is completely hidden by the abstract machine, and programmers must rely on knowing implementation details of the cache (for example, two values that are 64-byte-aligned may end up in the same cache line) to write efficient code.

Over a year ago, HardenedBSD switched to LibreSSL as the default cryptographic library in base for 12-CURRENT. 11-STABLE followed suit later on. Bernard Spil has done an excellent job at keeping our users up-to-date with the latest security patches from LibreSSL.

After recently updating 12-CURRENT to LibreSSL 2.7.2 from 2.6.4, it has become increasingly clear to us that performing major upgrades requires a team larger than a single person. Upgrading to 2.7.2 caused a lot of fallout in our ports tree. As of 28 Apr 2018, several ports we consider high priority are still broken. As it stands right now, it would take Bernard a significant amount of his spare personal time to fix these issues.

Until we have a multi-person team dedicated to maintaining LibreSSL in base along with the patches required in ports, HardenedBSD will use OpenSSL going forward as the default cryptographic library in base. LibreSSL will co-exist with OpenSSL in the source tree, as it does now. However, MK_LIBRESSL will default to "no" instead of the current "yes". Bernard will continue maintaining LibreSSL in base along with addressing the various problematic ports entries.

To provide our users with ample time to plan and perform updates, we will wait a period of two months prior to making the switch. The switch will occur on 01 Jul 2018 and will be performed simultaneously in 12-CURRENT and 11-STABLE. HardenedBSD will archive a copy of the LibreSSL-centric package repositories and binary updates for base for a period of six months after the switch (expiring the package repos on 01 Jan 2019). This essentially gives our users eight full months for an upgrade path.

As part of the switch back to OpenSSL, the default NTP daemon in base will switch back from OpenNTPd to ISC NTP. Users who have localopenntpdenable="YES" set in rc.conf will need to switch back to ntpd_enable="YES".

Users who build base from source will want to fully clean their object directories. Any and all packages that link with libcrypto or libssl will need to be rebuilt or reinstalled.

With the community's help, we look forward to the day when we can make the switch back to LibreSSL. We at HardenedBSD believe that providing our users options to rid themselves of software monocultures can better increase security and manage risk.

DigitalOcean

In the summer of 2008, security researcher Dan Kaminsky disclosed how he had found a huge flaw in the Internet that could let attackers redirect web traffic to alternate servers and disrupt normal operations. In this Hacker History video, Kaminsky describes the flaw and notes the issue remains unfixed.

“We were really concerned about web pages and emails 'cause that’s what you get to compromise when you compromise DNS,” Kaminsky says. “You think you’re sending an email to IBM but it really goes to the bad guy.”

As the phone book of the Internet, DNS translates easy-to-remember domain names into IP addresses so that users don’t have to remember strings of numbers to reach web applications and services. Authoritative nameservers publish the IP addresses of domain names. Recursive nameservers talk to authoritative servers to find addresses for those domain names and saves the information into its cache to speed up the response time the next time it is asked about that site. While anyone can set up a nameserver and configure an authoritative zone for any site, if recursive nameservers don’t point to it to ask questions, no one will get those wrong answers.

We made the Internet less flammable.

Kaminsky found a fundamental design flaw in DNS that made it possible to inject incorrect information into the nameserver's cache, or DNS cache poisoning. In this case, if an attacker crafted DNS queries looking for sibling names to existing domains, such as 1.example.com, 2.example.com, and 3.example.com, while claiming to be the official "www" server for example.com, the nameserver will save that server IP address for “www” in its cache.

“The server will go, ‘You are the official. Go right ahead. Tell me what it’s supposed to be,’” Kaminsky says in the video.

Since the issue affected nearly every DNS server on the planet, it required a coordinated response to address it. Kaminsky informed Paul Vixie, creator of several DNS protocol extensions and application, and Vixie called an emergency summit of major IT vendors at Microsoft’s headquarters to figure out what to do.

The “fix” involved combining the 16-bit transaction identifier that DNS lookups used with UDP source ports to create 32-bit transaction identifiers. Instead of fixing the flaw so that it can’t be exploited, the resolution focused on making it take more than ten seconds, eliminating the instantaneous attack.

“[It’s] not like we repaired DNS,” Kaminsky says. “We made the Internet less flammable.”

DNSSEC (Domain Name System Security Extensions), is intended to secure DNS by adding a cryptographic layer to DNS information. The root zone of the internet was signed for DNSSEC in July 2010 and the .com Top Level Domain (TLD) was finally signed for DNSSEC in April 2011. Unfortunately, adoption has been slow, even ten years after Kaminsky first raised the alarm about DNS, as less than 15 percent of users pass their queries to DNSSEC validating resolvers.

The Internet was never designed to be secure. The Internet was designed to move pictures of cats.

No one expected the Internet to be used for commerce and critical communications. If people lose faith in DNS, then all the things that depend on it are at risk.

“What are we going to do? Here is the answer. Some of us gotta go out fix it,” Kaminsky says.

• Userland software is rebuilt with GCC 6.

Tarsnap ad

iX Ad spot: iXsystems TrueNAS M-Series Blows Away Veeam Backup Certification Tests

OpenBSD 6.3 and DragonflyBSD 5.2 are released, bug fix for disappearing files in OpenZFS on Linux (and only Linux), understanding the FreeBSD CPU scheduler, NetBSD on RPI3, thoughts on being a committer for 20 years, and 5 reasons to use FreeBSD in 2018.

Big-ticket items

Yesterday was the twentieth anniversary of my FreeBSD commit bit, and tomorrow will be the twentieth anniversary of my first commit. I figured I’d split the difference and write a few words about it today.

My level of engagement with the FreeBSD project has varied greatly over the twenty years I’ve been a committer. There have been times when I worked on it full-time, and times when I did not touch it for months. The last few years, health issues and life events have consumed my time and sapped my energy, and my contributions have come in bursts. Commit statistics do not tell the whole story, though: even when not working on FreeBSD directly, I have worked on side projects which, like OpenPAM, may one day find their way into FreeBSD.

My contributions have not been limited to code. I was the project’s first Bugmeister; I’ve served on the Security Team for a long time, and have been both Security Officer and Deputy Security Officer; I managed the last four Core Team elections and am doing so again this year.

In return, the project has taught me much about programming and software engineering. It taught me code hygiene and the importance of clarity over cleverness; it taught me the ins and outs of revision control; it taught me the importance of good documentation, and how to write it; and it taught me good release engineering practices.

Last but not least, it has provided me with the opportunity to work with some of the best people in the field. I have the privilege today to count several of them among my friends.

For better or worse, the FreeBSD project has shaped my career and my life. It set me on the path to information security in general and IAA in particular, and opened many a door for me. I would not be where I am now without it.

I won’t pretend to be able to tell the future. I don’t know how long I will remain active in the FreeBSD project and community. It could be another twenty years; or it could be ten, or five, or less. All I know is that FreeBSD and I still have things to teach each other, and I don’t intend to call it quits any time soon.

San Jose, Calif., April 10, 2018 — iXsystems, the leader in Enterprise Open Source servers and software-defined storage, announced the TrueNAS M40 and M50 as the newest high-performance models in its hybrid, unified storage product line. The TrueNAS M-Series harnesses NVMe and NVDIMM to bring all-flash array performance to the award-winning TrueNAS hybrid arrays. It also includes the Intel® Xeon® Scalable Family of Processors and supports up to 100GbE and 32Gb Fibre Channel networking. Sitting between the all-flash TrueNAS Z50 and the hybrid TrueNAS X-Series in the product line, the TrueNAS M-Series delivers up to 10 Petabytes of highly-available and flash-powered network attached storage and rounds out a comprehensive product set that has a capacity and performance option for every storage budget.

As a unified file, block, and object sharing solution, TrueNAS can meet the needs of file serving, backup, virtualization, media production, and private cloud users thanks to its support for the SMB, NFS, AFP, iSCSI, Fibre Channel, and S3 protocols.

At the heart of the TrueNAS M-Series is a custom 4U, dual-controller head unit that supports up to 24 3.5” drives and comes in two models, the M40 and M50, for maximum flexibility and scalability. The TrueNAS M40 uses NVDIMMs for write cache, SSDs for read cache, and up to two external 60-bay expansion shelves that unlock up to 2PB in capacity. The TrueNAS M50 uses NVDIMMs for write caching, NVMe drives for read caching, and up to twelve external 60-bay expansion shelves to scale upwards of 10PB. The dual-controller design provides high-availability failover and non-disruptive upgrades for mission-critical enterprise environments.

By design, the TrueNAS M-Series unleashes cutting-edge persistent memory technology for demanding performance and capacity workloads, enabling businesses to accelerate enterprise applications and deploy enterprise private clouds that are twice the capacity of previous TrueNAS models. It also supports replication to the Amazon S3, BackBlaze B2, Google Cloud, and Microsoft Azure cloud platforms and can deliver an object store using the ubiquitous S3 object storage protocol at a fraction of the cost of the public cloud.

As a true enterprise storage platform, the TrueNAS M50 supports very demanding performance workloads with up to four active 100GbE ports, 3TB of RAM, 32GB of NVDIMM write cache and up to 15TB of NVMe flash read cache. The TrueNAS M40 and M50 include up to 24/7 and global next-business-day support, putting IT at ease. The modular and tool-less design of the M-Series allows for easy, non-disruptive servicing and upgrading by end-users and support technicians for guaranteed uptime. TrueNAS has US-Based support provided by the engineering team that developed it, offering the rapid response that every enterprise needs.

• Enterprise: Perfectly suited for private clouds and enterprise workloads such as file sharing, backups, M&E, surveillance, and hosting virtual machines.

The TrueNAS M40 and M50 will be generally available in April 2018 through the iXsystems global channel partner network. The TrueNAS M-Series starts at under $20,000 USD and can be easily expanded using a linear “per terabyte” pricing model. With typical compression, a Petabtye can be stored for under $100,000 USD. TrueNAS comes with an all-inclusive software suite that provides NFS, Windows SMB, iSCSI, snapshots, clones and replication.

```

A closer look shows the behaviour as follows (single CPU):

Lets run an I/O-active task, e.g, postgres VACUUM that would

pool alloc free read write read write

Now start an endless loop:

And the effect is:

pool alloc free read write read write

The VACUUM gets almost stuck! This figures with WCPU in "top":

PID USERNAME PRI NICE SIZE RES STATE TIME WCPU COMMAND

Hacking on kern.sched.quantum makes it quite a bit better:

kern.sched.quantum: 94488 -> 7874

pool alloc free read write read write

PID USERNAME PRI NICE SIZE RES STATE TIME WCPU COMMAND

Now, as usual, the "root-cause" questions arise: What exactly does

The docs for the quantum parameter are a bit unsatisfying - they say

11.1-RELEASE-p7, kern.hz=200. Switching tickless mode on or off does

[1]

I have committed about adding initial support for aarch64.

```

…

```

Now, multiuser mode works stably on fdt based boards (RPI3,SUNXI,TEGRA). But there are still some problems, more time is required for release. also SMP is not yet. See sys/arch/aarch64/aarch64/TODO for more detail. Especially the problems around TLS of rtld, and C++ stack unwindings are too difficult for me to solve, I give up and need someone's help (^o^)/ Since C++ doesn't work, ATF also doesn't work. If the ATF works, it will clarify more issues.

sys/arch/evbarm64 is gone and integrated into sys/arch/evbarm. One evbarm/conf/GENERIC64 kernel binary supports all fdt (bcm2837,sunxi,tegra) based boards. While on 32bit, sys/arch/evbarm/conf/GENERIC will support all fdt based boards...but doesn't work yet. (WIP)

My deepest appreciation goes to Tohru Nishimura (nisimura@) whose writes vector handlers, context switchings, and so on. and his comments and suggestions were innumerably valuable. I would also like to thank Nick Hudson (skrll@) and Jared McNeill (jmcneill@) whose added support FDT and integrated into evbarm. Finally, I would like to thank Matt Thomas (matt@) whose commited aarch64

TrueOS Stable 18.03 released, a look at F-stack, the secret to an open source business model, intro to jails and jail networking, FreeBSD Foundation March update, and the ipsec Errata.

The TrueOS team is pleased to announce the availability of a new STABLE release of the TrueOS project (version 18.03). This is a special release due to the security issues impacting the computing world since the beginning of 2018. In particular, mitigating the “Meltdown” and “Spectre” system exploits make it necessary to update the entire package ecosystem for TrueOS. This release does not replace the scheduled June STABLE update, but provides the necessary and expected security updates for the STABLE release branch of TrueOS, even though this is part-way through our normal release cycle.

• “Meltdown” security fixes: This release contains all the fixes to FreeBSD which mitigate the security issues for systems that utilize Intel-based processors when running virtual machines such as FreeBSD jails. Please note that virtual machines or jails must also be updated to a version of FreeBSD or TrueOS which contains these security fixes.

Most systems will need microcode updates for additional Spectre mitigations. The microcode updates are not enabled by default. This work is considered experimental because it is in active development by the upstream vendors. If desired, the microcode updates are available with the new devcpu-data package, which is available in the Appcafe. Install this package and enable the new microcode_update service to apply the latest runtime code when booting the system.

• LibreSSL is updated from version 2.6.3 -> 2.6.4

F-Stack is an user space network development kit with high performance based on DPDK, FreeBSD TCP/IP stack and coroutine API. http://www.f-stack.org

• Ultra high network performance which can achieve network card under full load, 10 million concurrent connections, 5 million RPS, 1 million CPS.

In order to deal with the increasingly severe DDoS attacks, authorized DNS server of Tencent Cloud DNSPod switched from Gigabit Ethernet to 10-Gigabit at the end of 2012. We faced several options, one is to continue to use the original model another is to use kernel bypass technology. After several rounds of investigation, we finally chose to develop our next generation of DNS server based on DPDK. The reason is DPDK provides ultra-high performance and can be seamlessly extended to 40G, or even 100G NIC in the future.

After several months of development and testing, DKDNS, high-performance DNS server based on DPDK officially released in October 2013. It's capable of achieving up to 11 million QPS with a single 10GE port and 18.2 million QPS with two 10GE ports. And then we developed a user-space TCP/IP stack called F-Stack that can process 0.6 million RPS with a single 10GE port.

With the fast growth of Tencent Cloud, more and more services need higher network access performance. Meanwhile, F-Stack was continuous improving driven by the business growth, and ultimately developed into a general network access framework. But this TCP/IP stack couldn't meet the needs of these services while continue to develop and maintain a complete network stack will cost high, we've tried several plans and finally determined to port FreeBSD(11.0 stable) TCP/IP stack into F-Stack. Thus, we can reduce the cost of maintenance and follow up the improvement from community quickly.Thanks to libplebnet and libuinet, this work becomes a lot easier.

With the rapid development of all kinds of application, in order to help different APPs quick and easily use F-Stack, F-Stack has integrated Nginx, Redis and other commonly used APPs, and a micro thread framework, and provides a standard Epoll/Kqueue interface.

Currently, besides authorized DNS server of DNSPod, there are various products in Tencent Cloud has used the F-Stack, such as HttpDNS (D+), COS access module, CDN access module, etc..

iXsystems

Jails basically partition a FreeBSD system into various isolated sub-systems called jails. The syscall and userspace tools first appeared in FreeBSD 4.0 (~ March 2000) with subsequent releases expanding functionality and improving existing features as well as usability.

• > kcov provides code coverage collection for coverage-guided fuzzing (randomized testing). Coverage-guided fuzzing is a testing technique that uses coverage feedback to determine new interesting inputs to a system.

Next year, we have the opportunity to have a BSD track, similar to the BSD Devroom at FOSDEM. We are looking for some volunteers in Southern California who can help organize this one or two-day event and help us educate more people about the BSDs. Let us know if you

Roll Call: #WhoUsesFreeBSD

Many of you probably saw our post on social media asking Who Uses FreeBSD. Please help us answer this question to assist us in determining FreeBSD market share data, promote how companies are successfully using FreeBSD to encourage more companies to embrace

New Hosting Partner: Oregon State University Open Source Lab

Tarsnap

Second round of ZFS improvements in FreeBSD, Postgres finds that non-FreeBSD/non-Illumos systems are corrupting data, interview with Kevin Bowling, BSDCan list of talks, and cryptographic right answers.

One of the first tasks during the pool load process is to parse a config provided from userland that describes what devices the pool is composed of. A vdev tree is generated from that config, and then all the vdevs are opened.

• 7614 zfs device evacuation/removal

• 7446 zpool create should support efi system partition

Some time ago I ran into an issue where a user encountered data corruption after a storage error. PostgreSQL played a part in that corruption by allowing checkpoint what should've been a fatal error.

iXsystems

We’re less interested in empowering developers and a lot more pessimistic about the prospects of getting this stuff right.

But if you’re a developer and not a cryptography engineer, you shouldn’t do any of that. You should keep things simple and conventional and easy to analyze; “boring”, as the Google TLS people would say.

Percival, 2009: AES-CTR with HMAC.

Percival, 2009: Use 256-bit keys.

Percival, 2009: Use HMAC.

Percival, 2009: Use SHA256 (SHA-2).

Percival, 2009: Use 256-bit random numbers.

Percival, 2009: scrypt or PBKDF2.

Percival, 2009: Use RSAES-OAEP with SHA256 and MGF1+SHA256 bzzrt pop ffssssssst exponent 65537.

Percival, 2009: Use RSASSA-PSS with SHA256 then MGF1+SHA256 in tricolor systemic silicate orientation.

Percival, 2009: Operate over the 2048-bit Group #14 with a generator of 2.

Percival, 2009: Use OpenSSL.

Percival, 2009: Distribute the server’s public RSA key with the client code, and do not use SSL.

Percival, 2009: Use Tarsnap.

I am adding IPv6 addresses to each of my servers. This post assumes the server is up and running FreeBSD 11.1 and you already have an IPv6 address block. This does not cover the creation of an IPv6 tunnel, such as that provided by HE.net. This assumes native IPv6.

In this post, I am using the IPv6 addresses from the IPv6 Address Prefix Reserved for Documentation (i.e. 2001:DB8::/32). You should use your own addresses.

The IPv6 block I have been assigned is 2001:DB8:1001:8d00/64.

I added this to /etc/rc.conf:

The IPv6 address I have assigned to this host is completely random (with the given block). I found a random IPv6 address generator and used it to select d389:119c:9b57:396b as the address for this service within my address block.

I don’t have the reference, but I did read that randomly selecting addresses within your block is a better approach.

In order to invoke these changes without rebooting, I issued these commands:

```

[dan@tallboy:~] $ sudo route add -inet6 default 2001:DB8:1001:8d00::1

If you do the route add first, you will get this error:

Tarsnap

New ZFS features landing in FreeBSD, MAP_STACK for OpenBSD, how to write safer C code with Clang’s address sanitizer, Michael W. Lucas on sponsor gifts, TCP blackbox recorder, and Dell disk system hacking.

With compressed ARC (6950) we use up to 25% of our CPU to decompress indirect blocks, under a workload of random cached reads. To reduce this decompression cost, we would like to increase the size of the dbuf cache so that more indirect blocks can be stored uncompressed.

The idea of Storage Pool Checkpoint (aka zpool checkpoint) deals with exactly that. It can be thought of as a “pool-wide snapshot” (or a variation of extreme rewind that doesn’t corrupt your data). It remembers the entire state of the pool at the point that it was taken and the user can revert back to it later or discard it. Its generic use case is an administrator that is about to perform a set of destructive actions to ZFS as part of a critical procedure. She takes a checkpoint of the pool before performing the actions, then rewinds back to it if one of them fails or puts the pool into an unexpected state. Otherwise, she discards it. With the assumption that no one else is making modifications to ZFS, she basically wraps all these actions into a “high-level transaction”.

In pursuit of improving performance on multi-core systems, we should implements fanned out counters and use them to improve the performance of some of the arc statistics. These stats are updated extremely frequently, and can consume a significant amount of CPU time.

Almost 2 decades ago we started work on W^X. The concept was simple. Pages that are writable, should not be executable. We applied this concept object by object, trying to seperate objects with different qualities to different pages. The first one we handled was the signal trampoline at the top of the stack. We just kept making changes in the same vein. Eventually W^X came to some of our kernel address spaces also.

We wanted to improve our password strength algorithm, and decided to go for the industry-standard zxcvbn, from the people at Dropbox. Our web front-end would use the default Javascript library, and for mobile and desktop, we chose to use the C implementation as it was the lowest common denominator for all platforms.

Note the little stack of customs forms off to the side. It’s like I’ve learned a lesson from standing at the post office counter filling out those stupid forms. Sponsors should get their books soon.

This seems like an apropos moment to talk about what I do for print sponsors. I say I send them “a gift,” but what does that really mean? The obvious thing to ship them is a copy of the book I’ve written. Flat-out selling print books online has tax implications, though.

Sponsors might have guessed that they’d get a copy of the book. But I shipped them the hardcover, which isn’t my usual practice.

That’s because I send sponsors a gift. As it’s a gift, I get to choose what I send. I want to send them something nice, to encourage them to sponsor another book. It makes no sense for me to send a sponsor a Singing Wedgie-O-Gram. (Well, maybe a couple sponsors. You know who you are.)

The poor bastards who bought into my scam–er, sponsored my untitled book–have no idea what’s coming. As of right now, their sensible guesses are woefully incomplete.

Future books? They might get a copy of the book. They might get book plus something. They might just get the something. Folks who sponsor the jails book might get a cake with a file in it. Who knows?

It’s a gift. It’s my job to make that gift worthwhile.

And to amuse myself. Because otherwise, what’s the point?

KDE4 has been rudely moved aside on FreeBSD. It still installs (use x11/kde4) and should update without a problem, but this is another step towards adding modern KDE (Plasma 5 and Applications) to the official FreeBSD Ports tree.

KDE4 users will see a lot of packages moving around and being renamed, but no functional changes. Curiously, the KDE4 desktop depends on Qt5 and KDE Frameworks 5 — and it has for quite some time already, because the Oxygen icons are shared with KDE Frameworks, but primarily because FileLight was updated to the modern KDE Applications version some time ago (the KDE4 version had some serious bugs, although I can not remember what they were). Now that the names are cleaned up, we could consider giving KDE4 users the buggy version back.

From here on, we’ve got the following things lined up:

So we’ve been saying Real Soon Now ™ for years, but things are Realer Sooner Nower ™ now.

A while back I reviewed the Dell FS12-NV7 – a 2U rack server being sold cheap by all and sundry. It’s a powerful box, even by modern standards, but one of its big drawbacks is the disk system it comes with. But it needn’t be.

There are two viable solutions, depending on what you want to do. You can make use of the SAS backplane, using SAS and/or SATA drives, or you can go for fewer SATA drives and free up one or more PCIe slots as Plan B. You probably have an FS12 because it looks good for building a drive array (or even FreeNAS) so I’ll deal with Plan A first.

Like most Dell servers, this comes with a Dell PERC RAID SAS controller – a PERC6/i to be precise. This ‘I’ means it has internal connectors; the /E is the same but its sockets are external.

The PERC connects to a twelve-slot backplane forming a drive array at the front of the box. More on the backplane later; it’s the PERCs you need to worry about.

The PERC6 is actually an LSI Megaraid 1078 card, which is just the thing you need if you’re running an operating system like Windows that doesn’t support a volume manager, striping and other grown-up stuff. Or if your OS does have these features, but you just don’t trust it. If you are running such an OS you may as well stick to the PERC6, and good luck to you. If you’re using BSD (including FreeNAS), Solaris or a Linux distribution that handles disk arrays, read on. The PERC6 is a solution to a problem you probably don’t have, but in all other respects its a turkey. You really want a straightforward HBA (Host Bus Adapter) that allows your clever operating system to talk directly with the drives.

Any SAS card based on the 1078 (such as the PERC6) is likely to have problems with drives larger than 2Tb. I’m not completely sure why, but I suspect it only applies to SATA. Unfortunately I don’t have any very large SAS drives to test this theory. A 2Tb limit isn’t really such a problem when you’re talking about a high performance array, as lots of small drives are a better option anyway. But it does matter if you’re building a very large datastore and don’t mind slower access and very significant resilvering times when you replace a drive. And for large datastores, very large SATA drives save you a whole lot of cash. The best capacity/cost ratio is for 5Gb SATA drives

Some Dell PERCs can be re-flashed with LSI firmware and used as a normal HBA. Unfortunately the PERC6 isn’t one of them. I believe the PERC6/R can be, but those I’ve seen in a FS12 are just a bit too old. So the first thing you’ll need to do is dump them in the recycling or try and sell them on eBay.

There are actually two PERC6 cards in most machine, and they each support eight SAS channels through two SFF-8484 connectors on each card. Given there are twelve drives slots, one of the PERCs is only half used. Sometimes they have a cable going off to a battery located near the fans. This is used in a desperate attempt to keep the data in the card’s cache safe in order to avoid write holes corrupting NTFS during a power failure, although the data on the on-drive caches won’t be so lucky. If you’re using a file system like that, make sure you have a UPS for the whole lot.

But we’re going to put the PERCs out of our misery and replace them with some nice new LSI HBAs that will do our operating system’s bidding and let it talk to the drives as it knows best. But which to pick? First we need to know what we’re connecting.

Moving to the front of the case there are twelve metal drive slots with a backplane behind. Dell makes machines with either backplanes or expanders. A backplane has a 1:1 SAS channel to drive connection; an expander takes one SAS channel and multiplexes it to (usually) four drives. You could always swap the blackplane with an expander, but I like the 1:1 nature of a backplane. It’s faster, especially if you’re configured as an array. And besides, we don’t want to spend more money than we need to, otherwise we wouldn’t be hot-rodding a cheap 2U server in the first place – expanders are expensive. Bizarrely, HBAs are cheap in comparison. So we need twelve channels of SAS that will connect to the sockets on the backplane.

The HBA you will probably want to go with is an LSI, as these have great OS support. Other cards are available, but check that the drivers are also available. The obvious choice for SAS aficionados is the LSI 9211-8i, which has eight internal channels. This is based on an LSI 2000 series chip, the 2008, which is the de-facto standard. There’s also four-channel -4i version, so you could get your twelve channels using one of each – but the price difference is small these days, so you might as well go for two -8i cards. If you want cheaper there are 1068-based equivalent cards, and these work just fine at about half the price. They probably won’t work with larger disks, only operate at 3Gb and the original SAS standard. However, the 2000 series is only about £25 extra and gives you more options for the future. A good investment. Conversely, the latest 3000 series cards can do some extra stuff (particularly to do with active cables) but I can’t see any great advantage in paying megabucks for one unless you’re going really high-end – in which case the NV12 isn’t the box for you anyway. And you’d need some very fast drives and a faster backplane to see any speed advantage. And probably a new motherboard….

Whether the 6Gb SAS2 of the 9211-8i is any use on the backplane, which was designed for 3Gb, I don’t know. If it matters that much to you you probably need to spend a lot more money. A drive array with a direct 3Gb to each drive is going to shift fast enough for most purposes.

Once you have removed the PERCs and plugged in your modern-ish 9211 HBAs, your next problem is going to be the cable. Both the PERCs and the backplane have SFF-8484 multi-lane connectors, which you might not recognise. SAS is a point-to-point system, the same as SATA, and a multi-lane cable is simply four single cables in a bundle with one plug. (Newer versions of SAS have more). SFF-8484 multi-lane connectors are somewhat rare, (but unfortunately this doesn’t make them valuable if you were hoping to flog them on eBay). The world switched quickly to the SFF-8087 for multi-lane SAS. The signals are electrically the same, but the connector is not.

Please generate and paste your ad code here. If left empty, the ad location will be highlighted on your blog pages with a reminder to enter your code. Mid-Post

First things first – there is actually a jumper on the backplane to tell it whether it’s talking to a PERC or a standard LSI HBA. All you need to do is find it and change it. Fortunately there are very few jumpers to choose from (i.e. two), and you know the link is already in the wrong place. So try them one at a time until it works. The one you want may be labelled J15, but I wouldn’t like to say this was the same on every variant.

Second problem: the cable. You can get cables with an SFF-8087 on one end and an SFF-8484 on the other. These should work. But they’re usually rather expensive. If you want to make your own, it’s a PITA but at least you have the connectors already (assuming you didn’t bin the ones on the PERC cables).

I don’t know what committee designed SAS cable connectors, but ease of construction wasn’t foremost in their collective minds. You’re basically soldering twisted pair to a tiny PCB. This is mechanically rubbish, of course, as the slightest force on the cable will lift the track. Therefore its usual to cover the whole joint in solidified gunk (technical term) to protect it. Rewiring SAS connectors is definitely not easy.

I’ve tried various ways of soldering to them, none of which were satisfactory or rewarding. One method is to clip the all bare wires you wish to solder using something like a bulldog clip so they’re at lined up horizontally and then press then adjust the clamp so they’re gently pressed to the tracks on the board, making final adjustments with a strong magnifying glass and a fine tweezers. You can then either solder them with a fine temperature-controlled iron, or have pre-coated the pads with solder paste and flash across it with an SMD rework station. I’d love to know how they’re actually manufactured – using a precision jig I assume.

The “easy” way is to avoid soldering the connectors at all; simply cut existing cables in half and join one to the other. I’ve used prototyping matrix board for this. Strip and twist the conductors, push them through a hole and solder. This keeps things compact but manageable. We’re dealing with twisted pair here, so maintain the twists as close as possible to the board – it actually works quite well.

However, I’ve now found a reasonably-priced source of the appropriate cable so I don’t do this any more. Contact me if you need some in the UK.

So all that remains is to plug your HBAs to the backplane, shove in some drives and you’re away. If you’re at this stage, it “just works”. The access lights for all the drives do their thing as they should. The only mystery is how you can get the ident LED to come on; this may be controlled by the PERC when it detects a failure using the so-called sideband channel, or it may be operated by the electronics on the backplane. It’s workings are, I’m afraid, something of a mystery still – it’s got too much electronics on board to be a completely passive backplane.

Plan B: SATA

If you plan to use only SATA drives, especially if you don’t intend using more than six, it makes little sense to bother with SAS at all. The Gigabyte motherboard comes with half a dozen perfectly good 3Gb SATA channels, and if you need more you can always put another controller in a PCIe slot, or even USB. The advantages are lower cost and you get to free up two PCIe slots for more interesting things.

The down-side is that you can’t use the SAS backplane, but you can still use the mounting bays.

Removing the backplane looks tricky, but it really isn’t when you look a bit closer. Take out the fans first (held in place by rubber blocks), undo a couple of screws and it just lifts and slides out. You can then slot and lock in the drives and connect the SATA connectors directly to the back of the drives. You could even slide them out again without opening the case, as long as the cable was long enough and you manually detached the cable it when it was withdrawn. And let’s face it – drives are likely to last for years so even with half a dozen it’s not that great a hardship to open the case occasionally.

Next comes power. The PSU has a special connector for the backplane and two standard SATA power plugs. You could split these three ways using an adapter, but if you have a lot of drives you might want to re-wire the cables going to the backplane plug. It can definitely power twelve drives.

And that’s almost all there is to it. Unfortunately the main fans are connected to the backplane, which you’ve just removed. You can power them from an adapter on the drive power cables, but there are unused fan connectors on the motherboard. I’m doing a bit more research on cooling options, but this approach has promising possibilities for noise reduction.

Tarsnap

OpenBSD firewalling Windows 10, NetBSD’s return to ptrace, TCP Alternative Backoff, the BSD Poetic license, and AsiaBSDcon 2018 videos available.

MP3 Feed | iTunes Feed | HD Vid Feed | HD Torrent Feed

Whilst setting up one of my development laptops to port some software to Windows I noticed Windows 10 doing crazy things like installing or updating apps and games by default after initial setup. The one I noticed in particular was Candy Crush Soda Saga which for those who don't know of it is some cheesy little puzzle game originally for consumer devices. I honestly did not want software like this near to a development machine. It has also been reported that Windows 10 now also updates core system software without notifying the user. Surely this destroys any vaguely deterministic behaviour, in my opinion making Windows 10 by default almost useless for development testbeds.

Deciding instead to start from scratch but this time to set the inbuilt Windows Firewall to be very restrictive and only allow a few select programs to communicate. In this case all I really needed to be online was Firefox, Subversion and Putty. To my amusement (and astonishment) I found out that the Windows firewall could be modified to give access very easily by programs during installation (usually because this task needs to be done with admin privileges). It also seems that Windows store Apps can change the windows firewall settings at any point. One way to get around this issue could be to install a 3rd party firewall that most software will not have knowledge about and thus not attempt to break through. However the only decent firewall I have used was Sygate Pro which unfortunately is no longer supported by recent operating systems. The last supported versions was 2003, XP and 2000. In short, I avoid 3rd party firewalls.

Instead I decided to trap Windows 10 (and all of it's rogue updaters) behind a virtual machine running OpenBSD. This effectively provided me with a full blown firewall appliance. From here I could then allow specific software I trusted through the firewall (via a proxy) in a safe, controlled and deterministic manner. For other interested developers (and security conscious users) and for my own reference, I have listed the steps taken here:

```

```

We will use this service for a limited selection of "safe and trusted" programs to connect to the outside world from within the Windows 10 host. You can also use putty on the host to connect to the VM via SSH and create a SOCKS proxy which software like Firefox can also use to connect externally.

• Firefox - go to the connection settings and specify the VMs IP address for the proxy.

I've managed to unbreak the LLDB debugger as much as possible with the current kernel and hit problems with ptrace(2) that are causing issues with further work on proper NetBSD support. Meanwhile, I've upstreamed all the planned NetBSD patches to sanitizers and helped other BSDs to gain better or initial support.

Since the last time I worked on LLDB, we have introduced many changes to the kernel interfaces (most notably related to signals) that apparently fixed some bugs in Go and introduced regressions in ptrace(2). Part of the regressions were noted by the existing ATF tests. However, the breakage was only marked as a new problem to resolve. For completeness, the ptrace(2) code was also cleaned up by Christos Zoulas, and we fixed some bugs with compat32.

I've fixed a crash in *NetBSD::Factory::Launch(), triggered on startup of the lldb-server application.

Here is the commit message:

```

The generic code for launching a process in:

Use the proper call processup->SetState(,false)

I suspended development of new features in sanitizers last month, but I was still in the process of upstreaming of local patches. This process was time-consuming as it required rebasing patches, adding dedicated tests, and addressing all other requests and comments from the upstream developers.

I'm not counting hot fixes, as some changes were triggering build or test issues on !NetBSD hosts. Thankfully all these issues were addressed quickly. The final result is a reduction of local delta size of almost 1MB to less than 100KB (1205 lines of diff). The remaining patches are rescheduled for later, mostly because they depend on extra work with cross-OS tests and prior integration of sanitizers with the basesystem distribution. I didn't want to put extra work here in the current state of affairs and, I've registered as a mentor for Google Summer of Code for the NetBSD Foundation and prepared Software Quality improvement tasks in order to outsource part of the labour.

I've also improved documentation for some of the features of NetBSD, described in man-pages. These pieces of information were sometimes wrong or incomplete, and this makes covering the NetBSD system with features such as sanitizers harder as there is a mismatch between the actual code and the documented code.

Some pieces of software also require better namespacing support, these days mostly for the POSIX standard. I've fixed few low-hanging fruits there and requested pullups to NetBSD-8(BETA).

I thank the developers for improving the landed code in order to ship the best solutions for users.

A One-man-show in human activity is usually less fun and productive than collaboration in a team. This is also true in software development. Last month I was helping as a reviewer to port LLVM features to FreeBSD and when possible to OpenBSD. This included MSan/FreeBSD, libFuzzer/FreeBSD, XRay/FreeBSD and UBSan/OpenBSD.

I've landed most of the submitted and reviewed code to the mainstream LLVM tree.

Part of the code also verified the correctness of NetBSD routes in the existing porting efforts and showed new options for improvement. This is the reason why I've landed preliminary XRay/NetBSD code and added missing NetBSD bits to ToolChain::getOSLibName(). The latter produced setup issues with the prebuilt LLVM toolchain, as the directory name with compiler-rt goodies were located in a path like ./lib/clang/7.0.0/lib/netbsd8.99.12 with a varying OS version. This could stop working after upgrades, so I've simplified it to "netbsd", similar to FreeBSD and Solaris.

I've prepared a build of Clang/LLVM with LLDB and compiler-rt features prebuilt on NetBSD/amd64 v. 8.99.12:

llvm-clang-compilerrt-lldb-7.0.0beta_2018-02-28.tar.bz2

With the approaching NetBSD 8.0 release I plan to finish backporting a few changes there from HEAD:

Once done, I will return to ptrace(2) debugging and corrections.

DigitalOcean

When working on complex systems, such as OS kernels, your attention span and cognitive energy are too valuable to be wasted on inefficiencies pertaining to ancillary tasks. After experimenting with different environmental setups for kernel debugging, some of which were awkward and distracting from my main objectives, I have arrived to my current workflow, which is described here. This approach is mainly oriented towards security research and the study of kernel internals.

Before delving into the details, this is the general outline of my environment:

My host system runs Linux. My target system is a QEMU guest.

I’m tracing and debugging on my host system by attaching GDB (with NetBSD x86-64 ABI support) to QEMU’s built-in GDB server.

• QEMU

• Now is a great time to familiarize yourself with the build.sh tool and its options. Be especially carefull with the following options:

Chance are, you do not want to use these options once you’ve successfully built the cross-compilation toolchain and your entire userland, because building those takes time and there aren’t many good reasons to recompile them from scratch. Here’s what to expect:

On my desktop, running a quad-core Intel i5-3470 at 3.20GHz with 24GB of RAM and underlying directory structure residing on a SSD drive, the entire process took about 55 minutes. I was running make with -j12, so the machine was quite busy.

• Preparing the guest system

```

More details about ABE can be found in the Internet-Draft:

The implementation introduces four new sysctls:

The values of beta and betaecn can also be set per-connection by way of the

Submitted by: Tom Jones [email protected]

The recent changes in -current mitigating the Meltdown vulnerability have been backported to the 6.1 and 6.2 (amd64) releases, and the syspatch update (for 6.2) is now available.

```

Changes by: [email protected] 2018/01/06 15:03:13

Changes by: [email protected] 2018/01/06 18:08:20

Changes by: [email protected] 2018/01/07 12:56:19

Changes by: [email protected] 2018/01/12 01:21:30

Changes by: [email protected] 2018/02/21 12:24:15

Changes by: [email protected] 2018/02/22 13:18:59

Changes by: [email protected] 2018/02/22 13:27:14

Changes by: [email protected] 2018/02/22 13:36:40

OpenBSD 6.1 errata 037

```

Changes by: [email protected] 2018/01/06 15:03:13

Changes by: [email protected] 2018/01/06 18:08:20

Changes by: [email protected] 2018/01/07 12:56:19

Changes by: [email protected] 2018/01/12 01:21:30

Changes by: [email protected] 2018/02/21 12:24:15

Changes by: [email protected] 2018/02/22 13:18:59

Changes by: [email protected] 2018/02/22 13:27:14

Changes by: [email protected] 2018/02/22 13:36:40

OpenBSD 6.2 errata 009

iXsystems

Ken Westerback (krw@) has sent in the first report from the (recently concluded) a2k18 hackathon:

```

Whew.

Once in Dunedin the hacking commenced. The background was a regular tick of new meltdown diffs to test in addition to whatever work one was actually engaged in. I was lucky (?) in that none of the problems with the various versions cropped up on my laptop.

```

More substantially the use of recorded leases was made less intrusive by avoiding continual reconfiguration of the interface with the same information. The 'request', 'require' and 'ignore' dhclient.conf(5) statement were changed so they are cumulative, making it easier to build longer lists of affected options.

I tweaked softraid(4) to remove a handrolled version of duid_format().

I sprinkled a couple of M_WAITOK into amd64 and i386 mpbios to document that there is really no need to check for NULL being returned from some malloc() calls.

I continued to help test the new filesystem quiescing logic that deraadt@ committed during the hackathon.

I only locked myself out of my room once!

Fueled by the excellent coffee from local institutions The Good Earth Cafe and The Good Oil Cafe, and the excellent hacking facilities and accommodations at the University of Otago it was another enjoyable and productive hackathon south of the equator. And I even saw penguins.

Thanks to Jim Cheetham and the support from the project and the OpenBSD Foundation that made it all possible

I found this when going through old documents. It looks like I wrote it and never posted it. Perhaps I didn’t consider it finished at the time. But looking at it now, I think it’s good enough to share. It’s a redrafting of the BSD licence, in poetic form. Maybe I had plans to do other licences one day; I can’t remember.

I’ve interleaved it with the original license text so you can see how true, or otherwise, I’ve been to it. Enjoy :-)

```

Redistribution and use in source and binary forms, with or without

You may redistribute and use –

The first is obvious, of course –

The third applies when you promote:

One final point to be laid out

WE ARE NOT LIABLE FOR LOSS

When all is told, we sum up thus –

Tarsnap ad

Looking at Lumina Desktop 2.0, 2 months of KPTI development in SmartOS, OpenBSD email service, an interview with Ryan Zezeski, NomadBSD released, and John Carmack's programming retreat with OpenBSD.

A few weeks ago I sat down with Lead Developer Ken Moore of the TrueOS Project to get answers to some of the most frequently asked questions about Lumina Desktop from the open source community. Here is what he said on Lumina Desktop 2.0. Do you have a question for Ken and the rest of the team over at the TrueOS Project? Make sure to read the interview and comment below. We are glad to answer your questions!

Ken: Lumina Desktop 2.0 is a significant overhaul compared to Lumina 1.x. Almost every single subsystem of the desktop has been streamlined, resulting in a nearly-total conversion in many important areas.

With Lumina Desktop 2.0 we will finally achieve our long-term goal of turning Lumina into a complete, end-to-end management system for the graphical session and removing all the current runtime dependencies from Lumina 1.x (Fluxbox, xscreensaver, compton/xcompmgr). The functionality from those utilities is now provided by Lumina Desktop itself.

Going along with the session management changes, we have compressed the entire desktop into a single, multi-threaded binary. This means that if any rogue script or tool starts trying to muck about with the memory used by the desktop (probably even more relevant now than when we started working on this), the entire desktop session will close/crash rather than allowing targeted application crashes to bypass the session security mechanisms. By the same token, this also prevents “man-in-the-middle” type of attacks because the desktop does not use any sort of external messaging system to communicate (looking at you dbus). This also gives a large performance boost to Lumina Desktop

The entire system for how a user’s settings get saved and loaded has been completely redone, making it a “layered” settings system which allows the default settings (Lumina) to get transparently replaced by system settings (OS/Distributor/SysAdmin) which can get replaced by individual user settings. This results in the actual changes in the user setting files to be kept to a minimum and allows for a smooth transition between updates to the OS or Desktop. This also provides the ability to “restrict” a user’s desktop session (based on a system config file) to the default system settings and read-only user sessions for certain business applications.

The entire graphical interface has been written in QML in order to fully-utilize hardware-based GPU acceleration with OpenGL while the backend logic and management systems are still written entirely in C++. This results in blazing fast performance on the backend systems (myriad multi-threaded C++ objects) as well as a smooth and responsive graphical interface with all the bells and whistles (drag and drop, compositing, shading, etc).

While I have never tried out Lumina in a MAC jail, I do not see anything on that page which should stop it from running in one right now. Lumina is already designed to be run as an unpriviledged user and is very smart about probing the system to find out what is/not available before showing anything to the user. The only thing that comes to mind is that you might need to open up some other system devices so that X11 itself can draw to the display (graphical environment setup is a bit different than CLI environment).

If I recall correctly, one of the very first versions of Lumina (pre-1.0) would occasionally flicker. If that is still happening, you might want to verify that you are using the proper video driver for your hardware and/or enable the compositor within the Lumina settings.

This was a common question about 4(?) years ago with the first release of the Lumina desktop and it basically boiled down to long-term support and reliability of the underlying toolkit. Some of the things we had to consider were: cross-platform/cross-architecture support, dependency reliability and support framework (Qt5 > EFL), and runtime requirements and dependency tracking (Qt5 is lighter than the EFL). That plus the fact that the EFL specifically states that it is linux-focused and the BSD’s are just an afterthought (especially at the time we were doing the evaluation).

Like many people who work on system security, I had read Anders Fogh's post about a "Negative Result" in speculative execution research in July of 2017. At the time I thought it was an interesting writeup and I remember being glad that researchers were looking into this area. I sent the post to Bryan and asked him about his thoughts on it at the time, to which he replied saying that "it would be shocking if they left a way to directly leak out memory in the speculative execution". None of us seriously thought that there would be low-hanging fruit down that research path, but we also felt it was important that there was someone doing work in the area who was committed to public disclosure.

At first, after reading the blog post on Monday, we thought (or hoped) that the bug might "just" be a KASLR bypass and wouldn't require a lot of urgency. We tried to reach out to Intel at work to get more information but were met with silence. (We wouldn't hear back from them until after the disclosure was already made public.) The speculation on Tuesday intensified, until finally on Wednesday morning I arrived at the office to find links to late Tuesday night tweets revealing exploits that allowed arbitrary kernel memory reads.

Wednesday was not a happy day. Intel finally responded to our emails -- after they had already initiated public disclosure. We all spent a lot of time reading. An arbitrary kernel memory read (an info leak) is not that uncommon as far as bugs go, but for the most part they tend to be fairly easy to fix. The thing that makes the Meltdown and Spectre bugs particularly notable is that in order to mitigate them, a large amount of change is required in very deep low-level parts of the kernel. The kind of deep parts of the kernel where there are 20-year old errata workarounds that were single-line changes that you have to be very careful to not accidentally undo; the kind of parts where, as they say, mortals fear to tread.

On Friday we saw the patches Matthew Dillon put together for DragonFlyBSD for the first time. These were the first patches for KPTI that were very straightforward to read and understand, and applied to a BSD-derived kernel that was similar to those I'm accustomed to working on.

To mitigate Meltdown (and partially one of the Spectre variants), you have to make sure that speculative execution cannot reach any sensitive data from a user context. This basically means that the pages the kernel uses for anything potentially sensitive have to be unmapped when we are running user code. Traditionally, CPUs that were built to run a multi-user, UNIX-like OS did this by default (SPARC is an example of such a CPU which has completely separate address spaces for the kernel and userland). However, x86 descends from a single-address-space microcontroller that has grown up avoiding backwards-incompatible changes, and has never really introduced a clean notion of multiple address spaces (segmentation is the closest feature really, and it was thrown out for 64-bit AMD64). Instead, operating systems for x86 have generally wound up (at least in the post-AMD64 era) with flat address space models where the kernel text and data is always present in the page table no matter whether you're in user or kernel mode. The kernel mappings simply have the "supervisor" bit set on them so that user code can't directly access them.

The mitigation is basically to stop doing this: to stop mapping the kernel text, data and other memory into the page table while we're running in userland. Unfortunately, the x86 design does not make this easy. In order to be able to take interrupts or traps, the CPU has to have a number of structures mapped in the current page table at all times. There is also no ability to tell an x86 CPU that you want it to switch page tables when an interrupt occurs. So, the code that we jump to when we take an interrupt, as well as space for a stack to push context onto have to be available in both page tables. And finally, of course, we need to be able to figure out somehow what the other page table we should switch to is when we enter the kernel.

When we looked at the patches for Linux (and also the DragonFlyBSD patches at the time) on Friday and started asking questions, it became pretty evident that the initial work done by both was done under time constraints. Both had left the full kernel text mapped in both page tables, and the Linux trampoline design seemed over-complex. I started talking over some ideas with Robert Mustacchi about ways to fix these and who we should talk to, and reached out to some of my old workmates from the University of Queensland who were involved with OpenBSD. It seemed to me that the OpenBSD developers would care about these issues even more than we did, and would want to work out how to do the mitigation right.

I ended up sending an email to Philip Guenther on Friday afternoon, and on Saturday morning I drove an hour or so to meet up with him for coffee to talk page tables and interrupt trampolines. We wound up spending a good 6 hours at the coffee shop, and I came back with several pages of notes and a half-decent idea of the shape of the work to come.

One detail we missed that day was the interaction of per-CPU structures with per-process page tables. Much of the interrupt trampoline work is most easily done by using per-CPU structures in memory (and you definitely want a per-CPU stack!). If you combine that with per-process page tables, however, you have a problem: if you leave all the per-CPU areas mapped in all the processes, you will leak information (via Meltdown) about the state of one process to a different one when taking interrupts. In particular, you will leak things like %rip, which ruins all the work being done with PIE and ASLR pretty quickly. So, there are two options: you can either allocate the per-CPU structures per-process (so you end up with $NCPUS * $NPROCS of them); or you can make the page tables per-CPU.

OpenBSD, like Linux and the other implementations so far, decided to go down the road of per-CPU per-process pages to solve this issue. For illumos, we took the other route.

In illumos, it turned out that we already had per-CPU page tables. Robert and I re-discovered this on the Sunday of that week. We use them for 32-bit processes due to having full P>V PAE support in our kernel (which is, as it turns out, relatively uncommon amongst open-source OS). The logic to deal with creating and managing them and updating them was all already written, and after reading the code we concluded we could basically make a few small changes and re-use all of it. So we did.

By the end of that second week, we had a prototype that could get to userland. But, when working on this kind of kernel change we have a rule of thumb we use: after the first 70% of the patch is done and we can boot again, now it's time for the second 70%. In fact it turned out to be more like the second 200% for us -- a tedious long tail of bugs to solve that ended up necessitating some changes in the design as well.

At first we borrowed the method that Matt Dillon used for DragonFlyBSD, by putting the temporary "stack" space and state data for the interrupt trampolines into an extra page tacked onto the end of *%gs (in illumos the structure that lives there is the cpu_t).

If you read the existing logic in interrupt handlers for dealing with %gs though, you will quickly notice that the corner cases start to build up. There are a bunch of situations where the kernel temporarily alters %gs, and some of the ways to mess it up have security consequences that end up being worse than the bug we're trying to fix. As it turns out, there are no less than 3 different ways that ISRs use to try to get to having the right cpu_t in %gs on illumos, as it turns out, and they are all subtly different. Trying to tell which you should use when requires a bunch of test logic that in turn requires branches and changes to the CPU state, which is difficult to do in a trampoline where you're trying to avoid altering that state as much as possible until you've got the real stack online to push things into.

I kept in touch with Philip Guenther and Mike Larkin from the OpenBSD project throughout the weeks that followed. In one of the discussions we had, we talked about the NMI/MCE handlers and the fact that their handling currently on OpenBSD neglected some nasty corner-cases around interrupting an existing trap handler. A big part of the solution to those issues was to use a feature called IST, which allows you to unconditionally change stacks when you take an interrupt.

Traditionally, x86 only changes the stack pointer (%rsp on AMD64) while taking an interrupt when there is a privilege level change. If you take an interrupt while already in the kernel, the CPU does not change the stack pointer, and simply pushes the interrupt stack frame onto the stack you're already using. IST makes the change of stack pointer unconditional. If used unwisely, this is a bad idea: if you stay on that stack and turn interrupts back on, you could take another interrupt and clobber the frame you're already in. However, in it I saw a possible way to simplify the KPTI trampoline logic and avoid having to deal with %gs.

A few weeks into the project, John Levon joined us at work. He had previously worked on a bunch of Xen-related stuff as well as other parts of the kernel very close to where we were, so he quickly got up to speed with the KPTI work as well. He and I drafted out a "crazy idea" on the whiteboard one afternoon where we would use IST for all interrupts on the system, and put the "stack" they used in the KPTI page on the end of the cpu_t. Then, they could easily use stack-relative addresses to get the page table to change to, then pivot their stack to the real kernel stack memory, and throw away (almost) all the conditional logic. A few days later, we had convinced each other that this was the way to go.

Two of the most annoying x86 issues we had to work around were related to the SYSENTER instruction. This instruction is used to make "fast" system calls in 32-bit userland. It has a couple of unfortunate properties: firstly, it doesn't save or restore RFLAGS, so the kernel code has to take care of this (and be very careful not to clobber any of it before saving or after restoring it). Secondly, if you execute SYSENTER with the TF ("trap"/single-step flag) set by a debugger, the resulting debug trap's frame points at kernel code instead of the user code where it actually happened. The first one requires some careful gymnastics on the entry and return trampolines specifically for SYSENTER, while the second is a nasty case that is incidentally made easier by using IST. With IST, we can simply make the debug trap trampoline check for whether we took the trap in another trampoline's code, and reset %cr3 and the destination stack. This works for single-stepping into any of the handlers, not just the one for SYSENTER.

To make debugging easier, we decided that traps like the debug/single-step trap (as well as faults like page faults, #GP, etc.) would push their interrupt frame in a different part of the KPTI state page to normal interrupts. We applied this change to all the traps that can interrupt another trampoline (based on the instructions we used). These "paranoid" traps also set a flag in the KPTI struct to mark it busy (and jump to the double-fault handler if it is), to work around some bugs where double-faults are not correctly generated.

It's been a long and busy two months, with lots of time spent building, testing, and validating the code. We've run it on as many kinds of machines as we could get our hands on, to try to make sure we catch issues. The time we've spent on this has been validated several times in the process by finding bugs that could have been nasty in production.

One great example: our patches on Westmere-EP Xeons were causing busy machines to throw a lot of L0 I-cache parity errors. This seemed very mysterious at first, and it took us a few times seeing it to believe that it was actually our fault. This was actually caused by the accidental activation of a CPU errata for Westmere (B52, "Memory Aliasing of Code Pages May Cause Unpredictable System Behaviour") -- it turned out we had made a typo and put the "cacheable" flag into a variable named flags instead of attrs where it belonged when setting up the page tables. This was causing performance degradation on other machines, but on Westmere it causes cache parity errors as well. This is a great example of the surprising consequences that small mistakes in this kind of code can end up having. In the end, I'm glad that that erratum existed, otherwise it may have been a long time before we caught that bug.

As of this week, Mike and Philip have committed the OpenBSD patches for KPTI to their repository, and the patches for illumos are out for review. It's a nice kind of symmetry that the two projects who started on the work together after the public disclosure at the same time are both almost ready to ship at the same time at the other end. I'm feeling hopeful, and looking forward to further future collaborations like this with our cousins, the BSDs.

By design, email message headers need to be public, for exchanges to happen. The body of the message can be encrypted by the user, if desired. Moreover, there is no way to prevent the host from having access to the virtual machine. Therefore, full disk encryption (at rest) may not be necessary.

Given our low memory requirements, and the single-purpose concept of email service, Roundcube or other web-based IMAP email clients should be on a different VPS.

Antivirus software users (usually) have the service running on their devices. ClamAV can easily be incorporated into this configuration, if affected by the types of malware it protects against, but will require around 1GB additional RAM (or another VPS).

Every email message is important, if properly delivered, for Bayes classification. At least 200 ham and 200 spam messages are required to learn what one considers junk. By default (change to use case), a rspamd score above 50% will send the message to Spam/. Moving messages in and out of Spam/ changes this score. After 95%, the message is flagged as "seen" and can be safely ignored.

Spamd is effective at greylisting and stopping high volume spam, if it becomes a problem. It will be an option when IPv6 is supported, along with bgp-spamd.

System mail is delivered to an alias mapped to a virtual user served by the service. This way, messages are guaranteed to be delivered via encrypted connection. It is not possible for real users to alias, nor mail an external mail address with the default configuration. e.g. [email protected] is wheel, with an alias mapped to (virtual) [email protected], and user (puffy) can be different for each.

After a several year gap, I finally took another week-long programming retreat, where I could work in hermit mode, away from the normal press of work. My wife has been generously offering it to me the last few years, but I’m generally bad at taking vacations from work.

Recently, Theo de Raadt (deraadt@) described a new type of mitigation he has been working on together with Stefan Kempf (stefan@):

This is now available in snapshots, and people are finding the first problems in the ports tree already. So far, few issues have been uncovered, but as Theo points out, more testing is necessary:

Fairly good results.

So, everybody, install the latest snapshot and try all your favorite ports. This is the time to report issues you find, so there is a good chance this additional security feature is present in 6.3 (and works with third party software from packages).

AsiaBSDcon review, Meltdown and Spectre Patches in FreeBSD stable, Interview with MidnightBSD founder, 8 months with TrueOS, mysteries of GNU and BSD split

Speculative execution vulnerability mitigation is a work in progress. This advisory addresses the most significant issues for FreeBSD 11.1 on amd64 CPUs. We expect to update this advisory to include 10.x for amd64 CPUs. Future FreeBSD releases will address this issue on i386 and other CPUs. freebsd-update will include changes on i386 as part of this update due to common code changes shared between amd64 and i386, however it contains no functional changes for i386 (in particular, it does not mitigate the issue on i386).

Recently, I have taken a little dip into the world of BSD. As part of my attempt to understand the BSD world a little better, I connected with Lucas Holt (MidnightBSD founder and lead developer) to ask him a few questions about his project. Here are his answers.

It’s FOSS: Please explain MidnightBSD in a nutshell. How is it different than other BSDs?

Lucas Holt: MidnightBSD is a desktop focused operating system. When it’s considered stable, it will provide a full desktop experience. This differs from other efforts such as TrueOS or GhostBSD in that it’s not a distro of FreeBSD, but rather a fork. MidnightBSD has its own package manager, mport as well as unique package cluster software and several features built into user land such as mDNSresponder, libdispatch, and customizations throughout the system.

It’s FOSS: Who is MidnightBSD aimed at?

Lucas Holt: The goal with MidnightBSD has always been to provide a desktop OS that’s usable for everyday tasks and that even somewhat non technical people can use. Early versions of Mac OS X were certainly an inspiration. In practice, we’re rather far from that goal at this point, but it’s been an excellent learning opportunity.

It’s FOSS: What is your background in computers?

Lucas Holt: I started in technical support at a small ISP and moved into web design and system administration. While there, I learned BSDi, Solaris and Linux. I also started tinkering with programming web apps in ASP and a little perl CGI. I then did a mix of programming and system administration jobs through college and graduated with a bachelors in C.S. from Eastern Michigan University. During that time, I learned NetBSD and FreeBSD. I started working on several projects such as porting Apple’s HFS+ code to FreeBSD 6 and working on getting the nforce2 chipset SATA controller working with FreeBSD 6, with the latter getting committed. I got a real taste for BSD and after seeing the lack of interest in the community for desktop BSDs, I started MidnightBSD. I began work on it in late 2005.

It’s FOSS: I recently installed TrueOS. I was disappointed that a couple of the programs I wanted were not available. The FreeBSD port system looked mildly complicated for beginners. I’m used to using pacman to get the job done quickly. How does MidnightBSD deal with ports?

Lucas Holt: MidnightBSD has it’s own port system, mports, which shared similarities with FreeBSD ports as well as some ideas from OpenBSD. We decided early on that decent package management was essential for regular users. Power users will still use ports for certain software, but it’s just so time consuming to build everything. We started work on our own package manager, mport.

It’s FOSS: During your June 2015 interview on BSDNow, you mentioned that even though you support both i386 and amd64, that you recommend people choose amd64. Do you have any plans to drop i386 support in the future, like many have done?

Lucas Holt: Yes, we do plan to drop i386 support, mostly because of the extra work needed to build and maintain packages. I’ve held off on this so far because I had a lot of feedback from users in South America that they still needed it. For now, the plan is to keep i386 support through 1.0 release. That’s probably a year or two out.

It’s FOSS: What desktop environments does MidnightBSD support?

Lucas Holt: The original plan was to use Etoile as a desktop environment, but that project changed focus. We currently support Xfce, Gnome 3, WindowMaker + GNUstep + Gworkspace as primary choices. We also have several other window managers and desktop environments available such as Enlightenment, rat poison, afterstep, etc.

It’s FOSS: What is MidnightBSD’s default filesystem? Do you support DragonflyBSD’s HAMMER filesystem? What other filesystems?

Lucas Holt: Boot volumes are UFS2. We also support ZFS for additional storage. We have read support for ExFat, NTFS, ext2, CD9660. NFS v3 and v4 are also supported for network file systems.

It’s FOSS: Is MidnightBSD affected by the recent Spectre and Meltdown issues?

Lucas Holt: Yes. Most operating systems were affected by these issues. We were not informed of the issue until the general public became aware. Work is ongoing to come up with appropriate mitigations. Unfortunately, we do not have a patch yet.

It’s FOSS: The Raspberry Pi and its many clones have made the ARM platform very popular. Are there any plans to make MidnightBSD available on that platform?

Lucas Holt: No immediate plans. ARM is an interesting architecture, but by the very nature of SoC designs, takes a lot of work to support a broad number of devices. It might be possible when we stop supporting i386 or if someone volunteers to work on the ARM port.

It’s FOSS: Does MidnightBSD offer support for Linux applications?

Lucas Holt: Yes, we offer Linux emulation. It’s emulating a 2.6.16 kernel currently and that needs to be updated so support newer apps. It’s possible to run semi-recent versions of Firefox, Thunderbird, Java, and OpenOffice on it though. I’ve also used it to host game servers in the past and play older games such as Quake 3, enemy territory, etc.

It’s FOSS: Could you comment on the recent dust-up between the Pale Moon browser developers and the team behind the OpenBSD ports system?

Lucas Holt: I found this discussion frustrating. Many of the BSD projects hear a lot of complaints about browser availability and compatibility. With Firefox moving to Rust, it makes it even more difficult. Then you get into branding issues. Like Firefox, the Pale Moon developers have decided to protect their brand at the cost of users. Unlike the Firefox devs, they’ve made even stranger requirements for branding. It is not possible to use a system library version of anything with Pale Moon and keep their branding requirements. As such, we cannot offer Pale Moon in MidnightBSD.

It’s FOSS: If someone wanted to contribute to your project, both financial and technical, how can they do that?

Lucas Holt: Financial assistance for the project can be submitted online. We have a page outlining how to make donations with Patreon, Paypal or via bitcoin. Donations are not tax deductible. You can learn more at http://www.midnightbsd.org/donate/

I would like to thank Lucas for taking the time to reply to my many questions. For more information about MidnightBSD or to download it, please visit their website. The most recent version of MidnightBSD is 0.8.6.

I vowed to write down what I felt about TrueOS if I ever got to the six month mark of usage. This is just that. This is neither a tutorial, nor a piece of evangelism dedicated towards it.

This is also not a review of specific parts of TrueOS such as Lumina or AppCafe, since I don't use them at all.

In the spirit of presenting a screen shot, here is my i3wm displaying 4 windows in one screen - a configuration that I never use. https://inflo.ws/blog/images/trues-screenshot.png

I need a tiling wm with multi-desktop capability. As regards what I do with a computer, it is fairly straightforward to describe if I just list down my most frequently used applications.

All of these are available as binary packages from the repository. Since I use Intellij Ultimate edition, I decided to download the no-jdk linux version from the website rather than install it. This would make sure that it gets updated regularly.

I ran various Linux distributions from 2001 all the way till 2009, till I discovered Arch, and continued with it till 2017. I tried out Void for two months before I switched to TrueOS.

Over the last few years, I started feeling like no matter which Linux distribution I touched, they all just stopped making a lot of sense. Generally in the way things were organised, and particularly in terms of software like systemd, which just got pushed down my throat. I couldn't wrap my head around half the things going on in my computer.

Mostly I found that Linux distributions stopped becoming a collection of applications that got developed together to something more coupled by software mechanisms like systemd - and that process was more and more opaque. I don't want to talk about the merits and de-merits of systemd, lets just say that I found it of no use and an unnecessary hassle.

In February, I found myself in charge of the entire technology stack of a company, and I was free to make choices. A friend who was a long time FreeBSD user convinced me to try it on the servers. My requirement then was to run Postgres, Rabbitmq, Nginx and a couple of JVM processes. The setup was zero hassle and it hasn't changed much in a year.

About three months of running FreeBSD-11.x on servers was enough for me to consider it for my laptop. I was very apprehensive of hardware support, but luckily my computer is a Thinkpad, and Thinkpads sort of work out of the box with various BSDs.

My general requirements were:

I had to pick from FreeBSD, NetBSD and OpenBSD, since these were the major BSDs that I was familiar with. One of my requirements was that I needed to be able to run VMs just in case I needed to test something on Windows/Linux. This ruled out OpenBSD. Then I was left with NetBSD and FreeBSD. NetBSD's driver support for newer Intel chip-sets were questionable, and FreeBSD was the only choice then.

When I was digging through FreeBSD forums, I found out that running the 11.x RELEASE on my laptop was out of the question since it didn't have proper drivers for my chip-set either.

A few more hours of digging led me to GhostBSD and TrueOS. I picked TrueOS straightaway because - well because TrueOS came from the old PC-BSD and it was built off FreeBSD-12-CURRENT with the latest drivers integrated.

I downloaded the UNSTABLE version available in June 2017, backed up ALL my data and home directory, and then installed it. There were no glitches during installation - I simply followed the installation as described in the handbook and everything was fine.

My entire switch from Arch/Void to TrueOS took about an hour, discounting the time it took to backup my data to an external hard disk. It was that easy. Everything I wanted to work just worked, everything was available in the repo.

Tweaks from cooltrainer.org : I discovered this excellent tutorial that describes setting up a FreeBSD 11 desktop. It documents several useful tweaks, some of which I applied. A few examples - Fonts, VirtualBox, Firewall, UTF-8 sections.

The open-rc init system is familiar and is well documented.

This was an alien concept to me, but the first time I did an update without waiting for a CDN sync to finish, my computer booted into the shell and remained there. The friendly people at TrueOS discourse asked me to roll back to an older BE and wait for sync to finish.

My experience with boot environments was enough to convince me about the utility of ZFS. I am still reading about it and trying things out, and whatever I read just convinces me more about why it is good.

Coming from the Linux world, how the FreeBSD file-system is laid out seemed odd at first. Then I realised that it was the Linux distros that were doing the odd thing. e.g : The whole OS is split into base system and applications. All the non base system configurations and apps go into /usr/local. That made a lot of sense.

The handbooks for both TrueOS and FreeBSD are really really good. For e.g, I kept some files in an LUKS encrypted drive (when I used Arch Linux). To find an equivalent, all I had to do was read the handbook and look at the GELI section. It is actually nice being able to go to a source like Handbook and things from there just work.

The TrueOS community on both Telegram as well as on Discourse are very friendly and patient. They help out a lot and do not get upset when I pose really stupid questions. TrueOS core developers hangout in the Telegram chat-room too, and it is nice being able to talk to them directly about things.

The following things that worked during my Linux tenure doesn't work in TrueOS.

These are not major concerns for the kind of work I do, so it doesn't bother me much. I run a WinXP VM to play some old games, and a Bunsenlabs installation for Linux things like Hangouts/Netflix.

I don't have a video calling system setup in TrueOS because I use my phone for both voice and video calls exclusively.

Great community - whether on Discourse or on the telegram channel, the people make you feel welcome. If things go unanswered, someone will promise to work on it/file a bug/suggest work-arounds.

Switching to TrueOS was philosophical as well - I thought a lot more about licenses, and I have arrived at the conclusion that I like BSD more than GPL. I believe it is a more practical license.

I believe TrueOS is improving continuously, and is a great desktop UNIX if you put some time into it.

Good news for people doing upgrades only once per year: syspatches will be provided for both supported releases. The commit from T.J. Townsend (tj@) speaks for itself:

Thanks to all the developers involved in providing these!

Recently, I was playing around with the split command-line tool on Mac OS X, and I decided to chop a 4000-line file into 4000 separate single-line files. However, when I attempted to run split -l1, I ran into a funny error:

split: too many files

Curious to see if any splitting had occurred, I ran ls and sure enough, a huge list of filenames appeared, such as:

Now I could see why you'd run out of unique filenames - there are only 26 letters in the alphabet and these filenames were only three letters long. Also, they all seemed to begin with the letter "x".

I checked the manual for split's defaults and confirmed what I was seeing:

each file into which the file is split is named by the prefix followed by a lexically ordered suffix using suffix_length characters in the range 'a-z'. If -a is not specified, two letters are used as the suffix....with the prefix 'x' and with suffixes as above.

Got it, so running split with the defaults for prefix name and suffix length will give me filenames that always start with the letter "x" followed by two-letter alphabetical permutations composed of a-z letters, with repeats allowed. I say "repeats allowed" because I noticed filenames such as xaa and xbb in the output.

Side node: The reason why I say "permutations" rather than "combinations" is because letter order matters. For example, xab and xba are two distinct and legitimate filenames. Here's a nice explanation about the difference between permutations and combinations.

So how many filenames can you get from the BSD split tool using the defaults? There are permutation formulas out there for repeating values and non-repeating values. Based on split's behavior, I wanted to use the repeating values formula:

nr where n equals the number of possible values (26 for a-z) and r equals the number of values (2, since there are only 2 letters after "x" in the filename).

262 = 676

So the total number of filename permutations allowed with BSD split's defaults should be 676.

To double check, I ran ls | wc -l to get the total number of files in my split_test directory. The output was 677. If you subtract my original input file, input.txt, then you have 676, or the number of permutations split would allow before running out of filenames!

Neat. But I still wanted my 4000 files.

While 262 permutations doesn't support 4000 different filenames, I wondered if I could increase r to 3. Then, I'd have 17,576 different filename permutations to play with - more than enough.

Earlier, I remembered the manual mentioning suffix length:

-a suffix_length

So I passed 3 in with the -a flag and guess what? I got my 4000 files!

But that was a lot of work. It would be great if split would just handle these permutations and suffix lengths by default!

In fact, I vaguely remember splitting large files into smaller ones with numerical filenames, which I prefer. I also remember not having to worry about suffixes in the past. But numerical filenames didn't seem to be an option with split installed on Mac OS X - there was no mention of it in the manual.

Turns out that I was remembering GNU split from using the Debian OS two years ago, a different flavor of the split tool with different defaults and behaviors.

Michael Lucas is speaking at mug.org 10 April 2018

We’ll cover OpenBSD’s defensive approach to OS security, help you Understanding Syscall Conventions for Different Platforms, Mishandling SMTP Sender Verification, how the cd command works, and the LUA boot loader coming to FreeBSD.

The meaning of Pledge is same as in the real world, that is, “a solemn promise or undertaking”.

So, in OpenBSD: Calling pledge in a program means to promise that the program will only use certain resources.

It limits the operation of a program. Example: You wrote a program named ‘abc’ that only needed the stdio to just print something to stdout.

This happens because before executing other codes of your program, the code first pledges not to use anything other than stdio promise/operations. But, opening a shell or root shell will call several other system-calls which are distributed in lots of other promises like “stdio”, “proc”, “exec” etc. They are all forbidden because the program has already promised not to use any promises other than stdio.

Pledge is not a system call filter. So, it is not used to restrict system calls.

Pledge works on stdio, dns, inet, etc. promises but not directly on system calls like read, write, etc. And, unique functionality of pledge() is that it works on behavioral approach not just like 1:1 approach with the system calls.

On 11 December 2017, Theo de Raadt said:

Not long ago I started looking into FreeBSD kernel exploitation. There are only a few resources but probably the best starting point is argp's Phrack article from 2009[0]. And while he does only provide one technique, I wanted to understand it and port it to a modern FreeBSD release before describing new, own researched techniques.

Well, at least this was my plan. In reality I ended researching how different operating systems resp. the same operating system but for different architectures implement syscalls. Hence, new exploiting methods have to wait for another post. In this one I want to describe my personal FAIL while porting argp's exploit example to a FreeBSD 11.1-RELEASE running on a 64bit processor. Maybe this will give other people interested in kernel stuff some insights they didn't know before. If you already know how syscalls work on 32bit and 64bit *BSD because you are an experienced exploit or kernel developer, you will probably want to search for something

argp described in his article vulnerable code consisting of a loadable kernel module which exposes a syscall to the userland. Because it was written around the time when FreeBSD 8-RELEASE came out and because he has written himself that the code needs smaller adjustments to work with this version (it was written for FreeBSD 7) I thought I will first port it to

As written above, the syscall convention for the 32bit architecture is different from the one for the 64bit architecture. Indeed, a syscall on a 32bit FreeBSD system passes the arguments via the stack while the syscall offset is stored in the EAX register. The transfer into the kernel address space is done in 'cpu_fetch_syscall_args' in 'sys/i386/i386/trap.c'.

That is, 'params' points to ESP+4 bytes offset. Later, the arguments are copied into the kernel space which is referenced by 'sa->args'. 'args' is an array of eight 'register_t' which is defined as 'int32_t' on the 32bit platform in comparison to the 64bit platform. And as 'struct args' only

To bring this to an end: argp's version only worked for his special choice of arguments and only on 32bit. On 32bit FreeBSD platforms the arguments are transferred into kernel space by 4 byte integers, hence it will only work for integers anyway. On 64bit FreeBSD platforms we have to use syscall(2) in the intended way.

It all started with one of those rare spam mails that got through.

This one was hawking address lists, much like the ones I occasionally receive to addresses that I can not turn into spamtraps. The message was addressed to, of all things, [email protected]. (The message with full headers has been preserved here for reference).

Yes, that's right, they sent their spam to root@. And a quick peek at the headers revealed that like most of those attempts at hawking address lists for spamming that actually make it to a mailbox here, this one had been sent by an outlook.com customer.

The problem with spam delivered via outlook.com is that you can't usefully blacklist the sending server, since the largish chunk of the world that uses some sort of Microsoft hosted email solution (Office365 and its ilk) have their usually legitimate mail delivered via the very same infrastructure.

And since outlook.com is one of the mail providers that doesn't play well with greylisting (it spreads its retries across no less than 81 subnets (the output of 'echo outlook.com | doas smtpctl spf walk' is preserved here), it's fairly common practice to just whitelist all those networks and avoid the hassle of lost or delayed mail to and from Microsoft customers.

I was going to just ignore this message too, but we've seen an increasing number of spammy outfits taking advantage of outlook.com's seeming right of way to innocent third parties' mail boxes.

So I decided to try both to do my best at demoralizing this particular sender and alert outlook.com to their problem. I wrote a messsage (preserved here) with a Cc: to [email protected] where the meat is,

What happened next is quite amazing.

If my analysis is correct, it may not be possible for senders who are not themselves outlook.com customers to actually reach the outlook.com abuse team.

Any student or practitioner of SMTP mail delivery should know that SPF records should only happen on ingress, that is at the point where the mail traffic enters your infrastructure and the sender IP address is the original one. Leave the check for later when the message may have been forwarded, and you do not have sufficient data to perform the check.

Whenever I encounter incredibly stupid and functionally destructive configuration errors like this I tend to believe they're down to simple incompetence and not malice.

But this one has me wondering. If you essentially require incoming mail to include the contents of spf.outlook.com (currently no less than 81 subnets) as valid senders for the domain, you are essentially saying that only outlook.com customers are allowed to communicate.

If that restriction is a result of a deliberate choice rather than a simple configuration error, the problem moves out of the technical sphere and could conceivably become a legal matter, depending on what outlook.com have specified in their contracts that they are selling to their customers.

But let us assume that this is indeed a matter of simple bad luck or incompetence and that the solution is indeed technical.

I would have liked to report this to whoever does technical things at that domain via email, but unfortunately there are indications that being their customer is a precondition for using that channel of communication to them.

I hope they fix that, and soon. And then move on to terminating their spamming customers' contracts.

The main lesson to be learned from this is that when you shop around for email service, please do yourself a favor and make an effort to ensure that your prospective providers actually understand how the modern-ish SMTP addons SPF, DKIM and DMARC actually work.

Otherwise you may end up receiving more of the mail you don't want than what you do want, and your own mail may end up not being delivered as intended.

As I have previously attempted several times in the past, I am (finally) very close to switch to OpenBSD, a more stable and reliable operating system that I like. Before starting to make the actual change on both personal and work computer, I started testing some of the tools I’m currently using, and understand what are the expectations.

In general I didn’t encounter issues, or when I did, I found the answers in the documentation (which is really great), or various forums. I didn’t find however any questions regarding Proxy Minions on OpenBSD which is why I thought it might be helpful to share my experience.

With these said, I started playing with Salt, and it was simple and straightforward. First step - install Salt: pkg_add salt. This will bring several ports for Python futures, ZeroMQ, or Tornado which are needed for Salt.

After configuring the pillar_roots in the /etc/salt/master config file for the Master, I started up the master process using rcctl:

The Salt package for OpenBSD comes with the rc file for salt-proxy as well, /etc/rc.d/salt_proxy

I have managed to startup a Proxy Minion, but what about many? Executing the three commands above for each and every device is tedious and cannot scale very well. I thus have figured the following way:

Extending the same to a (very) large number of Proxy Minions, you can easily manage the rc files and start the services using a Salt State executed on the regular Minion:

These two steps would suffice to start an arbitrary number of Proxy Minions, and the command executed will always be the same regardless how many processes you aim to manage.

I am still a novice when it comes to OpenBSD, I have plenty to learn, but it looks like the transition will be much smoother than I expected. I am already looking forward to the handover, and - most importantly - I will no longer be using systemd. :-)

As you may know, the Lua (http://www.lua.org) boot loader has been in the works for some time. It started out life as a GSoC in 2014 by Pedro Souza mentored by Wojciech A. Koszek. Rui Paulo created a svn project branch to try to integrate it. I rebased that effort into a github branch which Pedro Arthur fixed up. Over the past year, I've been cleaning up the boot loader

I'm pleased to announce that all these threads of development have converged and I'll be pushing the FreeBSD Lua Loader later today. This loader uses Lua as its scripting language instead of FORTH. While co-existance is planned, the timeline for it is looking to be a few weeks

To try the loader, you'll need to build WITHOUT_FORTH=yes and WITH_LOADER_LUA=yes. Fortunately, you needn't do a full world to do this, you can do it in src/stand and install the result (be sure to have the options for both the build and the install). This will replace your current

While it's possible to build all the /boot/loader variants with Lua, I've just tested a BIOS booting /boot/loader both with and without menus enabled. I've not tested any of the other variants and the instructions for testing some of them may be rather tedious (especially UEFI, if you want a

I put MFC after a month on the commit. Due to the heroic (dare I say almost crazy) work of Kyle Evans on merging all the revs from -current to 11, I'm planning a MFC to 11 after the co-existence issues are hammered out. In 11, FORTH will be the default, and Lua will be built by default, but users will have to do something to use it. 12, both FORTH and Lua will be built and installed, with Lua as default (barring unforeseen complications). Once the co-existence stuff goes in, I imagine we'll make the switch to Lua by default shortly after that. In 13, FORTH will be removed unless there's a really really compelling case made to keep it.

So please give it a spin and give me any feedback, documentation updates and/or bug fixes. I'm especially interested in reviews from people that have embedded Lua in other projects or experts in Lua that can improve the robustness of the menu code.

Bitcoin is a valuable popular open-source cryptocurrency that was invented by Satoshi Nakamoto in 2009. Bitcoins have value because they possess same characteristics like money (durability, portability, fungibility, scarcity, divisibility, and recognizability), but based on the properties of mathematics rather than on physical properties (like gold and silver) or trust in central authorities (like fiat currencies). In short, Bitcoin is backed by mathematics.

A wallet is nothing more than a pair of public and private keys that are created by a client to store the digital credentials for your bitcoin.

There are several types of wallets:

A blockchain is a ledger that records bitcoin transactions. The blockchain is a distributed database that achieves independent verification of the chain of ownership. Each network node stores its own copy of the blockchain. Transactions will broadcast on the bitcoin network, and about 2400 transactions create a block. These blocks are building blocks of the blockchain.

Mining is the process of dedicating computing power to process transactions, secure the network, and keep everyone in the system synchronized together. It has been designed to be fully decentralized.

You have more chances if you participate with others to create a block. In a pool, all participating miners get paid every time a participating server solves a block. The payment depends on the amount of work an individual miner contributed to help find that block.

A full node is a client that fully validates transactions and blocks. Full nodes also help the network by accepting transactions and blocks from other full nodes, validating those transactions and blocks, and then relaying them to further full nodes.

bitcoind is a Bitcoin client under the MIT license in 32-bit and 64-bit versions for Windows, GNU/Linux-based OSes, Mac OS X, OpenBSD and FreeBSD as well.

Conclusion

In my last blog post, I dove into some of the code behind the sudo command. I thought this was pretty fun. sudo is one of those commands that I use quite often but haven’t had the chance to look into truly. I started thinking about other commands that I use on a daily basis but had little understanding of the internals of. The first command that came to mind is cd. cd stands for change directory. Simply put, it allows you to set your current working directory to a different directory.

I could dive in a little bit more into how #4 works, but let’s be honest, I’ve already read too much code at this point, and my eyes are starting to hurt.

How the term open source was created, running FreeBSD on ThinkPad T530, Moving away from Windows, Unknown Giants, as well as OpenBSD and FreeDOS.

In a few days, on February 3, the 20th anniversary of the introduction of the term "open source software" is upon us. As open source software grows in popularity and powers some of the most robust and important innovations of our time, we reflect on its rise to prominence.

I am the originator of the term "open source software" and came up with it while executive director at Foresight Institute. Not a software developer like the rest, I thank Linux programmer Todd Anderson for supporting the term and proposing it to the group.

This is my account of how I came up with it, how it was proposed, and the subsequent reactions. Of course, there are a number of accounts of the coining of the term, for example by Eric Raymond and Richard Stallman, yet this is mine, written on January 2, 2006.

It has never been published, until today.

The introduction of the term "open source software" was a deliberate effort to make this field of endeavor more understandable to newcomers and to business, which was viewed as necessary to its spread to a broader community of users. The problem with the main earlier label, "free software," was not its political connotations, but that—to newcomers—its seeming focus on price is distracting. A term was needed that focuses on the key issue of source code and that does not immediately confuse those new to the concept. The first term that came along at the right time and fulfilled these requirements was rapidly adopted: open source.

This term had long been used in an "intelligence" (i.e., spying) context, but to my knowledge, use of the term with respect to software prior to 1998 has not been confirmed. The account below describes how the term open source software caught on and became the name of both an industry and a movement.

In late 1997, weekly meetings were being held at Foresight Institute to discuss computer security. Foresight is a nonprofit think tank focused on nanotechnology and artificial intelligence, and software security is regarded as central to the reliability and security of both. We had identified free software as a promising approach to improving software security and reliability and were looking for ways to promote it. Interest in free software was starting to grow outside the programming community, and it was increasingly clear that an opportunity was coming to change the world. However, just how to do this was unclear, and we were groping for strategies.

At these meetings, we discussed the need for a new term due to the confusion factor. The argument was as follows: those new to the term "free software" assume it is referring to the price. Oldtimers must then launch into an explanation, usually given as follows: "We mean free as in freedom, not free as in beer." At this point, a discussion on software has turned into one about the price of an alcoholic beverage. The problem was not that explaining the meaning is impossible—the problem was that the name for an important idea should not be so confusing to newcomers. A clearer term was needed. No political issues were raised regarding the free software term; the issue was its lack of clarity to those new to the concept.

On February 2, 1998, Eric Raymond arrived on a visit to work with Netscape on the plan to release the browser code under a free-software-style license. We held a meeting that night at Foresight's office in Los Altos to strategize and refine our message. In addition to Eric and me, active participants included Brian Behlendorf, Michael Tiemann, Todd Anderson, Mark S. Miller, and Ka-Ping Yee. But at that meeting, the field was still described as free software or, by Brian, "source code available" software.

While in town, Eric used Foresight as a base of operations. At one point during his visit, he was called to the phone to talk with a couple of Netscape legal and/or marketing staff. When he was finished, I asked to be put on the phone with them—one man and one woman, perhaps Mitchell Baker—so I could bring up the need for a new term. They agreed in principle immediately, but no specific term was agreed upon.

Between meetings that week, I was still focused on the need for a better name and came up with the term "open source software." While not ideal, it struck me as good enough. I ran it by at least four others: Eric Drexler, Mark Miller, and Todd Anderson liked it, while a friend in marketing and public relations felt the term "open" had been overused and abused and believed we could do better. He was right in theory; however, I didn't have a better idea, so I thought I would try to go ahead and introduce it. In hindsight, I should have simply proposed it to Eric Raymond, but I didn't know him well at the time, so I took an indirect strategy instead.

Todd had agreed strongly about the need for a new term and offered to assist in getting the term introduced. This was helpful because, as a non-programmer, my influence within the free software community was weak. My work in nanotechnology education at Foresight was a plus, but not enough for me to be taken very seriously on free software questions. As a Linux programmer, Todd would be listened to more closely.

Later that week, on February 5, 1998, a group was assembled at VA Research to brainstorm on strategy. Attending—in addition to Eric Raymond, Todd, and me—were Larry Augustin, Sam Ockman, and attending by phone, Jon "maddog" Hall.

The primary topic was promotion strategy, especially which companies to approach. I said little, but was looking for an opportunity to introduce the proposed term. I felt that it wouldn't work for me to just blurt out, "All you technical people should start using my new term." Most of those attending didn't know me, and for all I knew, they might not even agree that a new term was greatly needed, or even somewhat desirable.

Fortunately, Todd was on the ball. Instead of making an assertion that the community should use this specific new term, he did something less directive—a smart thing to do with this community of strong-willed individuals. He simply used the term in a sentence on another topic—just dropped it into the conversation to see what happened. I went on alert, hoping for a response, but there was none at first. The discussion continued on the original topic. It seemed only he and I had noticed the usage.

Not so—memetic evolution was in action. A few minutes later, one of the others used the term, evidently without noticing, still discussing a topic other than terminology. Todd and I looked at each other out of the corners of our eyes to check: yes, we had both noticed what happened. I was excited—it might work! But I kept quiet: I still had low status in this group. Probably some were wondering why Eric had invited me at all.

Toward the end of the meeting, the question of terminology was brought up explicitly, probably by Todd or Eric. Maddog mentioned "freely distributable" as an earlier term, and "cooperatively developed" as a newer term. Eric listed "free software," "open source," and "sourceware" as the main options. Todd advocated the "open source" model, and Eric endorsed this. I didn't say much, letting Todd and Eric pull the (loose, informal) consensus together around the open source name. It was clear that to most of those at the meeting, the name change was not the most important thing discussed there; a relatively minor issue. Only about 10% of my notes from this meeting are on the terminology question.

But I was elated. These were some key leaders in the community, and they liked the new name, or at least didn't object. This was a very good sign. There was probably not much more I could do to help; Eric Raymond was far better positioned to spread the new meme, and he did. Bruce Perens signed on to the effort immediately, helping set up Opensource.org and playing a key role in spreading the new term.

For the name to succeed, it was necessary, or at least highly desirable, that Tim O'Reilly agree and actively use it in his many projects on behalf of the community. Also helpful would be use of the term in the upcoming official release of the Netscape Navigator code. By late February, both O'Reilly & Associates and Netscape had started to use the term.

After this, there was a period during which the term was promoted by Eric Raymond to the media, by Tim O'Reilly to business, and by both to the programming community. It seemed to spread very quickly.

On April 7, 1998, Tim O'Reilly held a meeting of key leaders in the field. Announced in advance as the first "Freeware Summit," by April 14 it was referred to as the first "Open Source Summit."

These months were extremely exciting for open source. Every week, it seemed, a new company announced plans to participate. Reading Slashdot became a necessity, even for those like me who were only peripherally involved. I strongly believe that the new term was helpful in enabling this rapid spread into business, which then enabled wider use by the public.

A quick Google search indicates that "open source" appears more often than "free software," but there still is substantial use of the free software term, which remains useful and should be included when communicating with audiences who prefer it.

When an early account of the terminology change written by Eric Raymond was posted on the Open Source Initiative website, I was listed as being at the VA brainstorming meeting, but not as the originator of the term. This was my own fault; I had neglected to tell Eric the details. My impulse was to let it pass and stay in the background, but Todd felt otherwise. He suggested to me that one day I would be glad to be known as the person who coined the name "open source software." He explained the situation to Eric, who promptly updated his site.

Coming up with a phrase is a small contribution, but I admit to being grateful to those who remember to credit me with it. Every time I hear it, which is very often now, it gives me a little happy twinge.

The big credit for persuading the community goes to Eric Raymond and Tim O'Reilly, who made it happen. Thanks to them for crediting me, and to Todd Anderson for his role throughout. The above is not a complete account of open source history; apologies to the many key players whose names do not appear. Those seeking a more complete account should refer to the links in this article and elsewhere on the net.

As I stated my previous post, I recently dug up my old ThinkPad T530 after the embarrassing stream of OS X security bugs this month. Although this ThinkPad ran Gentoo faithfully during my time in graduate school at Clemson, these days I'd much rather spend time my wife and baby than fighting with emerge and USE flags. FreeBSD has always been my OS of choice, and laptop support seems to be much better than it was a few years ago. In this guide, I'll show you the tweaks I made to wrestle FreeBSD into a decent experience on a laptop.

Unlike my usual posts, this time I'm going to assume you're already pretty familiar with FreeBSD. If you're a layman looking for your first BSD-based desktop, I highly recommend checking out TrueOS (previously PC-BSD): they've basically taken FreeBSD and packaged it with all the latest drivers, along with a user-friendly installer and custom desktop environment out of the box. TrueOS is an awesome project–the only reason I don't use it is because I'm old, grumpy, and persnickety about having my operating system just so.

Anyway, if you'd still like to take the plunge, read on. Keep in mind, I'm using a ThinkPad T530, but other ThinkPads of the same generation should be similarly compatible.

I've managed to get the Memory Sanitizer to work for the elementary base system utilities, like ps(1), awk(1) and ksh(1). This means that the toolchain is ready for tests and improvements. I've iterated over the basesystem utilities and I looked for bugs, both in programs and in sanitizers. The number of detected bugs in the userland programs was low, there merely was one reading of an uninitialized variable in ps(1).

I've prepared a prebuilt toolchain with Clang, LLVM, LLDB and compiler-rt for NetBSD/amd64. I prepared the toolchain on 8.99.12, however I have received reports that it works on other older releases. Link: llvm-clang-compilerrt-lldb-7.0.0beta_2018-01-24.tar.bz2

The archive has to be untarballed to /usr/local (however it might work to some extent in other paths).

This toolchain contains a prebuilt tree of the LLVM projects from a snapshot of 7.0.0(svn). It is a pristine snapshot of HEAD with patches from pkgsrc-wip for llvm, clang, compiler-rt and lldb.

I've tidied up inclusion of the internal namespace.h header in libc. This has hidden the usage of public global symbol names of:

The purpose of these changes was to stop triggering interceptors recursively. Such interceptors lead to sanitization of internals of unprepared (not recompiled with sanitizers) prebuilt code. It's not trivial to sanitize libc's internals and the sanitizers are not designed to do so. This means that they are not a full replacement of Valgrind-like software, but a a supplement in the developer toolbox. Valgrind translates native code to a bytecode virtual machine, while sanitizers are designed to work with interceptors inside the pristine elementary libraries (libc, libm, librt, libpthread) and embed functionality into the executable's code.

I've also reverted the vadvise(2) syscall removal, from the previous month. This caused a regression in legacy code recompiled against still supported compat layers. Newly compiled code will use a libc's stub of vadvise(2).

I've also prepared a patch installing dedicated headers for sanitizers along with the base system GCC. It's still discussed and should land the sources soon.

I've decided to not open new battlefields and return now to porting LLDB and fixing ptrace(2).

Thanks to a combination of some really impressive malware, bad clicking, and poor website choices, I had to blow away my Windows 10 installation. Not that it was Window’s fault, but a piece of malware had infected my computer when I tried to download a long lost driver for an even longer lost RAID card for a server. A word of advice – the download you’re looking for is never on an ad-infested forum in another language. In any case, I had been meaning to switch away from Windows soon. I didn’t have my entire plan ready, but now was as good a time as any.

My line of work requires me to maintain some form of Windows installation, so I decided to keep it in a VM rather than dual booting as I was developing code and not running any high-end visual stuff like games. My first thought was to install Arch or Gentoo Linux, but the last time I attempted a Gentoo installation it left me bootless. Not that there is anything wrong with Gentoo, it was probably my fault, but I like the idea of some sort of installer so I looked at rock-solid Debian. My dad had installed Debian on his sweet new cutting-edge Lenovo laptop he received recently from work. He often raves about his cool scripts and much more effective customized experience, but often complains about his hybrid GPU support as he has an Intel/Nvidia hybrid display adapter (he has finally resolved it and now boasts his 6 connected displays).

I didn’t want to install Windows again, but something didn’t feel right about installing some flavour of Linux. Back at home I have a small collection of FreeBSD servers running in all sorts of jails and other physical hardware, with the exception of one Debian server which I had the hardest time dealing with (it would be FreeBSD too if 802.11ac support was there as it is acting as my WiFi/gateway/IDS/IPS). I loved my FreeBSD servers, and yes I will write posts about each one soon enough. I wanted that cleanliness and familiarity on my desktop as well (I really love the ports collection!). It’s settled – I will run FreeBSD on my laptop. This also created a new rivalry with my father, which is not a bad thing either.

The first thing I needed to do was backup my Windows data. This was easy enough, just run a Windows Image Backup and it will- wait, what? Why isn’t this working? I didn’t want to fiddle with this too long because I didn’t actually need an image just the data. I ended up just copying over the files to an external hard disk. Once that was done, I downloaded and verified the latest FreeBSD 11.1 RELEASE memstick image and flashed it to my trusty 8GB Verbatim USB stick. I’ve had this thing since 2007, it works great for being my re-writable “CD”. I booted it up and started the installation. I knew this installer pretty well as I had test-installed FreeBSD and OpenBSD in VMs when I was researching a Unix style replacement OS last year. In any case, I left most of the defaults (I didn’t want to play with custom kernels right now) and I selected all packages. This downloaded them from the FreeBSD FTP server as I only had the memstick image. The installer finished and I was off to my first boot. Great! so far so good. FreeBSD loaded up and I did a ‘pkg upgrade’ just to make sure that everything was up to date.

Alright, time to get down to business. I needed nano. I just can’t use vi, or just not yet. I don’t care about being a vi-wizard, that’s just too much effort for me. Anyway, just a ‘pkg install nano’ and I had my editor. Next was obvious, I needed x11. XFCE was common, and there were plenty of tutorials out there. I wont bore you with those details, but it went something like ‘pkg install xfce’ and I got all the dependencies. Don’t forget to install SLiM to make it seamless. There are some configs in the .login I think. SLiM needs to be called once the boot drops you to the login so that you get SLiM’s nice GUI login instead of the CLI login screen. Then SLiM passes you off to XFCE. I think I followed this and this. Awesome. Now that x11 is working, it’s time to get all of my apps from Windows. Obviously, I can’t get everything (ie. Visual Studio, Office). But in my Windows installation, I had chosen many open-source or cross-compiled apps as they either worked better or so that I was ready to move away from Windows at a moments notice. ‘pkg install firefox thunderbird hexchat pidgin gpa keepass owncloud-client transmission-qt5 veracrypt openvpn’ were some immediate picks. There are a lot more that I downloaded later, but these are a few I use everyday. My laptop also has the same hybrid display adapter config that my dad’s has, but I chose to only run Intel graphics, so dual screens are no problem for me. I’ll add Nvidia support later, but it’s not a priority.

After I had imported my private keys and loaded my firefox and thunderbird settings, I wanted to get my Windows VM running right away as I was burning productive days at work fiddling with this. I had only two virtualisation options; qemu/kvm and bhyve. qemu/kvm wasn’t available in pkg, and looked real dirty to compile, from FreeBSD’s point of view. My dad is using qemu/kvm with virt-manager to manage all of his Windows/Unix VMs alike. I wanted that experience, but I also wanted packages that could be updated and I didn’t want to mess up a compile. bhyve was a better choice. It was built-in, it was more compatible with Windows (from what I read), and this is a great step-by-step article for Windows 10 on FreeBSD 11 bhyve! I had already tried to get virt-manager to work with bhyve with no luck. I don’t think libvirt connects with bhyve completely, or maybe my config is wrong. But I didn’t have time to fiddle with it. I managed it all through command lines and that has worked perfectly so far. Well sorta, there was an issue installing SQL Server, and only SQL Server, on my Windows VM. This was due to a missing ‘sectorsize=512’ setting on the disk parameter on the bhyve command line. That was only found after A LOT of digging because the SQL Server install didn’t log the error properly. I eventually found out that SQL Server only likes one sector size of disks for the install and my virtual disk geometry was incorrect.

I installed Windows 10 on my bhyve VM and I got that all setup with the apps I needed for work. Mostly Office, Visual Studio, and vSphere for managing our server farm. Plus all of the annoying 3rd party VPN software (I’m looking at you Dell and Cisco). Alright, with the Windows VM done, I can now work at work and finish FreeBSD mostly during the nights. I still needed my remote files (I setup an ownCloud instance on a FreeNAS jail at home) so I setup the client. Now, normally on Windows I would come to work and connect to my home network using OpenVPN (again, I have a OpenVPN FreeNAS jail at home) and the ownCloud desktop would be able to handle changing DNS destination IPs Not on FreeBSD (and Linux too?). I ended up just configuring the ownCloud client to just connect to the home LAN IP for the ownCloud server and always connecting the OpenVPN to sync things. It kinda sucks, but at least it works. I left that running at home overnight to get a full sync (~130GB cloud sync, another reason I use it over Google or Microsoft). Once that was done I moved onto the fstab as I had another 1TB SSD in my laptop with other files. I messed around with fstab and my NFS shares to my FreeNAS at home, but took them out as they made the boot time so long when I wasn’t at home. I would only mount them when my OpenVPN connected or manually. I really wanted to install SpaceFM, but it’s only available as a package on Debian and their non-package install script doesn’t work on FreeBSD (packages are named differently). I tried doing it manually, but it was too much work. As my dad was the one who introduced me to it, he still uses it as a use-case for his Debian setup. Instead I kept to the original PCManFM and it works just fine. I also loaded up my Bitcoin and Litecoin wallets and pointed them to the blockchain that I has used on Windows after their sync, they loaded perfectly and my balances were there. I kinda wish there was the Bitcoin-ABC full node Bitcoin Cash wallet package on FreeBSD, but I’m sure it will come out later.

The rest is essentially just tweaks and making the environment more comfortable for me, and with most programs installed as packages I feel a lot better with upgrades and audit checking (‘pkg audit -F’ is really helpful!). I will always hate Python, actually, I will always hate any app that has it’s own package manager. I do miss the GUI GitHub tool on Windows. It was a really good-looking way to view all of my repos. The last thing (which is increasing it’s priority every time I go to a social media site or YouTube) is fonts. My god I never thought it was such a problem, and UTF support is complicated. If anyone knows how to get all UTF characters to show up, please let me know. I’d really like Wikipedia articles to load perfectly (I followed this post and there are still some missing). There are some extra tweaks I followed here and here.

I successfully migrated from Windows 10 to FreeBSD 11.1 with minimal consequence. Shout out goes to the entire FreeBSD community. So many helpful people in there, and the forums are a great place to find tons of information. Also thanks to the ones who wrote the how-to articles I’ve referenced. I never would have gotten bhyve to work and I’d still probably be messing with my X config without them. I guess my take home from this is to not be afraid to make changes that may change how comfortable I am in an environment. I’m always open to comments and questions, please feel free to make them below. I purposefully didn’t include too many technical things or commands in this article as I wanted to focus on the larger picture of the migration as a whole not the struggles of xorg.conf, but if you would like to see some of the configs or commands I used, let me know and I’ll include some!

We believe code is truly agnostic and embrace inclusiveness regardless of a person’s individual beliefs. As such we only ask the following when participating in TrueOS public events and digital forums:

I decided to write this article as a gratitude for the recent fast answer of the FreeBSD/TrueOS community with my questions and doubts. I am impressed how fast and how they tried to help me about this operating system which I used in the past(2000-2007) but recently in 2017 I began to use it again.

Yes sir, yes. Our family, composed until now by OpenBSD, Alpine Linux and Docker is rapidly growing. And yes, sir. Yes. All together we're fighting against your best friends, the infamous, the ugliest, the worst...the dudes called the privacy cannibals. Do you know what i mean, sure?

But why we want to build a bootable usb stick with FreeDOS under our strong OpenBSD? The answer is as usual to fight against the privacy cannibals!

But let's start preparing our OpenBSD to put order in this chaos:

$ mkdir -p freedos/stuff

$ doas pkg_add -U nasm unzip dosfstools cabextract p7zip

Create another working directory, cd into it, unzip the archive that we've downloaded, return to the working root and create another twos directories. dd is one of the most important utilities in the unix world to manipulate at byte level input and output:

The dd utility copies the standard input to the standard output, applying any specified conversions. Input data is read and written in 512-byte blocks. If input reads are short, input from multiple reads are aggregated to form the output block. When finished, dd displays the number of complete and partial input and output blocks and truncated input records to the standard error output.

We're creating here a virtual disk with bs=1024 we're setting both input and output block to 1024bytes; with seek=20480 we require 20480bytes. This is the result:

-rw-r--r-- 1 taglio taglio 20971520 Feb 3 00:11 freedos.img.

Next we format the virtual disk using the MS-DOS filesystem. Go ahead:

We use the perl utility from syslinux to write the MBR of our virtual disk freedos.img. Next we create to loop virtual node using the OpenBSD utility vnconfig. Take care here because it is quite different from Linux, but as usual is clear and simple. The virtual nodes are associated to the downloaded fdboot.img and the newly created freedos.img. Next we mount the two virtual nodes cpartitions; in OpenBSD cpartition describes the entire physical disk. Quite different from Linux, take care.

Copy all files and directories in the new virtual node partition, extract the Lenovo cabinet in a new directory, copy the result in our new image, extract the afudos utility and like the others copy it. Umount the partitions and destroy the loop vnode.