BSD Now

By JT PenningtonNews Education How To Tech News

BSD Now episodes

April 20, 2016138: Rushing into BSD
This week on the show, we will be talking to Benedict Reushling about his role with the FreeBSD foundation and the journey that took him
This episode was brought to you by
Headlines
HardenedBSD introduces full PIE support
PIE base for amd64 and i386
Only nine applications are not compiled as PIEs
Tested PIE base on several amd64 systems, both virtualized and bare metal
Hoped to be to enabled it for ARM64 before or during BSDCan.
Shawn will be bringing ten Raspberry Pi 3 devices (which are ARM64) with to BSDCan, eight of which will be given out to lucky individuals. “We want the BSD community to hack on them and get ARM64/Aarch64 fully functional on them.”
***
Lessons learned from 30 years of MINIX
Eat your own dog food.
By not relying on idiosyncratic features of the hardware, one makes porting to new platforms much easier.
The Internet is like an elephant; it never forgets.
When standards exist (such as ANSI Standard C) stick to them.
Even after you have adopted a strategy, you should nevertheless reexamine it from time to time.
Keep focused on your real goal, Einstein was right: Things should be as simple as possible but not simpler.
***
pfSense 2.3 released
Rewrite of the webGUI utilizing Bootstrap
TLS v1.0 disabled for the GUI
Moved to a FreeBSD 10.3-RELEASE base
PHP Upgraded to 5.6
The "Full Backup" feature has been deprecated
Closed 760 total tickets of which 137 are fixed bugs
Known Regressions
OpenVPN topology change
IP aliases with CARP IP parent lose their parent interface association post-upgrade
IPsec IPComp does not work.
IGMP Proxy does not work with VLAN interfaces.
Many other updates and changes
***
OPNsense 16.1.10 released
openvpn: revive windows installer binaries
system: improved config history and backup pages layout
system: increased backup count default from 30 to 60
system: /var /tmp MFS awareness for crash dumps added
trust: add “IP security IKE intermediate” to server key usage
firmware: moved reboot, halt and defaults pages to new home
languages: updates to Russian, French, German and Japanese
Many other updates and changes
***
Interview - Benedict Reuschling - [email protected]
FreeBSD Foundation in Europe
***
News Roundup
Write opinionated workarounds
Colin Percival has written a great blog post this past week, specifically talking about his policy of writing “opinionated workarounds”.
The idea came about due to his working on multi-platform software, and the frustrations of dealing with POSIX violations
The crux of the post is how he deals with these workarounds. Specifically by only applying them to the particular system in which it was required. And doing so loudly.
This has some important benefits. First, it doesn’t potentially expose other systems to bugs / security flaws when a workaround doesn’t
“work” on a system for which it wasn’t designed. Secondly it’s important to complain. Loudly. This lets the user know that they are running on a system that doesn’t adhere to POSIX compliance, and maybe even get the attention of a developer who could remedy the situation.
***
Privilege escalation in calendar(1)
File this one under “Ouch that hurts” a new security vuln has been posted, this time against NetBSD’s ‘calendar’ command.
Specifically it looks like some of the daily scripts uses the ‘-a’ flag, which requires super-user privs in order to process all users calendar files and mail the results.
However the bug occurred because the calendar command didn’t drop priv properly before executing external commands (whoops!)
To workaround you can set run_calendar=NO in the daily.conf file, or apply the fixed binary from upstream.
***
PGCon 2016
PGCon 2016 is now only 4 weeks away
The conference will be held at the University of Ottawa (same venue as BSDCan) from May 17th to 20th
Tutorials: 17-18 May 2016 (Tue & Wed)
Talks: 19-20 May 2016 (Thu-Fri)
Wednesday is a developer unconference.
Saturday is a user unconference.
“PGCon is an annual conference for users and developers of PostgreSQL, a leading relational database, which just happens to be open source. PGCon is the place to meet, discuss, build relationships, learn valuable insights, and generally chat about the work you are doing with PostgreSQL. If you want to learn why so many people are moving to PostgreSQL, PGCon will be the place to find out why. Whether you are a casual user or you've been working with PostgreSQL for years, PGCon will have something for you.”
New to PGSQL? Just a user? Long time developers? This conference has something for you. A great lineup of talks, plus unconference days focused on both users and developers
***
CfP EuroBSDCon 2016
The call for papers has been issued for EuroBSDCon 2016 in Belgrade, Serbia
The conference will be held from the 22nd to 25th of September, 2016
The deadline for talk submissions is: Sunday the 8th of May, 2016
Submit your talk or tutorial proposal before it is too late
***
Beastie Bits
“FreeBSD Mastery: Advanced ZFS” has officially been released
Support of OpenBSD pledge(2) in programming Languages
pkgsrcCon 2016 -Call for Presentations
Christos Zoulas talks about blacklistd
Penguicon 2016 Lucas Track Schedule
Feedback/Questions
Peter - NVME
Jeremy - Wireless Gear
Ted - Rpi2 Packages - Cross Building Wiki
Geoff - Jail Failover
Zach - Graphical Bhyve?
***
1 hr 37 min
April 13, 2016137: FreeNAS Mini XL
This week on BSD Now, I’m out of town for the week, but we have a special unboxing video to share with you, that you won’t want to miss. That, plus the latest BSD news, is coming your way right now!
This episode was brought to you by
Headlines
Example of a FreeBSD bug hunting session by a simple user
Don’t be fooled, Olivier Cochard-Labbé is a bit more than just a FreeBSD user
Original founder of the FreeNAS project many years ago, and currently leads the BSD Router Project (designed as a replacement for “Big Iron” routers like Cisco’s etc)
However, he is not actually a committer on any of the BSD projects, and is mostly focused on networking, rather than development, so it is fair to call him a user
He walks us through a bug hunting session that started when he updated his wireless router
“My wireless-router configuration was complex: it involves routing, wireless in hostap mode, ipfw, snort, bridge, openvpn, etc.”
Provides helpful advice on writing problem reports to developers, including trying to reproduce your issue with as minimal a setup as possible. This both reduces the amount of setup a developer has to do to try to recreate your issue, and can often make it more obvious where the problem actually lies
As you might expect, the more he researched the problem, the more questions he had
The journey goes through the kernel debugger, learning dtrace, and reading some source code
In the end it seems the problem is that the bridge interface marks itself as down if none of the interfaces are in an ‘UP’ state. The wireless interface was in the unknown state, and was actually up, but when the wired interface was disconnected, this caused the bridge to mark it self as down.
***
How-to Install OpenBSD 5.9 plus XFCE desktop and basic applications
Now this is the way to do videos. Over at the RibaLinux blogspot site, we have a great video showing how to setup and install OpenBSD 5.9 with XFCE and basic desktop applications.
Along with the video tutorial, another nicety is the commands-used script, so you can see exactly how the setup was done, without having to pause/rewind the video to keep up. How to install PC-BSD 10.3
In addition to the OpenBSD 5.9 setup video, they just published a PC-BSD 10.3 installation video as well, check it out!
***
FreeBSD on xhyve tutorial
Originally only able to boot linux, xhyve, a “sort of” port of bhyve to OS X, can now run FreeBSD
This tutorial makes it much easier, providing a script
There are a few small command line flag differences from bhyve on FreeBSD
The tutorial also covers sharing a directory between the guest and the host, resizing and growing the disk for the guest, and converting a QEMU image to be run under xhyve
***
How to Configure SSHguard With IPFW Firewall On FreeBSD
It’s been a while, but UNIXMen has dropped on us another FreeBSD tutorial, this time on how to setup IPFW and ‘sshguard’ to protect your system.
In this tutorial they first lay down the rationale for picking IPFW as the firewall, but the reasons mainly boil down to IPFW being developed primarily on FreeBSD, and as such isn’t lagging behind when it comes to features / support.
Interestingly enough, they also go the route of adding their own /usr/local/etc/rc.firewall script which will be used to specify TCP/UDP ports to open through IPFW via the rc.conf file
Once that setup is complete (which you can just copy-n-paste) they then move onto ‘sshguard’ setup.
Specifically you’ll need to be sure to install the correct port/pkg, sshguard-ipfw in order to work in this setup, although sshguard-pf and friends are available also.
The article mentions that the name ‘sshguard’ can also be misleading, since it can be used to detect brute force attempts into a number of services.
From there a bunch of configuration is thrown at you, which will allow you to start making the most out of sshguard’s potential, well worth your read if you are using IPFW, or even PF and want to get the basics down of using sshguard properly.
***
FreeNAS Mini XL Video Unboxing
Beastie Bits
Amazon lists FreeBSD as 'Other Linux'
sbin/hammer: Make hammer commands print root volume path
sbin/hammer: Print volume list after volume-add|del
Front cover reveal for the upcoming 'FreeBSD Mastery: Advanced ZFS" book
If you don’t already have one, get your FreeBSD Pillow
Feedback/Questions
Daniel - SysVIPC
Shane - OpenToonz
***
40 min
April 06, 2016136: This is GNN
This week on the show, we will be interviewing GNN of the FreeBSD project to talk about the new TeachBSD initiative. That plus the latest BSD headlines, all coming your way right now!
This episode was brought to you by
Headlines
FreeBSD 10.3-RELEASE Announcement
FreeBSD 10.3 has landed, with extended support until April 30, 2018
This is likely to be the last extended support release, as starting with 11, the new support model will encourage upgrading to the latest minor version by ending support for the previous minor version approximately 2 months after each point release. The Major version / stable branch will still be supported for the same 5 year term. This will allow the FreeBSD project to move forward more quickly, while still providing the same level of long term support
The UEFI boot loader is much improved, and now supports booting root-on-ZFS, and the beastie menu
The beastie menu itself has been updated with support for ZFS Boot Environments
The CAM Target Layer (CTL) now supports High Availability, allowing the construction of much more advanced storage systems
The 64bit Linux Emulation Layer was backported
Reroot support was added, allowing the system to boot off of a minimal image, such as a mfsroot and then reload all of userland from a different root file system (such as iSCSI, NFS, etc)
The version of xz(1) has been updated to support multi-threaded compression
sesutil(8) has been introduced, making it easier to manage large storage nodes
Various ZFS updates
As usual, a huge number of driver updates are also included
***
How to use OpenBSD with Libreboot: detailed instructions
This tutorial covers installing OpenBSD on a Thinkpad X200 using Libreboot, a replacement for the traditional BIOS/firmware that comes from the manufacturer
“Since 5.9, OpenBSD supports EFI boot mode, which means that it also have had to support framebuffer out of the box, so lack of proprietary VGA BIOS blob is no longer a problem and you can boot it with unmodified Libreboot binary release 20150518.”
“In order to install OpenBSD on such a machine you will need someadditional preparations, since regular install59.fs won't work because bsd.rd doesn't have a framebuffer console.”
A few extra steps are required to get it going, but they are outlined in the post
This may be very interesting to those who prefer not to depend on binary blobs
***
Linking the FreeBSD base system with lld -- status update
The FreeBSD Foundation’s Ed Maste provides an update on the LLVM mailing list about the progress of replacing the GNU linker with the lld in the FreeBSD base system
“I'm pleased to report that I can now build a runnable FreeBSD system using lld as the linker (for buildworld), with a few workarounds and work-in-progress patches. I have not yet extensively tested the result but it is possible to login to the resulting system, and basic sanity tests I've tried are successful. Note that the kernel is still linked with ld.bfd.”
Outstanding Issues
Symbol version support (PR 23231). FreeBSD uses symbol versioning for backwards compatibility
Linker script expression support (PR 26731). The FreeBSD kernel linker scripts contain expressions not currently supported by lld
Library search paths. GNU LD automatically searches /lib, and lld does not
the -N flag makes the text and data sections RW and does not page-align data.
It is used by boot loader components.
The -dc flag assigns space to common symbols when producing relocatable output (-r). It is used by the /rescue build, which is a single binary assembled from a collection of individual tools (sh, ls, fsck, ...)
-Y adds a path to the default library search path. It is used by the lib32 build, which provides i386 builds of the system libraries for compatibility with i386 applications.
With the ongoing work, it might be possible for FreeBSD 11 to use lld by default, although it might be best to wait to throw that particular switch
***
Your favorite billion user company using BSD just flipped on encryption for all their users -- and it took 15 Engineers to do it
With the help of Moxie Marlinspike’s Open Whisper Systems, WhatsApp has integrated the ‘Signal’ encryption system for all messages, class, pictures, and videos sent between individuals or groups
It uses public key cryptography, very similar to GPG, but with automated public key servers
It also includes a system of QR codes to verify the identity of individuals in person, so you can be sure the person you are talking to is actually the person you met with
WhatsApp runs their billion user network, using FreeBSD, with only about 50 engineers
Only 15 of those engineers we needed to work on the project that has now deployed complete end-to-end encryption across the entire network
The Wired article is very detailed and well worth the read
***
Interview - George Neville-Neil - [email protected] / @gvnn3
Teaching BSD with Tracing
News Roundup
Faces of FreeBSD 2016: Scott Long
It’s been awhile since we’ve had a new entry into the “Faces of FreeBSD” series, but due to popular demand it’s back!
This installment features developer Scott Long, who currently works at NetFlix, previously at Yahoo and Adaptec.
Scott got a very early start into BSD, first discovering i386BSD 0.1 on a FTP server at Berkeley, back at 1992. From there on it’s been a journey, following along with FreeBSD since version 1.0 in 1993.
So what stuff can we blame Scott for? In his own words:
I’ve been a source committer since 2000. I got my start by taking over maintainership of the Adaptec ‘aac’ RAID driver. From 2002-2006 I was the Release Engineer and was responsible for the 5.x and 6.x releases. Though the early 5.x releases were not great, they were necessary stepping stones to the success of FreeBSD 6.x and beyond. I’m exceptionally proud of my role in helping FreeBSD move forward during that time.
I authored and maintained the ‘mfi’ and ‘mps’ storage drivers, the ‘udf’ filesystem driver, and several smaller sound and USB drivers. I’ve maintained, or at least touched, most of the storage device drivers in the system to some extent, and I implemented medium-grained locking on the CAM storage stack. Recently I’ve been working on overall system scalability and performance.
ASCII Flow
A website that lets to draw and share ASCII diagrams
Great for network layout maps, rack diagrams, protocol analysis etc
Use it in your presentations and slides
Sample
***
System Under Test: FreeBSD
Part of a series looking at testing across a number of projects
Outlines the testing framework of FreeBSD
Provides a mini-tutorial on how to run the tests
There are some other tests that are now covered, but this is due to a lack of documentation on the fact that the tests exist, and how to run them
There is much ongoing work in this area
***
Worst April Fools Joke EVER!
While a bad April Fool’s joke, it also shows some common misconceptions
The FreeBSD Foundation does not own the source repository, it is only the care taken of the trademark, and other things that require a single legal entity
OpenBSD and NetBSD are not ‘sub brands’ of FreeBSD
Bash was not ported to Windows, but rather Windows gained a system similar to FreeBSD’s linux_compat
It would be nice to have ZFS on Windows
***
Beastie Bits
Credit where credit's due...
M:Tier's OpenBSD packages and binpatches updated for 5.9
NYC BUG Meeting (2016-04-06) - Debugging with LLVM, John Wolfe
Need to create extremely high traffic loads? kq_sendrecv is worth checking out
If you're in the Maryland region, CharmBug has a meetup next week
How to get a desktop on DragonFly
Linux vs BSD Development Models
Feedback/Question
Paulo - ZFS Setup
Jonathan - Installation
Andrew - Career / School
1 hr 36 min
March 30, 2016135: Speciality MWL
This week on the show, we interview author Michael W Lucas to discuss his new book in the FreeBSD
This episode was brought to you by
Headlines
OpenBSD 5.9 Released early
Finished ahead of schedule! OpenBSD 5.9 has officially landed
We’ve been covering some of the ongoing changes as they landed in the tree, but with the official release it’s time to bring you the final list of the new hotness which landed.
First up:
Pledge - Over 70%! Of the userland utilities have been converted to use it, and the best part, you probably didn’t even notice
UEFI - Laptops which are pre-locked down to boot UEFI only can now be installed and used - GPT support has also been greatly improved
‘Less’ was replaced with a fork from Illumos, and has been further improved
Xen DomU support - OpenBSD now plays nice in the cloud
X11 - Broadwell and Bay Trail are now supported
Initial work on making the network stack better support SMP has been added, this is still ongoing, but things are starting to happen
802.11N! Specifically for the iwn/iwm drivers
In addition to support for UTF-8, most other locales have been ripped out, leaving only C and UTF-8 left standing in the wake
All and all, sounds like a solid new release with plenty of new goodies to play with. Go grab a copy now!
***
New routing table code (ART) enabled in -current
While OpenBSD 5.9 just landed, we also have some interesting work landing right now in -CURRENT as well. Specifically the new routing table code (ART) has landed:
“I just enabled ART in -current, it will be the default routing table backend in the next snapshots.
The plan is to squash the possible regressions with this new routing table backend then when we're confident enough, take its route lookup out of the KERNEL_LOCK(). Yes, this is one of the big steps for our network SMP improvements.
In order to make progress, we need your help to make sure this new backend works well on your setup. So please, go download the next snapshot and report back.
If you encounter any routing table regression, please make sure that you cannot reproduce it with your old kernel and include the output of # route -n show
for the 2 kernels as well as the dmesg in your report.
I know that simple dhclient(8) based setups work with ART, so please do not flood us too much. It's always great to know that things work, but it's also hard to keep focus ;)
Thank your very much for your support!”
There you have it folks! If 5.9 is already too stale for you, time to move over to -CURRENT and give the new routing tables a whirl.
***
fractal cells - FreeBSD-based All-In-One solution for software development startups
Fractal Cells is a suite that transforms a stock FreeBSD installation into an instant “Startup Software Development Platform”
It Integrates ZFS, PostgreSQL, OpenSMTPD, NGINX, OpenVPN, Redmine, Jenkins, Zabbix, Gitlab, and Ansible, all under OpenLDAP common authentication
The suite is available under the 2-clause BSD license
Provides all of the tools and infrastructure to build your application, including code review, issue tracking, continuous integration, and monitoring
An interesting way to make it easier for people to start building new applications and startups on top of FreeBSD
***
LinuxSecrets publishes guide on installing FreeBSD ezJail
Covers all of the steps of setting up ezjail on FreeBSD
Includes the instructions for updating the version of the OS in the jail
In a number of places the tutorial uses:
> cat << EOF >> /etc/rc.conf > setting=”value”
Instead, use: sysrc setting=”value”
It is safer, and easier to type
When you create the jail, if you specify an IP address, it is expected that this IP address is already setup on the host machine
If instead you specify: ‘em0|192.168.1.105’ (where em0 is your network interface), the IP address will be added as an alias when the jail starts, and removed from the host when the jail is stopped
You can also comma separate a list of addresses to have multiple IPs (possibly on different interfaces) in the jail
Although recently posted, this appears as if it might be an update to a previous tutorial, as there are a few old references that have not been updated (pkg_add, rc.d/ezjail.sh), while the start of the article clearly covers pkg(8)
***
Interview - Michael W. Lucas - [email protected] /
@mwlauthor
New Book: “FreeBSD Mastery: Specialty Filesystems”
News Roundup
NetBSD on Dreamcast
Ahh the dreamcast, so much promise. So much potential. If you are still holding onto your beloved dreamcast hoping that someday Sega will re-enter the console market… Then give it up now!
In the meantime, you can now do something more interesting with that box taking up space in the closet. We have a link to a GitHub repo where a user has uploaded his curses-based slide-show for the upcoming Fort-Wayne, Indiana meetup.
Aside from the novelty of using a curses-based slide setup, the presenter will also be displaying them from his beloved dreamcast, which “of course” runs NetBSD 7
The slide source code is available, which you too can view / compile and find out details of getting NetBSD boot-strapped on the DC.
***
OPNsense 16.1.7 Released
captive portal: add session timeout to status info
firewall: fix non-report of errors when filter reload errors couldn’t be parsed
proxy: adjust category visibility as not all of them were shown before
firmware: fix an overzealous upgrade run when the package tool only changes options
firmware: fixed the binary upgrade patch from 15.7.x in FreeBSD’s package tool
system: removed NTP settings from general settings
access: let only root access status.php as it leaks too much info
development: remove the automount features
development: addition of “opnsense-stable” package on our way to nightly builds
development: opnsense-update can now install locally available base and kernel sets
***
“FreeBSD Mastery: Advanced ZFS” in tech review
Most of the tech review is finished
It was very interesting to hear from many ZFS experts that they learned something from reading the review copy of the book, I was not expecting this
Many minor corrections and clarifications have been integrated
The book is now being copy edited
***
Why OpenBSD?
Frederic Cambus gives us a nice perspective piece today on what his particular reasons are for choosing OpenBSD.
Frederic is no stranger to UNIX-Like systems, having used them for 20 years now. In particular starting on Slackware back in ‘96 and moving to FreeBSD from 2000-2005 (around the 4.x series)
His adventure into OpenBSD began sometime after 2005 (specific time unknown), but a bunch of things left a very good impression on him throughout the years.
First, was the ease of installation, with its very minimalistic layout, which was one of the fastest installs he had ever done.
Second was the extensive documentation, which extends beyond just manpages, but into other forms of documentation, such as presentations and papers as well.
He makes the point about an “ecosystem of quality” that surrounds OpenBSD:
OpenBSD is an ecosystem of quality. This is the result of a culture of code auditing, reviewing, and a rigorous development process where each commit hitting the tree must be approved by other developers. It has a slower evolution pace and a more carefully planned development model which leads to better code quality overall. Its well deserved reputation of being an ultra secure operating system is the byproduct of a no compromise attitude valuing simplicity, correctness, and most importantly proactivity. OpenBSD also deletes code, a lot of code. Everyone should know that removing code and keeping the codebase modern is probably as important as adding new one. Quoting Saint-Exupery: "It seems that perfection is attained not when there is nothing more to add, but when there is nothing more to remove".
The article then covers security mechanisms, as well as the defaults which are turned specifically with an eye towards security.
All-in-all a good perspective piece about the reasons why OpenBSD is the right choice for Frederic, worth your time to read up on it if you want to learn more about OpenBSD’s differences.
***
BeastieBits
Call for 2016Q1 quarterly status reports
FreeBSD Mastery: Advanced ZFS” sponsorships ending soon
Shawn Webb from HardenedBSD talking about giving away RPi3’s at BSDCan and hacking on them to get FreeBSD working
xterm(1) now UTF-8 by default
Call For Artists: New Icon Theme
Happy 23rd Birthday, src!
Feedback/Questions
Alison - Readahead and Wayland
Kenny - Gear
Ben - IPFW2/3
Brad - ZFS Writeback
Simon - BSD Toonz
***
1 hr 41 min
March 24, 2016134: Marking up the Ports tree
This week on the show, Allan and I have gotten a bit more sleep since AsiaBSDCon, which is excellent since there is a LOT of news to cover. That plus our interview with Ports SecTeam member Mark Felder. So keep it
This episode was brought to you by
Headlines
FreeNAS 9.10 Released
OS:
The base OS version for FreeNAS 9.10 is now FreeBSD 10.3-RC3, bringing in a huge number of OS-related bug fixes, performance improvements and new features.
+Directory Services:
You can now connect to large AD domains with cache disabled.
+Reporting:
Add the ability to send collectd data to a remote graphite server.
+Hardware Support:
Added Support for Intel I219-V & I219-LM Gigabit Ethernet Chipset
Added Support for Intel Skylake architecture
Improved support for USB devices (like network adapters)
USB 3.0 devices now supported.
+Filesharing:
Samba (SMB filesharing) updated from version 4.1 to 4.3.4
Added GUI feature to allow nfsv3-like ownership when using nfsv4
Various bug fixes related to FreeBSD 10.
+Ports:
FreeBSD ports updated to follow the FreeBSD 2016Q1 branch.
+Jails:
FreeBSD Jails now default to a FreeBSD 10.3-RC2 based template.
Old jails, or systems on which jails have been installed, will still default to the previous FreeBSD 9.3 based template. Only those machinesusing jails for the first time (or deleting and recreating their jails dataset) will use the new template.
+bhyve:
++In the upcoming 10 release, the CLI will offer full support for managing virtual machines and containers. Until then, the iohyve command is bundled as a stop-gap solution to provide basic VM management support -
***
Ubuntu BSD's first Beta Release
Under the category of “Where did this come from?”, we have a first beta release of Ubuntu BSD.
Specifically it is Ubuntu, respun to use the FreeBSD kernel and ZFS natively.
From looking at the minimal information up on sourceforge, we gather that is has a nice text-based installer, which supports ZFS configuration and iSCSI volume creation setups.
Aside from that, it includes the XFCE desktop out of box, but claims to be suitable for both desktops and servers alike right now.
We will keep an eye on this, if anybody listening has already tested it out, maybe drop us a line on your thoughts of how this mash-up works out.
***
FreeBSD - a lesson in poor defaults
Former BSD producer, and now OpenBSD developer, TJ, writes a post detailing the defaults he changes in a fresh FreeBSD installation
Maybe some of these should be the defaults
While others are definitely a personal preference, or are not as security related as they seem
A few of these, while valid criticisms, but some are done for a reason
Specifically, the OpenSSH changes.
So, you’re a user, you install FreeBSD 10.0, and it comes with OpenSSH version X, which has some specific defaults
As guaranteed by the FreeBSD Project, you will have a nice smooth upgrade path to any version in the 10.x branch
Just because OpenSSH has released version Y, doesn’t mean that the upgrade can suddenly remove support for DSA keys, or re-adding support for AES-CBC (which is not really weak, and which can be hardware accelerated, unlikely most of the replacements)
“FreeBSD is the team trying to increase the risk.” Is incorrect, they are trying to reduce the impact on the end user
Specifically, a user upgrading from 10.x to 10.3, should not end up locked out of their SSH server, or otherwise confronted by unexpected errors or slowdowns because of upstream changes
I will note again, (and again), that the NONE cipher can NOT allow a user to “shoot themselves in the foot”, encryption is still used during the login phase, it is just disabled for the file transfer phase. The NONE cipher will refuse to work for an interactive session.
While the post states that the NONE cipher doesn’t improve performance that much, it infact does
In my own testing, chacha20-poly1305 1.3 gbps, aes128-gcm (fastest) 5.0 gbps, NONE cipher 6.3 gbps
That means that the NONE cipher is an hour faster to transfer 10 TB over the LAN.
The article suggests just removing sendmail with no replacement. Not sure how they expect users to deliver mail, or the daily/weekly reports
Ports can be compiled as a regular user. Only the install phase requires root
for ntpd, it is not clear that there is an acceptable replacement yet, but I will not that it is off by default
In the sysctl section, I am not sure I see how enabling tcp blackhole actually increases security at all
I am not sure that linking to every security advisory in openssl since 2001 is actually useful
Encrypted swap is an option in bsdinstall now, but I am not sure it is really that important
FreeBSD now uses the Fortuna PRNG, upgraded to replace the older Yarrow, not vanilla RC4.
“The resistance from the security team to phase out legacy options makes mewonder if they should be called a compatibility team instead.”
I do not think this is the choice of the security team, it is the ABI guarantee that the project makes. The stable/10 branch will always have the same ABI, and a program or driver compiled against it will work with any version on that branch
The security team doesn’t really have a choice in the matter. Switching the version of OpenSSL used in FreeBSD 9.x would likely break a large number of applications the user has installed
Something may need to be done differently, since it doesn’t look like any version of OpenSSL, (or OpenSSH), will be supported for 5 years ever again
***
ZFS Raidz Performance, Capacity and Integrity
An updated version of an article comparing the performance of various ZFS vdev configurations
The settings users in the test may not reflect your workload
If you are benchmarking ZFS, consider using multiple files across different datasets, and not making all of the writes synchronous
Also, it is advisable to run more than 3 runs of each test
Comparing the numbers from the 12 and 24 disk tests, it is surprising to see that the 12 mirror sets did not outperform the other configurations. In the 12 drive tests, the 6 mirror sets had about the same read performance as the other configurations, it is not clear why the performance with more disks is worse, or why it is no longer in line with the other configurations
More investigation of this would be required
There are obviously so other bottlenecks, as 5x SSDs in RAID-Z1 performed the same as 17x SSDs in RAID-Z1
Interesting results none the less
***
iXSystems
FreeNAS Mini Review
Interview - Mark Felder - [email protected] / @feldpos
Ports, Ports and more Ports
DigitalOcean
Digital Ocean's guide to setting up an OpenVPN server
News Roundup
AsiaBSDCon OpenBSD Papers
Undeadly.org has compiled a handy list of the various OpenBSD talks / papers that were offered a few weeks ago at AsiaBSDCon 2016.
Antoine Jacoutot (ajacoutot@) - OpenBSD rc.d(8) (slides | paper)
Henning Brauer (henning@) - Running an ISP on OpenBSD (slides)
Mike Belopuhov (mikeb@) - Implementation of Xen PVHVM drivers in OpenBSD (slides | paper)
Mike Belopuhov (mikeb@) - OpenBSD project status update (slides)
Mike Larkin (mlarkin@) - OpenBSD vmm Update (slides)
Reyk Floeter (reyk@) - OpenBSD vmd Update (slides)
Each talk provides slides, and some the papers as well. Also included is the update to ‘vmm’ discussed at bhyveCon, which will be of interest to virtualization enthusiasts.
***
Bitcoin Devs could learn a lot from BSD
An interesting article this week, comparing two projects that at first glance may not be entirely related, namely BitCoin and BSD.
The article first details some of the woes currently plaguing the BitCoin development community, such as toxic community feedback to changes and stakeholders with vested financial interests being unable to work towards a common development purpose.
This leads into the crux or the article, about what BitCoin devs could learn from BSD:
First and foremost, the way code is developed needs change to stop the current negative trend in Bitcoin. The FreeBSD project has a rigid internal hierarchy of people with write access to their codebase, which the various Bitcoin implementations also have, but BSD does this in a way that is very open to fresh eyes on their code, allowing parallel problem solving without the petty infighting we see in Bitcoin. Anyone can propose a commit publicly to the code, make it publicly available, and democratically decide which change ends up in the codebase. FreeBSD has a tiny number of core developers compared to the size of their codebase, but at any point, they have a huge community advancing their project without hard forks popping up at every small disagreement. Brian Armstrong commented recently on this flaw with Bitcoin development, particularly with the Core Devs:
“Being high IQ is not enough for a team to succeed. You need to make reasonable tradeoffs, collaborate, be welcoming, communicate, and be easy to work with. Any team that doesn’t have this will be unable to attract top talent and will struggle long term. In my opinion, perhaps the biggest risk in Bitcoin right now is, ironically, one of the things which has helped it the most in the past: the Bitcoin Core developers.”
A good summary of the culture that could be adopted is summed up as follows:
The other thing Bitcoin devs could learn from is the BSD community’s adoption of the Unix Design philosophy. Primarily “Worse is Better,” The rule of Diversity, and Do One Thing and Do It Well. “Worse is Better” emphasizes using extant functional solutions rather than making more complex ones, even if they would be more robust. The Rule of Diversity stresses flexibility of the program being developed, allowing for modification and different implementations without breaking. Do one Thing and Do it well is a mantra of the BSD and Unix Communities that stresses modularity and progress over “perfect” solutions. Each of these elements help to make BSD a wildly successful open source project with a healthy development community and lots of inter-cooperation between the different BSD systems. While this is the opposite of what we see with Bitcoin at present, the situation is salvageable provided changes like this are made, especially by Core Developers.
All in all, a well written and interesting take on the FreeBSD/BSD project. We hope the BitCoin devs can take something useful from it down the road.
***
FreeBSD cross-compiling with gcc and poudriere
Cross-Compiling, always a challenge, has gotten easier using poudriere and qemu in recent years.
However this blog post details some of the particular issues still being face when trying to compile some certain ports for ARM (I.E. rPi) that don’t play nicely with FreeBSD’s default CLANG compiler.
The writer (Ben Slack) takes us through some of the work-arounds he uses to build some troublesome ports, namely lsof and libatomic_ops.
Note this is not just an issue with cross compile, the above mentioned ports also don’t build with clang on the Pi directly.
After doing the initial poudriere/qemu cross-compile setup, he then shows us the minor tweaks to adjust which compiler builds specific ports, and how he triggers the builds using poudriere.
With the actual Makefile adjustment being so minor, one wonders if this shouldn’t just be committed upstream, with some if (ARM) - USE_GCC=yes type conditional.
***
Nvidia releases new Beta graphics driver for FreeBSD
Added support for the following GPUs: GeForce 920MX & GeForce 930MX
Added support for the Vulkan API version 1.0.
Fixed a bug that could cause incorrect frame rate reporting on Quadro Sync configurations with multiple GPUs.
Added a new RandR property, CscMatrix, which specifies a 3x4 color-space conversion matrix.
Improved handling of the X gamma ramp on GF119 and newer GPUs. On these GPUs, the RandR gamma ramp is always 1024 entries and now applies to the cursor and VDPAU or workstation overlays in addition to the X root window.
Fixes for bugs and added several other EGL extensions
***
Beastie Bits
New TN Bug started
DragonFlyBSD Network/TCP Performance's gets a bump
FreeBSD Foundation introduces a new website and logo
Our producer made these based on the new logo:
http://q5sys.sh/2016/03/a-new-freebsd-foundation-logo-means-its-time-for-some-new-wallpapers/
http://q5sys.sh/2016/03/pc-bsd-and-lumina-desktop-wallpapers/
https://github.com/pcbsd/lumina/commit/60314f46247b7ad6e877af503b3814b0be170da8
IPv6 errata for 5.7/5.8, pledge errata for 5.9
Sponsoring “PAM Mastery”
A visualization of FreeBSD commits on GitHub for 2015
The VAX platform is no more
Feedback/Questions
Hunter - Utils for Blind
Chris - ZFS Quotas
Anonymous - Tun, Tap and Me!
Andrew - Navigating the BSDs
Brent - Wifi on BSD
***
2 hr 6 min
March 16, 2016133: The Tokyo Debrief
This week on BSDNow, Allan and I are back from AsiaBSDCon and we have an interview with Brad Davis about the new “Packaging Base” call-for-testing. We’ll be sharing our thoughts and stories on how the week
This episode was brought to you by
Headlines
AsiaBSDCon 2016 - Wrap-up
FreeBSD gets Haswell graphics support in time for 11.0-RELEASE
The moment that many have been waiting for has finally arrived, support for Haswell graphics has been committed to FreeBSD -CURRENT
The brings the DRM/i915 code up to date with Linux kernel 3.8.13
Work has already started on updating to Linux kernel 3.9
It is hoped that subsequent updates will be much easier, and much faster
It does not appear to require setting the i915.preliminary_hw_support loader tunable
***
OpenBSD vmm/vmd Update
For the third year running, bhyvecon was held last week, during the lead up to AsiaBSDCon
Bhyvecon has expanded, and now covers all virtualization on BSDs
There were presentations on bhyve, Xen Dom0 on FreeBSD, Xen DomU for OpenBSD, and OpenBSD’s vmm
OpenBSD vmm started at the Brisbane 2015 hackathon in Australia
Work continued through the summer and fall thanks to funding by the OpenBSD Foundation
The presentation answered some outstanding questions, such as, why not just port bhyve?
Initial focus is OpenBSD on OpenBSD
Loader currently supports FreeBSD and NetBSD as well
After the initial commits, other developers joined in to help with the work
Reyk reworked the vmd and vmctl commands, to provide a better user interface
Future plans:
Nested VMX
i386 support
AMD SVM support
Filesystem passthru
Live migration (with ZFS like command syntax)
Other developers are working on related projects:
qemu interface: Allow qemu to be accelerated by the vmm backend, while providing emulated hardware, for legacy systems
KVM interface: Make vmm look like KVM, so existing tools like openstack “just work”
***
Interview - Brad Davis - [email protected] / @so14k
Packaging Base
News Roundup
Packaging the base system with pkg(8)
The official call for testing for FreeBSD’s pkg(8)’d base is out
Users are requested to checkout the release-pkg branch, and build it as normal (buildworld, buildkernel)
Instead of installworld, run: make packages
This will produce a pkg repo in the /usr/obj directory
The post to the mailing list includes an example pkg repo config file to point to those packages
Run: pkg update -r FreeBSD-base
This will read the metadata from the new repository
Then run: pkg install -g 'FreeBSD-*'
This will find all packages that start with ‘FreeBSD-’ and install them
In the future, there will be meta packages, so you can just install FreeBSD-base and it will pull in other packages are dependencies
Currently, there are a large number of packages (over 700), because each shared library is packaged separately, and almost all optional features are in a separate package
The number of packages is also increased because there are separate -debug, -profiling, etc versions of each package
New features are being added to pkg(8) to mark important system components, like libc, as ‘vital’, so they cannot be deleted accidently
However, in the case of using pkg(8)’d base to create a jail, the administrator should be able to delete the entire base system
Classic conundrum: “UNIX does not stop you doing something stupid, as that would also stop you doing something clever”
Work is still ongoing
At AsiaBSDCon, after the interview was recorded, bapt@ and brd@ had a whiteboarding session and have come up with how they expect to handle the kernel package, to ensure there is a /boot/kernel.old for you to fall back to incase the newly installer kernel does not work correctly.
***
FreeBSD 10.3-RC2 Now Available
The second release candidate for FreeBSD 10.3 is now available for testing
Notable changes include:
Import an upstream fix for ‘zfs send -i’ to avoid data corruption in specific instances
Boot loaders and kernel have been taught to handle ELF sections of type SHT_AMD64_UNWIND. This does not really apply to FreeBSD 10.3, but is required for 11.0, so will make upgrades easier
Various mkdb commands (/etc/services, /etc/login.conf, etc) commands now use fsync() instead of opening the files as O_SYNC, greatly increasing the speed of the database generation
From the earlier BETA3, the VFS improvements that were causing ZFS hangs, and the new ‘tryforward’ routing code, have been reverted
Work is ongoing to fix these issues for FreeBSD 11.0
There are two open issues:
A fix for OpenSSH CVE-2016-3115 has not be included yet
the re-addition of AES-CBC ciphers to the default server proposal list. AES-CBC was removed as part of the update to OpenSSH version 7.1p2, but the plan is to re-add it, specifically for lightweight clients who rely on hardware crypto offload to have acceptable SSH performance
Please go out and test
***
OPNsense 16.1.6 released
A new point-release of OPNsense has dropped, and apart from the usual security updates, some new features have been included
firmware: bootstrap utility can now directly install e.g. the development version
dhcp: all GUI pages have been reworked for a polished look and feel
proxy: added category-based remote file support if compressed file contains multiple files
proxy: added ICAP support (contributed by Fabian Franz)
proxy: hook up the transparent FTP proxy
proxy: add intercept on IPv6 for FTP and HTTP proxy options
logging: syslog facilities, like services, are now fully pluggable
vpn: stripped an invalid PPTP server configuration from the standard configuration
vpn: converted to pluggable syslog, menu and ACL
dyndns: all GUI pages have been reworked for a polished look and feel
dyndns: widget now shows IPv6 entries too
dns forwarder: all GUI pages have been reworked for a polished look and feel
dns resolver: all GUI pages have been reworked for a polished look and feel
dns resolver: rewrote the dhcp lease registration hooks
dns resolver: allow parallel operation on non-standard port when dns forwarder is running as well
firewall: hide outbound nat rule input for "interface address" option and toggle bitmask correctly
interfaces: fix problem when VLAN tags weren't generated properly
interfaces: improve interface capability reconfigure
ipsec: fix service restart behaviour from GUI
captive portal: add missing chain in certificate generation
configd: improve recovery and reload behaviour
load balancer: reordered menu entries for clarity
ntp: reordered menu entries for clarity
traffic shaper: fix mismatch for direction + dual interfaces setup
languages: updated German and French
Call for testing - ASLR patch
A patch that provides a first pass implementation of basic ASLR (Address Space Layout Randomization) for FreeBSD has been posted to the mailing list
“Stack gap, WX, shared page randomization, KASLR and other techniques are explicitly out of scope of this work.”
“ASLR is enabled on per-ABI basis, and currently it is only enabled on native i386 and amd64 (including compat 32bit) ABIs. I expect to test and enable ASLR for armv6 and arm64 as well, later”
“Thanks to Oliver Pinter and Shawn Webb of the HardenedBSD project for pursuing ASLR for FreeBSD. Although this work is not based on theirs, it was inspired by their efforts.”
***
Feedback/Questions
Daniel - OpenZFS
Florian - JBODS
Hunter - SSL on DO
Ben - Backups
Damian - Bug’in Me!
***
1 hr 17 min
March 08, 2016132: Scaling up with BSD
This week, Allan and I are away at AsiaBSDCon! (If you aren’t there, you are missing out). We will be back with a live episode next week. However, we’ve been asked for Allan to tell us about ScaleEngine’s
This episode was brought to you by
Interview - Allan Jude - [email protected] / @allanjude
Spotlight on ScaleEngine
***
Beastie Bits
NetBSD on an RPi Zero
DragonFly tips for printing with CUPS
Fighting fraudulent networks using secure connections (SSL) blacklisting with OPNsense. Blocks known-bad certificates as listed at abuse.ch

Fix for running NetBSD/amd64 7.0 on kvm based virtual machines
Michael W. Lucas’s new book, FreeBSD Mastery: Specialty Filesystems is now escaping
The Penguicon Lucas Tech Track
FreeBSD based nginx/ffmpeg camera recording and live streaming
CFT: New Jenkins Builder for FreeNAS / PC-BSD
Status Update: PC-BSD’s SysAdm Server
Status Update: PC-BSD’s SysAdm Client UI
1 hr 5 min
March 03, 2016 BSD Behind The Chalkboard | BSD Now 131
This week on the show, we have an interview with Jamie McParland of the Newburg public school district in Oregon. We are going to get an inside look at how they use BSD in various aspects operations across their network, as well as bringing you the latest news and questions. Keep it tuned to BSDNow, the place to B...SD!
1 hr 42 min
March 02, 2016131: BSD behind the chalkboard
This week on the show, we have an interview with Jamie
This episode was brought to you by
Headlines
BSDCan 2016 List of Talks
We are all looking forward to BSDCan
Make sure you arrive in time for the Goat BoF, the evening of Tuesday June 7th at the Royal Oak, just up the street from the university residence
There will also be a ZFS BoF during lunch of one of the conference days, be sure to grab your lunch and bring it to the BoF room
Also, don’t forget to get signed up for the various DevSummits taking place at BSDCan.
***
What does Load Average really mean
Chris Siebenmann, a sysadmin at the University of Toronto, does some comparison of what “Load Average” means on different unix systems, including Solaris/IllumOS, FreeBSD, NetBSD, OpenBSD, and Linux
It seems that no two OSes use the same definition, so comparing load averages is impossible
On FreeBSD, where I/O does not affect load average, you can divide the load average by the number of CPU cores to be able to compare across machines with different core counts
***
GPL violations related to combining ZFS and Linux
As we mentioned in last week’s episode, Ubuntu was preparing to release their next version with native ZFS support. + As expected, the Software Freedom Conservancy has issued a statement detailing the legal argument why they believe this is a violation of the GPL license for the Linux kernel.
It’s a pretty long and complete article, but we wanted to bring you the summary of the whole, and encourage you to read the rest, since it’s good to be knowledgeable about the various open-source projects and their license conditions.
“We are sympathetic to Canonical's frustration in this desire to easily support more features for their users. However, as set out below, we have concluded that their distribution of zfs.ko violates the GPL. We have written this statement to answer, from the point of view of many key Linux copyright holders, the community questions that we've seen on this matter. Specifically, we provide our detailed analysis of the incompatibility between CDDLv1 and GPLv2 — and its potential impact on the trajectory of free software development — below.
However, our conclusion is simple: Conservancy and the Linux copyright holders in the GPL Compliance Project for Linux Developers believe that distribution of ZFS binaries is a GPL violation and infringes Linux's copyright. We are also concerned that it may infringe Oracle's copyrights in ZFS. As such, we again ask Oracle to respect community norms against license proliferation and simply relicense its copyrights in ZFS under a GPLv2-compatible license.”
The Software Freedom Law Center’s take on the issue
Linux SCSI subsystem Maintainer, James Bottomley, asks “where is the harm”
FreeBSD and ZFS
***
DragonFly i915 reaches Linux 4.2
The port of the Intel i915 DRM/KMS Linux driver to DragonFlyBSD has been updated to match Linux kernel 4.2
Various improvements and better support for new hardware are included
One big difference, is that DragonFlyBSD will not require the binary firmware blob that Linux does
François Tigeot explains: "starting from Linux 4.2, a separate firmware blob is required to save and restore the state of display engines in some low-power modes. These low-power modes have been forcibly disabled in the DragonFly version of this driver in order to keep it blob-free."
Obviously this will have some disadvantage, but as those modes were never available on DragonFlyBSD before, users are not likely to miss them
***
Interview - Jamie McParland - [email protected] / @nsdjamie
FreeBSD behind the chalkboard
***
iXsystems
My New IXSystems Mail Server
News Roundup
Installing ELK on FreeBSD, Tutorial Part 1
Are you an ELK user, or interested in becoming one? If so, Gruppo Utenti has a nice blog post / tutorial on how to get started with it on FreeBSD.
Maybe you haven’t heard of ELK, but its not the ELK in ports, specifically in this case he is referring to “ElasticSearch/Logstash/Kibana” as a stack.
Getting started is relatively simply, first we install a few ports/packages:
textproc/elasticsearch
sysutils/logstash
textproc/kibana43
www/nginx
After enabling the various services for those (hint: sysrc may be easier), he then takes us through the configuration of ElasticSearch and LogStash. For the most part they are fairly straightforward, but you can always copy and paste his example config files as a template.
Follow up to Installing ELK on FreeBSD
Jumping directly into the next blog entry, he then takes us through the “K” part of ELK, specifically setting up Kibana, and exposing it via nginx publically.
At this point most of the CLI work is finished, and we have a great walkthrough of doing the Kibana configuration via their UI. We are still awaiting the final entry to the series, where the setup of ElastAlert will be detailed, and we will bring that to your attention when it lands.
***
From 1989: An Empirical Study of the Reliablity of Unix Utilities
A paper from 1989 on the results of fuzz testing various unix utilities across a range of available unix operating systems
Very interesting results, it is interesting to look back at before the start of the modern BSD projects
New problems are still being found in utilities using similar testing methodologies, like afl (American Fuzzy lop)
***
Google Summer of Code
Both
FreeBSD
and
NetBSD
Are running 2016 Google Summer of Code projects.
Students can start submitting proposals on March 14th.
In the meantime, if you have any ideas, please post them to the
Summer Of Code Ideas Page on the FreeBSD wiki
Students can start looking at the list now and try to find mentors to get a jump start on their project.
***
High Availablity Sync for ipfw3 in Dragonfly
Similar to pfsync, this new protocol allows firewall dynamic rules (state) to be synchronized between two firewalls that are working together in HA with CARP
Does not yet sync NAT state, it seems libalias will need some modernization first
Apparently it will be relatively easy to port to FreeBSD
This is one of the only features ipfw lacks when compared to pf
***
Beastie Bits
FreeBSD 10.3-BETA3 Now Available
LibreSSL isnt affected by the OpenSSL DROWN attack
NetBSD machines at the Open Source Conference 2016 in Toyko
OpenBSD removes Linux Emulation
Time is an illusion - George Neville-Neil
OpenSSH 7.2 Released
Feedback/Questions
Shane - IPSEC
Darrall - 14TB Zpool
Pedja - ZFS setup
***
1 hr 42 min
February 24, 2016130: Store all the Things | BSD Now 130
This week on BSDNow, Allan is back from the Storage Summit in Silicon Valley! We are going to get his thoughts on how the conference went, plus bring you the latest ZFS info discussed. That plus the usual BSD news is
This episode was brought to you by
Headlines
OpenBSD website operators urged to fix mind-alteringly bad bug
We start off a bit light-hearted this week, with the important, breaking news that finally a long-standing OpenBSD bug has been addressed for the HTTP daemon.
Specifically? It changes the default 404 page fonts away from Comic Sans, to a bit more crowd-pleasing alternative:
“For some reason the httpd status pages (e.g. 404) use the Comic Sans typeface. This patch removes comic sans and sets the typeface to the default sans-serif typeface of the client.
“This lowers the number of people contacting website maintainers with typeface complaints bordering on harassment”.
Operators running HTTPD are highly encouraged to update their systems to the latest code, right now……... No seriously, we are waiting for you. Get it done now and then we’ll continue with the show.
Registration for AsiaBSDCon 2016 is now open + Talk Schedule
After a few delays, the registration for AsiaBSDCon has now opened!
The conference starts in less than two weeks! now, so be sure to get signed up ASAP.
In addition the schedule has been posted, and here’s some of the highlights of this year’s conference.
In addition to FreeBSD and NetBSD dev summits on the first two days, we have some excellent tutorials being given this year by Kirk, Gnn, Dru and more! (https://2016.asiabsdcon.org/program.html.en)
The regular paper talks also have lots of good ones this year, including this crazy encrypted boot loader one given by our very own Allan Jude!
***
OPENBSD ON AWS : AN UNEXPECTED JOURNEY
We have a blog post from Antoine Jacoutot, talking about the process of getting OpenBSD up and running in AWS
It starts with his process of creating an AMI from scratch, which ended up not being that bad:
create and loopback-mount a raw image containing a UFS filesystem extract the OpenBSD base sets (which are just regular tarballs) and kernel enable console output (so that one could “aws ec2 get-console-output”)
install the boot loader on the image then use the ec2 tools to import the RAW image to S3, convert it into a volume (ec2-import-volume) which we can snapshot (ec2-create-snapshot) and create an AMI from (ec2-register)
The blog post also has a link to a script which automates this process, so don’t be daunted if you didn’t quite follow all of that.
Thanks to the recently landed DomU support, the final pieces of the puzzle fell into place, allowing OpenBSD to function as a proper guest (with networking!)
Next it details the process of injecting a public SSH key into the instances for instant remote access.
An ec2-init.sh script was created (also on github) which does the following:
setting the hostname
installing the provided SSH public key to /root/.ssh/authorized_keys
executing user-data (if it starts with a shebang)
displaying the host SSH fingerprints on the console (to match cloud-init)
With that done, OpenBSD is pretty much AWS ready! He then gives a brief walkthrough of setting up nginx for new users, but if you’ve already done this before then the instance is ready for you to hacking on.
Start thinking of ideas for things with FreeBSD for Google's 2016 Summer of Code
Students and Developers, listen up! It’s time to start thinking about GSoC again, and FreeBSD is looking to update its project ideas page.
There’s some good ones on the list, plus ones that should be pruned (such as GELI boot), but now is the time to start adding new ones before we get too deep into the process.
This goes for the other BSD’s as well, start thinking about your proposals, or if you are developer, which projects would be a good fit for mentoring.
(Improving the Linux Compat layer is one I think should be done!)
Guide to getting started with kernel hacking
One of the things that’s been asked frequently is how to contribute towards the efforts to bring updated DRM / X drivers to the FreeBSD kernel.
Jean-Sébastien Pédron has started a great guide on the Wiki which details how to get started with the porting effort, and that developers need not be afraid of helping.
***
Storage Summit Roundup
Earlier this week a number of developers from FreeBSD, as well as various vendors that use FreeBSD, or provide products used with FreeBSD met for a Storage Summit, to discuss the
future of these technologies
The summit was co-located with the USENIX FAST (Filesystems And Storage Technologies) conference
The summit was sponsored by the FreeBSD Foundation and FlightAware
After a short introduction, the event opened with a Networking Synergy panel
The focus of this panel was to see if there were techniques and lessons learned in improving the networking stack over the last 10 years that could be applied to improving the storage stack
A lot of time was spent discussing issues like multi-queue support, CPU scheduling, and ways to modernize the stack
CAM Scheduling & Locking Revamp
No notes posted
User Space Storage Stack
One of the user space storage stacks discussed was Diskmap
Like netmap, but for disks (diskmap)
Kernel bypass for accessing disks
Ilias Marinos, who is working on diskmap at Cambridge University, described diskmap to the group
A design discussion then followed in which the memory management was covered as that's an issue for any sort of "IO" map system
Action Items:
Discuss with Luigi the idea of code merges
Need a reset path API
Kernel buffer mapping for reliability
Support for other interfaces (SATA/SCSI)
GEOM layer adaptation
Adapting to New Storage Technologies
This working group was led by Adrian Palmer, from Seagate
SMR
Persistent Memory
Session 1: Device Identification and the structural requirements
Agenda: We'll look over the Identification nuances and what needs to change to support the structure. Support for IO order guarantees, forward-write only requirements, new commands and topology. Dig into CAM and GEOM layers. Solutions should be fast and have as few code paths as possible
Results: Small audience. We talked about zoned characteristics, and how it can be used in various workloads, projected to be implemented in years
Session 2: Information dissemination and consumption
Agenda: Where and how will information from the report_zones command be gathered, stored, combined and used. This will include userspace storage and multi-volume management. Will CAM store this data, or will GEOM? How frequently will this need to be queried/updated/verified from the drive?
Results: Merged with ZFS working group to discuss SMR. Came up with idea that could be implemented as circular buffer zone type. Began to discuss solutions among developers
ZFS
During the first session we discussed how to improve dedup support
+ A dedup throttle or cap was discussed. When the size of the DDT grows beyond this size, new entries would not be deduped.
An alternative to this was also discussed, where when the DDT reached the cap size, it would remove a random entry with only a single reference from the DDT to make room for the new entry. When a block is going to be freed, if it is not found in the DDT, it is assumed to have only 1 reference, and removed.
There was also discussion of replacing the DDT with an in-memory hash table and a “log” of increment/decrement operations, that is periodically compacted. The hash table is recreated from the log at pool import time. This would reduce the in-memory footprint of the DDT, as well as speed up all write operations as adding an entry to the dedup log will be less expensive than updating the DDT.
There was also discussion of using dedicated device(s) for the DDT, either using the DDT on SSD work by Nexenta, or the Metadata Classes work by Intel
The first session also discussed Secure Delete and related things
The desire for an implementation of TRIM that uses the “secure erase” functionality provided by some disks was expressed
Overwriting sectors with patterns of garbage may be insufficient because SSDs may internally remap where a specific LBA physically resides
The possibility of using something like the “eager zero” feature to periodically write zeros over all free blocks in the pool to erase any lingering data fragments
Problems with the FreeBSD TRIM implementation were discussed, as well as looking at ways to implement the new ZFS TRIM implementation on FreeBSD
ABD (ARC Buf Data) was discussed, a new design that lessens the requirement for contiguous memory. Only a small area of contiguous blocks is reserved at boot, and compressed ARC blocks are constructed of scatter-gather lists of individual pages
The second session combined with the SMR group and talked about SMR support in ZFS
Later in the second session ZFS Encryption was also discussed, mostly with a focus on what the use cases are
The third session combined all of the groups for an overview of upcoming ZFS features including device removal and channel programs
There was also a request for code review, for mostly finished projects like Persistent L2ARC, Writeback cache, and Large dnode support
Hallway Track
ZFS / VFS Interaction
Adrian Palmer has been a FreeBSD hobbyist since FreeBSD 7, and I think I managed to convince him to start contributing
***
News Roundup
One Week with NetBSD 7.0: Back to Unix basics
The author of this blog series is sending a week using NetBSD 7.0, following a previous series on Solaris 10
“This is actually familiar territory, as I've been using BSD variants almost exclusively since 2006. My recent SunOS explorations were triggered last summer by OpenBSD having choked on my current laptop's NVIDIA card, and from what I could see at the time, FreeBSD had the same problem, although I now know NVIDIA drivers exist for that system. The thing that keeps me from going all-in with FreeBSD 10.x, however, is the fact that Firefox crashes and leaves "core dump" messages in its wake, and I'm just not a Chrome kinda guy.”
“For those with a catholic taste in Unix, NetBSD is a keg party at the Vatican. If you're an absolute Unix beginner, or have been living on Ubuntu-based Linux distros for too long, then you may feel stranded at first by NetBSD's sparseness. You'll find yourself staring into the abyss and seeing only a blinking cursor staring back. If you have the presence of mind to type startx, you'll be greeted by twm, a window manager offering little more than an xterm window with the same blinking cursor until you learn how to configure the .twmrc file to include whatever applications you want or need in the right-click menu.”
“As for NetBSD itself, I can't think of any major productivity applications that can't be installed, and most multimedia stuff works fine.”
Issues the author hopes to sort out in later posts:
Audio playback (youtube videos in Firefox)
Wireless
Flash
Digital Camera SD Card readability, video playback
Audacity
A “fancy” desktop like Gnome 2, KDE, or xfce
In a follow-up post, the author got LibreOffice installed and sorted out the audio issues they were having
In a later follow-up XFCE is up and running as well
***
ZFS is for Containers in Ubuntu 16.04
As you may have heard, Ubuntu 16.04 will include ZFS -- baked directly into Ubuntu -- supported by Canonical
“ZFS one of the most beloved features of Solaris, universally coveted by every Linux sysadmin with a Solaris background. To our delight, we're happy to make to OpenZFS available on every Ubuntu system.”
What does “supported by Canonical” mean?
“You'll find zfs.ko automatically built and installed on your Ubuntu systems. No more DKMS-built modules”
“The user space zfsutils-linux package will be included in Ubuntu Main, with security updates provided by Canonical”
The article then provides a quick tutorial for setting up Linux Containers (LXC) backed by ZFS
In the example, ZFS is backed by a file on the existing disk, not by a real disk, and with no redundancy
However, the setup script seems to support using real block devices
The Software Freedom Conservancy is expected to issue a statement detailing their opinion on the legalities and licensing issues of bundling ZFS with Linux.
***
Polling is a Hack: Server Sent Events (EventSource) with gevent, Flask, nginx, and FreeBSD
A tutorial on setting up ‘Server-Sent Events’, also know as EventSource in javascript, to notify website clients of new data, rather than having the javascript constantly poll for new data.
The setup uses FreeBSD, nginx, gevent, Python, and the Flask framework
The tutorial walks through setting a basic Python application using the Flask framework
Then setting up the client side in Javascript
Then for the server side setup, it covers installing and configuring nginx, and py-supervisor on FreeBSD
The tutorial also includes links to additional resources and examples, including how to rate limit the Flash application
***
Why FreeBSD?
An excellent article written by Hamza Sheikh, discussing why FreeBSD is now his clear choice for learning UNIX.
The article is pretty well written and lengthy, but has some great parts which we wanted to share with you:
There were many rough edges in the Linux world and some of them exist even today. Choosing the right distribution (distro) for the task at hand is always the first and most difficult decision to make. While this is a strength of the Linux community it is also its weakness. This is exacerbated with the toxic infighting within the community in the last few years.
A herd of voices believes it is their right to bring down a distro community because it is not like their distro of choice. Forking upstream projects has somehow become taboo. Hurling abuse in mailing lists is acceptable. Helping new users is limited to lambasting their distro of choice. Creating conspiracy theories over software decisions is the way to go. Copyleft zealots roam social media declaring non-copyleft free software heretic abominations. It all boils down to an ecosystem soured by the presence of maniacs who have the loudest voices and they seem to be everywhere you turn.
Where is the engineering among all this noise? Btrfs - baking for a long time - is still nowhere near ZFS in stability or feature parity. systemd is an insatiable entity that feeds on every idea in sight and just devours indiscriminately. Wayland was promised years ago and its time has yet to arrive. Containers are represented by Docker that neither securely contains applications nor makes them easy to manage in production. Firewalling is dithering between firewalld, nftables, etc. SystemTap cannot match DTrace.
In the same time span what do various BSDs offer? pf, CARP, ZFS, Hammer, OpenSSH, jails, pkgsrc, (software) ports, DTrace, hardware portability; just to name a few. Few would deny that BSDs have delivered great engineering with free software licenses to the entire world. To me they appear to be better flag bearers of free software with engineering to back it.
He then goes through some of the various BSD’s and the specifics on why FreeBSD was the logical choice for his situation. But at the end has a great summary on the community as a whole:
Finally - and maybe repeating myself here - I have nothing but praise for the community. Be it BSD Now, mailing lists, Reddit, Twitter, LFNW, or SeaGL, people have encouraged me, answered my questions, and filed bugs for me. I have been welcomed and made a part of the community with open arms. These reasons are (good) enough for me to use FreeBSD and contribute to it.
BeastieBits
OPNsense 16.1.3 released
Copies of "FreeBSD Mastery: Specialty Filesystems" seen in the wild
pfsense training available in Europe
LiteBSD now has 50 ports in its ports tree
Ports tree locked for OpenBSD 5.9
“FreeBSD Filesystem Fun” at March semibug
Event #46 — Embedded Platforms (BSD, OpenWRT, Plan 9 & Inferno)
Feedback/Questions
Frank - ZFS RAM?
David - ARM Porting
Johnny - Lumina Default?
Adam - PC-BSD Install and Q’s
Jeremy - Video Card Q
***
2 hr 11 min

About BSD Now

From the publisher's feed

Created by three guys who love BSD, we cover the latest news and have an extensive series of tutorials, as well as interviews with various people from all areas of the BSD community. It also serves as…