BSD Now

By JT PenningtonNews Education How To Tech News

BSD Now episodes

June 29, 2016148: The place to B...A Robot!
This week on the show, Allan and I are going to be showing you a very interesting interview we did talking about using FreeBSD to drive
This episode was brought to you by
alt="DigitalOcean - Simple Cloud Hosting, Built for Developers" />
src="/images/3.png" alt="Tarsnap - Online Backups for the Truly Paranoid" />
Headlines
FreeBSD Core Team Election
Core.9 has been elected, and will officially take over from Core.8 on Wednesday, 6 July 2016
Many thanks to the outgoing members of the core team for their service over the last 2 years
214 out of 325 eligible voters (65.8%) cast their votes in an election counting 14 candidates.
The top nine candidates are, in descending order of votes received:
180 84.1% Ed Maste (incumbent)
176 82.2% George V. Neville-Neil (incumbent)
171 79.9% Baptiste Daroussin (incumbent)
168 78.5% John Baldwin
166 77.6% Hiroki Sato (incumbent)
147 68.7% Allan Jude
132 61.7% Kris Moore
121 56.5% Benedict Reuschling
108 50.5% Benno Rice
There was no tie for ninth.
BSDNow and the entire community would also like to extend their thanks to all those who stood for election to the core team
Next week’s core meeting will encompass the members of Core.8 and Core.9, as responsibility for any outstanding items will be passed from outgoing members of core to the new incoming members
***
Why I run OpenBSD
This week we have a good article / blog post talking about why the posted has moved to OpenBSD from Linux.
“One thing I learned during my travels between OSs: consistency is everything.
Most operating systems seem to, at least, keep a consistent interface between themselves and binaries / applications. They do this by keeping consistent APIs (Application Programming Interfaces) and ABIs (Application Binary Interfaces). If you take a binary from a really old version of Linux and run or build it on a brand-spanking new install of Linux, it will likely Just Work™. This is great for applications and developers of applications. Vendors can build binaries for distribution and worry less about their product working when it gets out in the wild (sure this binary built in 2016 will run on RedHat AS2.1!!).“
The author then goes through another important part of the consistency argument, with what he calls “UPI” or “User Program Interfaces”. In other words, while the ABI may be stable, what about the end-user tooling that the user directly has to interact with on a daily basis?
“This inconsistency seems to have come to be when Linux started getting wireless support. For some reason someone (vendors, maybe?) decided that ifconfig wasn’t a good place to let users interact with their wireless device. Maybe they felt their device was special? Maybe there were technical reasons? The bottom line is, someone decided to create a new utility to manage a wireless device… and then another one came along… pretty soon there was iwconfig(8), iw(8), ifconfig(8), some funky thing that let windows drivers interface with Linux.. and one called ip(8) I am sure there are others I am forgetting, but I prefer to forget. I have moved onto greener pastures and the knowledge of these programs no longer serves me.”
The article then goes through the rundown of how he evaluated the various BSD’s and ultimately settled on OpenBSD:
“OpenBSD won the showdown. It was the most complete, simple, and coherent system. The documentation was thorough, the code was easy to follow
and understand.
It had one command to configure all of the network interfaces!
I didn’t have wireless, but I was able to find a cheap USB adapter that worked by simply running man -k wireless and reading about the USB
entries.
It didn’t have some of the applications I use regularly, so I started reading about ports (intuitively, via man ports!).”
The ultimate NetBSD Router
“So yesterday I spent the day setting up a new firewall at home here, based off of this BSD Now tutorial. Having set up a couple of OpenBSD routers before, either based on old laptops, bulky old power-sucking desktops or completely over-specced machines like the Intel NUC, I wanted to get some kind of BSD onto a low-powered ARM board and use that instead.”
“I've had a couple of Cubietrucks lying around for a while now, I've used them in a couple of art installations, running Debian and Pure Data, but over all they've been a bit disappointing. It's more the manufacturer's fault but they require blobs for the graphics and audio, which Debian won't allow, so as a multimedia board they're dud for video, and only passable for audio work with a usb sound card. So they've been collecting dust.”
“Only thing missing is a second NIC, luckily I had an Apple USB->Ethernet dongle lying around, which when I bought it was the cheapest thing I could find on eBay that OpenBSD definitely supported. There, and on NetBSD, it's supported by the axe(4) driver. USB 2.0 works fine for me as I live in Australia and my ISP can only give me 30Mbps, so this should do for the forseeable future.”
The article then walks through installing and configuring NetBSD
Configuration includes: pf, unbound, and dhcpd
“This project has been really fun, I started with basically no experience with NetBSD and have finished with a really useful, low-powered and robust appliance. It's a testament to the simplicity of the NetBSD system, and the BSD design principles in general, that such a novice as myself could figure this out. The NetBSD project has easily the most polished experience on Allwinner ARM boards, even Debian doesn't make it this easy. It's been a joy running the system, it has the bits I love from OpenBSD; ksh(1), tmux(1), an http daemon in base and of course, pf(4). This is mixed with some of the pragmatism I see in FreeBSD; a willingness to accept blobs if that really is the only way to boot, or get audio, or a video console.”
***
bhyve-Bootable Boot Environments
We have a lengthy article also today from our friend Michael Dexter, who asks the basic question “What if multibooting and OS upgrades weren't horrible?”
No doubt if you’ve been a frequent listener to this show, you’ve heard Allan or Myself talking about ZFS Boot Environments, and how they can “change your life”.
Well today Michael goes further into detail on how the BE’s work, and how they can be leveraged to do neat things, like installing other versions of an operating system from the original running system.
“If you are reading this, you have probably used a personal computer with a BSD or GNU/Linux operating system and at some point attempted to multiboot between multiple operating systems on the same computer. This goal is typically attempted with complex disk partitioning and a BSD or GNU/Linux boot loader like LILO or GRUB, plus several hours of frustrating experimentation and perhaps data loss. While exotic OS experimentation has driven my virtualization work since the late 1990s, there are very pragmatic reasons for multibooting the same OS on the same hardware, notable for updates and failback to "known good" versions. To its credit, FreeBSD has long had various strategies including the NanoBSD embedded system framework with primary and secondary root partitions, plus the nextboot(8) utility for selecting the "next" kernel with various boot parameters. Get everything set correctly and you can multiboot "with impunity".
“That's a good start, and over time we have seen ZFS "boot environments" be used by PC-BSD and FreeNAS to allow for system updates that allow one to fall back to previous versions should something go wrong. Hats off to these efforts but they exist in essentially purpose-built appliance environments. I have long sensed that there is more fun to be had here and a wonderful thing happened with FreeBSD 10.3 and 11.0: Allan Jude added a boot environment menu to the FreeBSD loader”
From here Michael takes us through the mechanical bits of actually creating a new ZFS dataset (BE) and performing a fresh FreeBSD 10.3 installation into this new boot-environment.
The twist comes at the end, where he next sets up the BE to be a root NFS for booting in bhyve! This is interesting and gives you a way to test booting into your new environment via a VM, before rebooting the host directly into it.
***
Interview - Edicarla Andrade & Vinícius Zavam - @egypcio
BSD-Powered Robots
News Roundup
Tomohiro Kasumi explains what “@@” means, in the context of the Hammer filesystem
A post from the Dragonfly users’ mailing list about what the @@ construct means in the Hammer filesystem
“@@ represents the existence of a PFS which is logically separated pseudo filesystem space within HAMMER's B-Tree”
“HAMMER only has 1 large B-Tree per filesystem (not per PFS), so all the PFS exist within that single B-Tree. PFS are separated by localization parameter which is one of the B-Tree keys used to lookup the tree.”
Each substring in "@@-1:00001" means:
"@@" means it's a PFS or snapshot.
"-1" means it's a master.
":" is just a separator.
"00001" means it's PFS#1, where PFS#0 is the default PFS created on newfs. There is no "00000" because that's what's mounted on /HAMMER. PFS# is used for localization parameter.
“Localization parameter has the highest priority when inserting or looking up B-Tree elements, so fs elements that belong to the same PFS# tend to be localized (clustered) within the B-Tree”
There is also a note about how snapshots are named: "@@0x00..."
A user points out that having : in the path can confuse some applications, such as in the case of adding the current directory or a relative path to the $PATH environment variable, which is a colon delimited list of paths
This seems quite a bit more confusing that the datasets created by ZFS, but they might have other useful properties
***
FreeBSD 11.0 nearing RC1
We’ve all been eagerly awaiting the pending release of FreeBSD 11.0, and the schedule has now been updated!
The first release candidate is slated for July 29th!
If all goes well (and we stick to schedule) there will be another RC2 and possible RC3 release, before 11.0 officially drops near the end of August.
Start playing with those builds folks, be sure to send your feedback to the team to make this the best .0 release ever!
***
TensorFlow on FreeBSD
Next we have a blog post about the experience of a “new” FreeBSD user trying to deploy some non-ported software to his new system.
Specifically he was interested in running TensorFlow, but not doing a port himself, because in his words:
“First, I apologize for not supplying a port archive myself. After reading the FreeBSD handbook for creating a port, it's too complex of a task for me right now. I've only been using FreeBSD for two weeks. I would also not like to waste anyone's time giving them a terrible port archive and mess up their system.”
First of all, good ports are often born out of bad ports! Don’t let the porting framework daunt you, give it a go, since that's the only way you are going to learn how to write “good” ports over time. The porters-handbook is a good first place to start, plus the community usually is very helpful in providing feedback.
He then walks us through the changes made to the TensorFlow code (starting with the assumption that OSX was a good “flavor” to begin porting from) and ultimately compiling.
This ends up with the creation of a pip package which works!
A good tutorial, and also very similar to what goes on in the porting process. With this write-up perhaps somebody will take up creating a port of it… hint hint!
***
NetBSD: A New Beginning?
We don’t get enough NetBSD news at times, but this post by James Deagle talks about his adventure with NetBSD 7.0 and making it his “new beginning”
“After a few months of traipsing around the worlds of SunOS and Linux, I'm back to NetBSD for what I hope will be a lengthy return engagement. And while I'm enamored of NetBSD for all the previously-mentioned reasons, I'm already thinking ahead to some problems to solve, some of which have also been mentioned before.”
He then goes through and lists some of the small nits he’s still running into during the daily workflow
YouTube audio - Specifically he mentions that no audio is playing, but wonders if Flash plays some part. (Ideally you’re not using Flash though, in which case you need to check the audio backend FF is using. Try PulseAudio since it seems the best supported. If pulse is already enabled, install ‘pavucontrol’ to make sure audio is playing to the correct sound device)
Slow gaming performance (TuxKart and Celestia) - Check DRI / Xorg? Or is it CPU bound?
Lastly some unspecified Wireless issues, which typically end up being driver related. (Or use another chipset)
Beastie Bits
Reproducible NetBSD? 77.7% of the way there
Create FreeBSD virtual machine using qemu. Run the VM using xhyve.
FreeBSD PowerPC 32bit pkg repository (unofficial). ~19,500 packages, more to come
NetBSD machines at Open Source Conference 2016 Gunma
Adam Leventhal (of ZFS and DTrace) does an analysis of APFS
SemiBug June meeting summary
KnoxBug Meeting
Feedback/Questions
Andrew - iocage
Florian - Arm + GitHub
Clint - Synth
Leonardo - Translations
Zachary - Moving things to VMs
***
1 hr 45 min
June 22, 2016147: Release all the things!
On this episode of BSDNow, we will be talking to Glen Barber and Peter Wemm of the FreeBSD RE and Cluster Admin teams! That plus our
This episode was brought to you by
alt="DigitalOcean - Simple Cloud Hosting, Built for Developers" />
src="/images/3.png" alt="Tarsnap - Online Backups for the Truly Paranoid" />
Headlines
2016 FreeBSD Community Survey
We often get comments from our listeners, “I’m not a developer, how can I help out”?
Well today is your chance to do something. The FreeBSD Foundation has its 2016 Community Survey online, where they are asking for feedback from
you!
I just did the survey, it’ll take you about 5 minutes, but gives you a chance to provide valuable feedback to the foundation about things that
are important to you.
Be sure to answer in as much detail as possible and the foundation will review and use this feedback for its operations going forward.
***
ART, OpenBSDs new routing table, single thread performances
OpenBSD has changed the way routes are looked up in the kernel as part of their path to an SMP networking stack
The “Allotment Routing Table” (ART) is a performance tradeoff, where more memory is used to store the routing table, in exchange for faster
lookups
With this new arrangement, a full BGP routing table will grow from 130MB to 180MB of memory
“ART is a free multibit trie based routing table. To keep it simple, it can be seen as using more memory for fewer CPU cycles. In other words,
we get a faster lookup by wasting memory. The original paper presents some performance comparisons between
two ART configurations and the BSD Radix. But how does this apply to OpenBSD?”
“I asked Hrvoje Popovski to run his packet forwarding test on his Xeon box (E5-2620 v2 @ 2.10GHz, 2400.34 MHz) with ix(4) (82599) interfaces.
The test setup consist of three machines with the OpenBSD box in the middle”
“The simulations have been performed with an OpenBSD -current from June 9th. The machine is configured with pf(4) disabled in order to force a
single route lookup for every IPv4 packet. Based on the result of the lookup the kernel decide if it should forward, deliver or drop the packet”
***
BSDCan 2016 Playlist
The complete set of videos from BSDCan is online and ready to be consumed
Remember the good-ole days where we would wait months (or years) to get videos posted from conferences?
Well, who are we kidding, some conferences STILL do that, but we can’t count BSDCan among them.
Only two weeks out from this years exciting BSDCan, and all the videos have now landed on YouTube.
Granted, this is no substitute for actually being at the conference, but even if you attended you probably missed quite a few of the talks.
There are no videos of the hallway track, which is the best part of the conference
Except the dinner discussion of course.
and don’t forget the hacker lounge
***
Should you be scared of Unix signals?
Do you know much about UNIX Signals?
Are you afraid of their complexity?
Do you know there are signals other than SIGKILL?
This article talks about the practical implications of signals from a programming perspective
The things you need to consider when dealing with signals
Basically, you register a “signal handler”, the function that will be run when a signal arrives
As you program is running, if a signal arrives, your program will be interrupted. Its current state will be saved and any system calls in progress
will return EINTR (Error, Interrupted), then your signal handler will be run.
Once the signal handler is complete, the state of your application will be restored, and execution will resume
As long as your program properly handles this interruption, and errors that might result from it (getting EINTR from a read() call, instead of the
data you expected), then everything should be fine.
Of course, you need to be careful what you do inside your signal handler, as if you modify any variables or state in your application, it might be
very confused when it resumes.
***
Interview - Glen and Peter-
News Roundup
Unik - The Unikernel Compilation and Deployment Platform (uses NetBSD's Rump)
We’ve talked a bit about NetBSD’s RUMP (unikernel) in the past, including articles on how to deploy services using it.
Now we have an interesting project which makes the process super-easy, and dare-we-say almost “Docker-Like?”
The Unik project has a fairly complete walkthrough right on their GitHub project page, including details on installation and creating your own
unikernel containers.
In addition, it provides instructions on boot-strapping your own Go/Node.js/Python/Java applications, and supports out of Box VCenter / AWS / Qemu
/ VirtualBox providers.
***
PkgSrc 50th Release Highlights
pkgsrc is celebrating its 50th release, and to highlight this, they have posted a series of interviews from people who have been active in the
project
pkgsrc 50th release interviews - Jonathan Perkin
pkgsrc 50th release interviews - Ryo ONODERA
pkgsrc 50th release interviews - Joerg Sonnenberg
pkgsrc 50th release interviews - Sevan Janiyan
***
Migrating to FreeBSD from Solaris 11
Part 2
Part 3
Part 4
Part 5
***
How to chroot www/firefox on NetBSD
Looking for a jail-like method of running FireFox on NetBSD? (Or possibly other BSDs?)
We have a github repo with details on how to setup and run FireFox using a chroot using a “webuser” account for safety.
Think of this as a jail alternative, may be useful on systems with no jail support.
Of interest is the method used to do X forwarding. It uses Xorg TCP listen option (which is often off by default for security reasons). Perhaps SSH
X forwarding would be a better alternative. (Or nullfs mounts of /tmp)
***
Beastie Bits
Tredly - V1 Release Candidate
Call for Testing - ypldap testing against OpenLDAP and Microsoft Active
Directory
BSD Magazine, June 2016 Out Now
Hammer2 - Add xxhash to H2 and throw in debug stuff for performance
testing
chyves pre-announcement
***
Feedback/Questions
Michael - Versioning
Michael - Removing Encryption
Bostjan - PC-BSD Questions
Fong - ZFS Rollback
Jochen - Docker on FBSD
***
1 hr 41 min
June 16, 2016146: Music to Beastie’s ears
Kris is on vacation this week, so allan flies solo, provides a recap of BSDCan & cover's a boatload of news including Microsoft
This episode was brought to you by
Headlines
BSDCan Recap and Live Stream Videos
OpenBSD BSDCan 2016 papers now available
Allan’s slides and Paper
Michael W Lucas presents Allan with a gift
“FreeBSD Mastery: Advanced ZedFS”
Highlighted Tweets:
Groff Arrives at BSDCan
FreeBSD Foundation recognizes the contributions of Bryan Drewery, Rod Grimes, Warren Block, & Gleb Smirnoff
A moment of silence and shots in memory in Benjamin Perrault @creepingfur
@gvnn3 sells the FreeBSD Foundation shirt off of his back for Charity
Michael W. Lucas asks Matt Ahrens how to pronounce ZFS, “You can pronounce ZFS however you like, but if you pronounce it 'reiserfs', people might be confused.”
Sysadmin T-Shirt
FreeBSD Dev Summit ran out of room on the chalkboards listing accomplishments of 11.0
List of things people have or want for FreeBSD 12
Matt Ahrens signing Allan’s ZFS book
FreeBSD’s new marketing strategy
Charity Auction: systemd whoopie cushion
Embarass OpenBSD’s @HenningBrauer by donating $10 to charity for a selfie with him wearing a Linux t-shirt
@GroffTheBSDGoat changes handlers, from @HenningBrauer to @GavinAtkinson
Day 1 Video
Day 2 Video
Allan’s GELIBoot talk (day 2)
***
Media Coverage of Microsoft + FreeBSD story
Microsoft has released their own custom image of FreeBSD 10.3 for the Azure Cloud
“This means that not only can you quickly bring-up a FreeBSD VM in Azure, but also that in the event you need technical support, Microsoft support engineers can assist.”
“Microsoft is the publisher of the FreeBSD image in the marketplace rather than the FreeBSD Foundation. The FreeBSD Foundation is supported by donations from the FreeBSD community, including companies that build their solutions on FreeBSD. They are not a solution provider or an ISV with a support organization but rather rely on a very active community that support one another. In order to ensure our customers have an enterprise SLA for their FreeBSD VMs running in Azure, we took on the work of building, testing, releasing and maintaining the image in order to remove that burden from the Foundation. We will continue to partner closely with the Foundation as we make further investments in FreeBSD on Hyper-V and in Azure.”
"It's quite a significant milestone for FreeBSD community and for Microsoft to publish a supported FreeBSD image on Azure Marketplace. We really appreciate Microsoft's commitment and investment in FreeBSD project". - Justin T. Gibbs, President of FreeBSD Foundation
Microsoft took a FreeBSD 10.3-RELEASE image and added additional patches, most of which they have upstreamed but that were too late for the regular 10.3 release cycle.
Rather than requiring users to use a snapshot of the stable/10 branch, which would complicate the user experience, and complicate the job of the Microsoft support engineers, they created their own “certified” release
This allows Microsoft to selectively deploy errata fixes to the image as well
It is not clear how this affects update mechanisms like freebsd-update(8)
The Register
The Inquirer
Infoworld
The Hacker News
Windows Report
Windows Club
***
Select works poorly
“At the bottom of the OpenBSD man page for select is a little note. “Internally to the kernel, select() and pselect() work poorly if multiple processes wait on the same file descriptor.” There’s a similar warning in the poll man page. Where does this warning come from and what does it mean?”
Ted found that at first glance, OpenBSD’s select() appears to be quite bad:
“whenever some data gets written, we call wakeup(&selwait);. Based on what we’ve seen so far, one can conclude that this is likely to be inefficient. Every time any socket has some data available, we wake up every selecting process in the system. Works poorly indeed.”
After further investigation, it turns out to not be quite as bad
When the select() is first setup, the PID of the process that cares about the FD is recorded in the selinfo struct
If a second process runs select() on the same FD, the SI_COLL (Select Collision) flag is set on the selinfo struct
When selwakeup() is called, if SI_COLL is set, all select()ing processes are woken up, and the sysctl kern.nselcoll is incremented. If the flag is not set, and only a single PID is waiting for activity on that FD, only that process is woken up
“This is not an intractable problem. kevent avoids it entirely. Other implementations may too. But practically, does it need to be solved? My laptop says it’s happened 43 times. A server with substantially more uptime says 0. Doesn’t seem so bad.”
***
Interview - Hans Petter Selasky - [email protected] / @twitter
Designing FreeBSD’s USB drivers, hooking up a piano to FreeBSD & more!
***
News Roundup
Timeline of libexpat random vulnerability
Do you use FreeBSD as web server? Why or why not?
20 years of NetBSD code Bloat
HP Chromebook 13 now booting OpenBSD
UNIX for Poets
Comparing live version upgrade methods
My life with FreeBSD on a Thinkpad X220
1 hr 5 min
June 08, 2016145: At the Core of it all
It’s BSDCan time! Allan and I are both enjoying what is sure to be a super-busy week, but don’t think we’ve forgotten about
This episode was brought to you by
alt="DigitalOcean - Simple Cloud Hosting, Built for Developers" />
src="/images/3.png" alt="Tarsnap - Online Backups for the Truly Paranoid" />
Interview - Benno Rice - [email protected] / @jeamland
Manager, OS & Networking at EMC Isilon
Emily Dunham: Community Automation
iXsystems
1U Rackmount Server - 4 Bay Hot-Swap SAS/SATA Drive Bays 400W Redundant Power Supply - Single Socket Embedded CPU (48 cores) - 8 DIMM Slots with
16GB DIMMs for a total of 128GB RAM – Dual Gigabit LAN, Dual 10GbE SFP+ and 1 x 40Gb QSFP+ port, (1) PCI-E Expansion Slots + IPMI Dedicated LAN -
Cavium ThunderX ARM CN8890 48 Core ThunderX CPU - 2.5GHz per core
System has 128GB RAM, 4 x 2TB SATA HDD, Additional Intel i350 (2 x 1GbE)
Beastie Bits
file considered harmful
An open source talk on ZFS. “Intro to ZFS” as a set of open source slides for the community to build on, and to reuse. Go give this talk at your local conference.
ARMv7 now has a bootloader
SHA256/512 speed improvements in FreeBSD 11
pkgsrc 50th release interviews - Joerg Sonnenberg
DFly versus PC-BSD on a Laptop
FreeBSD ifconfig can print subnet masks in CIDR or dotted-quad, finally
Feedback/Questions
Eli - Getting rid of ports?
Morgan - Best way to admin jails?
Simon - Use existing pkgs in poudriere
Pete - Lots of Q’s
Van - Made the switch
***
1 hr 12 min
June 01, 2016144: The PF life
It’s only one-week away from BSDCan, both Allan and I are excited to meet some of you in person! However, the show keeps on
This episode was brought to you by
Headlines
dotSecurity 2016 - Theo de Raadt - Privilege Separation and Pledge
Video
Slides
Interested in Privilege Separation and security in general? If so, then you are in for a treat, we have both the video and slides from Theo de Raadt at dotSecurity 2016.
Specifically the the talk starts off looking at Pledge (no copyright issues with the pictures I hope??) and how their NTP daemon uses it.
After going through some internals, Theo reveals that around 10% of programs “pledged” so far were found to be trying to do actions outside of their security scope.
On the future-work side, they mention going back and looking at OpenSSH privilege separation next, as well as working with other OS’s that may want pledge support.
***
bhyve now supports UEFI GOP
The log awaited UEFI GOP (Graphics Output Protocol) features has landed in bhyve
This provides emulated graphics via an internal VNC server, allowing users to have full graphical access to the guest OS
This allows installation of Windows guests without needing to create a modified ISO with an unattended installation script
The code has not actually landed in FreeBSD head yet, but has been committed to a project branch
Following a few simple commands, you can compile the new bhyve binary on your -CURRENT system and get started right away
This feature is expected to be included in the upcoming FreeBSD 11.0
This commit drop also brings with it:
XHCI -- an emulated usb tablet device that provides exact mouse positioning in supported OSs
PS2 mouse for fallback if the guest does not support XHCI (Windows 7)
PS2 keyboard
“The code has been tested with Windows 7/8/8.1/10 and Server 2k12/2k16, Ubuntu 15.10, and FreeBSD 10.3/11-CURRENT”
“For VNC clients, TightVNC, TigherVNC, and RealVNC (aka VNC Viewer) have been tested on various hosts. The OSX VNC client is known not to work.”
The VNC server supports an optional ‘wait’ parameter, that causes the VM to not actually boot until the VNC client connects, allowing you to interrupt the boot process if need be
Related user blog post
SVN commit
***
zfsd lands in FreeBSD HEAD, in time for 11.0-RELEASE
zfsd has been committed to FreeBSD -CURRENT in time to be included in FreeBSD 11.0
zfsd is the missing piece required to make ‘hot spares’ work properly in FreeBSD ZFS
“zfsd attempts to resolve ZFS faults that the kernel can't resolve by itself. It listens to devctl(4) events, which is how the kernel notifies of events such as I/O errors and disk removals. Zfsd attempts to resolve these faults by activating or deactivating hotspares and onlining offline vdevs.”
“The administrator never interacts with zfsd directly. Instead, he controls its behavior indirectly through zpool configuration. There are two ways to influence zfsd: assigning hotspares and setting pool properties. Currently, only the autoreplace property has any effect. See zpool(8) for details.”
So, what example does it do?
Device Removal: “When a leaf vdev disappears, zfsd will activate any available hotspare.”
Device Arrival: “When a new GEOM device appears, zfsd will attempt to read its ZFS label, if any. If it matches a previously removed vdev on an active pool, zfsd will online it. Once resilvering completes, any active hotspare will detach automatically.”
So if you disconnect a drive, then reconnect it, it will automatically be brought back online. Since ZFS is smart, the resilver will only have to copy data that has changed since the device went offline.
“If the new device has no ZFS label but its physical path matches the physical path of a previously removed vdev on an active pool, and that pool has the autoreplace property set, then zfsd will replace the missing vdev with the newly arrived device. Once resilvering completes, any active hotspare will detach automatically.”
If the new drive is in the same slot in your hot swap array as a failed device, it will be used as a replacement immediately.
vdev degrade or fault events: “If a vdev becomes degraded or faulted, zfsd will activate any available hotspare. If a leaf vdev generates more than 50 I/O errors in a 60 second period, then zfsd will mark that vdev as FAULTED. zfs(4) will no longer issue any I/Os to it. zfsd will activate
a hotspare if one is available.” Same for checksum errors.
So if zfsd detects a drive is going bad, it brings the hotspare online before it is too late
Spare addition: “If the system administrator adds a hotspare to a pool that is already degraded, zfsd will activate the spare.”
Resilver complete: “zfsd will detach any hotspare once a permanent replacement finishes resilvering.”
Physical path change: “If the physical path of an existing disk changes, zfsd will attempt to replace any missing disk with the same physical path, if its pool's autoreplace property is set.”
In general, this tool means less reliance on the system administrator to keep the pool healthy
***
WX now mandatory in OpenBSD
We’ve talked a bit about WX in the past. (Refresher: Memory being writable and executable at once)
Well, this major security no-no is no-more on OpenBSD. Theo has committed a change which now prevents violations of this policy:
“WX violations are no longer permitted by default. A kernel log message is generated, and mprotect/mmap return ENOTSUP. If the sysctl(8) flag kern.wxabort is set then a SIGABRT occurs instead, for gdb use or coredump creation.”
There are a few cases where you may still need WX, which Theo points out can be enabled on a file-system basis.
“WX violating programs can be permitted on a ffs/nfs filesystem-basis, using the "wxallowed" mount option. One day far in the future upstream software developers will understand that WX violations are a tremendously risky practice and that style of programming will be banished outright. Until then, we recommend most users need to use the wxallowed option on their /usr/local filesystem. At least your other filesystems don't permit such programs.”
This is a great ability to grow, since now users can begin doing auditing of programs that violate this principle and making noise to upstream.
***
Interview - Kristof Provost - [email protected] @kprovst
pf improvements on FreeBSD
***
News Roundup
GELI Support for the EFI Loader
We’ve had Allan’s work to bring GELI support to the GPT / BIOS / ZFS loader for a while now, but the missing piece has been support for EFI.
No longer, Eric McCorkle has posted a blog entry (with relevant github links) introducing us to his work to bring GELI encryption support to EFI.
First the bad-news. This won’t make it into 11.0. (Maybe PC-BSD, TBD)
Next he explains why this is more than just a new feature, but a re-factor of the EFI boot code:
I have already written extensively about my EFI refactoring here. The reason for undertaking this effort, however, was driven by GELI support. Early in my work on this, I had implemented a non-EFI “providers” framework in boot1 in order to support the notion of disk partitions that may contain sub-partitions.
This was deeply unsatisfying to me for several reasons:
It implemented a lot of the same functionality that exists in the EFI framework.
It involved implementing a GPT partition driver to deal with partition tables inside GELI partitions (GPT detection and support is guaranteed by the EFI spec).
The interface between the EFI framework and the custom “providers” framework was awkward.
The driver was completely boot1-specific, and exporting it to something like GRUB probably involved a total rewrite.
Implementing it within loader was going to involve a lot of code duplication.
There was no obvious was to pass keys between boot1, loader, and the kernel.
With the issues known, Eric seems pleased with the results of the conversion so far:
The GELI driver can be extracted from the FreeBSD codebase without too much trouble.
While I was unable to go all the way to the EFI driver model, the only blocker is the bcache code, and once that is resolved, we can have hotplug support in the boot loader!
The boot1 and loader codebases are now sharing all the backend drivers, and boot1 has been reduced to one very small source file.
An interesting read, looking forward to playing with EFI more in the future!
***
Faces of FreeBSD 2016: Michael W. Lucas
On this edition of “Faces of FreeBSD”, Michael W Lucas tells the story of how he got started with FreeBSD
After an amusing re-telling of his childhood (The words “Purina Monkey Chow” were mentioned), he then tells us how he got into BSD.
His being thrown into the project may sound familiar to many:
I came in at 11 PM one night and was told “The DNS administrator just got walked out the door. You’re the new lead DNS administrator. Make those servers work. Good luck.”
From there (because he wanted more sleep), he began ripping out the systems that had been failing and waking him up at night. Good-bye UnixWare, Good-bye Solaris, hello BSD!
A very amusing read, check it out!
***
High Availability with PostgreSQL on FreeBSD
A talk by Sean Chittenden, who we interviewed previously on episode Episode 95
Explains how to setup Multi Data Center High Availability for PostgreSQL using consul
Goes into how consul works, how it does the election, the gossip protocol, etc
The HA setup uses DNS Failover, and the pros and cons of that approach are discussed
Then he walks through the implementation details, and example configuration
***
New FreeBSD i915 testing images
Still need users to test the Linux Kernel 4.6 DRM update to FreeBSD’s graphics stack
Download the test image and write it to a USB stick and boot from it
It will not modify your installed system, it runs entirely off of the USB drive
Allows you to test the updated drivers without having to install the development branch on your device
you can tell them that ATI/AMD support will be coming shortly
and that stability has been steadily improving
and that I'll do another announcement as soon as I've had a chance to test the newest Xorg bits
***
Beastie Bits
Comfortable on the CLI: Series Part 1
FreeBSD Booting on the Netgate uFW, a smaller-than-a-raspberry-pi dual port firewall
Picture of uFW
uFW OpenSSL Benchmarks
***
1 hr 4 min
May 25, 2016143: One small step for DRM, one giant leap for BSD
This week on BSDNow, we have an interview with Matthew Macy, who has some exciting news to share with us regarding the state of graphics
This episode was brought to you by
Headlines
How the number of states affects pf’s performance of FreeBSD
Our friend Olivier of FreeNAS and BSDRP fame has an interesting blog post this week detailing his unique issue with finding a firewall that can handle upwards of 4 million state table entries.
He begins in the article with benchmarking the defaults, since without that we don’t have a framework to compare the later results. All done on his Netgate RCC-VE 4860 (4 cores ATOM C2558, 8GB RAM) under FreeBSD 10.3.
“We notice a little performance impact when we reach the default 10K state table limit: From 413Kpps with 128 states in-used, it lower to 372Kpps.”
With the initial benchmarks done and graphed, he then starts the tuning process by adjusting the “net.pf.states_hashsize”sysctl, and then playing with the number of states for the firewall to keep.
“For the next bench, the number of flow will be fixed for generating 9800 pf state entries, but I will try different value of pf.states_hashsize until the maximum allowed on my 8GB RAM server (still with the default max states of 10k):”
Then he cranks it up to 4 million states
“There is only 12% performance penalty between pf 128 pf states and 4 million pf states.”
“With 10M state, pf performance lower to 362Kpps: Still only 12% lower performance than with only 128 states”
He then looks at what this does of pfsync, the protocol to sync the state table between two redundant pf firewalls
Conclusions:
There need to be a linear relationship between the pf hard-limit of states and the pf.states_hashsize; RAM needed for pf.states_hashsize = pf.states_hashsize * 80 Byte and pf.states_hashsize should be a power of 2 (from the manual page); Even small hardware can manage large number of sessions (it's a matter of RAM), but under too lot's of pressure pfsync will suffer.
Introducing the BCHS Stack = BSD, C, httpd, SQLite
Pronounced Beaches
“It's a hipster-free, open source software stack for web applications”
“Don't just write C. Write portable and secure C.”
“Get to know your security tools. OpenBSD has systrace(4) and pledge(2). FreeBSD has capsicum(4).”
“Statically scan your binary with LLVM” and “Run your application under valgrind”
“Don't forget: BSD is a community of professionals. Go to conferences (EuroBSDCon, AsiaBSDCon, BSDCan, etc.)”
This seems like a really interesting project, we’ll have to get Kristaps Dzonsons back on the show to talk about it
***
Installing OpenBSD's httpd server, MariaDB, PHP 5.6 on OpenBSD 5.9
Looking to deploy your next web-stack on OpenBSD 5.9? If so this next article from rootbsd.net is for you.
Specifically it will walk you through the process of getting OpenBSD’s own httpd server up and running, followed by MariaDB and PHP 5.6.
Most of the setup is pretty straight-forward, the httpd syntax may be different to you, if this is your first time trying it out.
Once the various packages are installed / configured, the rest of the tutorial will be easy, walking you through the standard hello world PHP script, and enabling the services to run at reboot.
A good article for those wanting to start hosting PHP/DB content (wordpress anyone?) on your OpenBSD system.
***
The infrastructure behind Varnish
Dogfooding. It’s a term you hear often in the software community, which essentially means to “Run your own stuff”. Today we have an article by PKH over at varnish-cache, talking about what that means to them.
Specifically, they recently went through a website upgrade, which will enable them to run more of their own stuff.
He has a great quote on what OS they use:“So, dogfood: Obviously FreeBSD. Apart from the obvious reason that I wrote a lot of FreeBSD and can get world-class support by bugging my buddies about it, there are two equally serious reasons for the Varnish Project to run on FreeBSD: Dogfood and jails.Varnish Cache is not “software for Linux”, it is software for any competent UNIX-like operating system, and FreeBSD is our primary “keep us honest about this” platform.“
He then goes through the process of explaining how they would setup a new Varnish-cache website, or upgrade it.
All together a great read, and if you are one of the admin-types, you really should pay attention to how they build from the ground up. Some valuable knowledge here which every admin should try to replicate.
I can not reiterate the value of having your config files in a private source control repo strongly enough
The biggest take-away is: “And by doing it this way, I know it will work next time also.”
***
Interview - Matt Macy - [email protected]Graphics Stack Update
News Roundup
Followup on packaging base with pkg(8)
In spite of the heroic last minute effort by a team of contributors, pkg’d base will not be ready in time for FreeBSD 11.0
There are just too many issues that were discovered during testing
The plan is to continue using freebsd-update in the meantime, and introduce a pkg based upgrade mechanism in FreeBSD 11.1
With the new support model for the FreeBSD 11 branch, 11.1 may come sooner than with previous major releases
***
FreeBSD Core Election
It is time once again for the FreeBSD Core Election
Application period begins: Wednesday, 18 May 2016 at 18:00:00 UTC
Application period ends: Wednesday, 25 May 2016 at 18:00:00 UTC
Voting begins: Wednesday, 25 May 2016 at 18:00:00 UTC
Voting ends: Wednesday, 22 June 2016 at 18:00:00 UTC
Results announced Wednesday, 29 June 2016
New core team takes office: Wednesday, 6 July 2016
As of the time I was writing these notes, 3 hours before the application deadline, the candidates are:
Allan Jude: Filling in the potholes
Marcelo Araujo: We are not vampires, but we need new blood.
Baptiste Daroussin (incumbent): Keep on improving
Benedict Reuschling: Learn and Teach
Benno Rice: Revitalising The Community
Devin Teske: Here to help
Ed Maste (incumbent): FreeBSD is people
George V. Neville-Neil (incumbent): There is much to do…
Hiroki Sato (incumbent): Keep up with our good community and technical strength
John Baldwin: Ready to work
Juli Mallett: Caring for community.
Kris Moore: User-Focused
Mathieu Arnold: Someone ask for fresh blood ?
Ollivier Robert: Caring for the project and you, its developers
The deadline for applications is around the time we finish recording the live show
We welcome any of the candidates to schedule an interview in the next few weeks. We will make an attempt to hunt many of them down at BSDCan as well.
***
Wayland/Weston with XWayland works on DragonFly
We haven’t talked a lot about Wayland on BSD recently (or much at all), but today we have a post from Peter to the dragonfly mailing list, detailing his experience with it.
Specifically he talks about getting XWayland working, which provides the compat bits for native X applications to run on WayLand displays.
So far on the working list of apps:
“gtk3:
gedit
nautilus
evince
xfce4:
xfce4-terminal
atril
firefox
spyder
scilab”
A pretty impressive list, although he said “chrome” failed with a seg-fault
This is something I’m personally interested in. Now with the newer DRM bits landing in FreeBSD, perhaps it’s time for some further looking into Wayland.
***
Broadcom WiFi driver update
In this blog post Adrian Chadd talks about his recent work on the bwn(4) driver for Broadcom WiFi chips
This work has added support for a number of older 802.11g chips, including the one from 2009-era Macbooks
Work is ongoing, and the hope is to add 802.11n and 5ghz support as well
Adrian is mentoring a number of developers working on embedded or wifi related things, to try to increase the projects bandwidth in those areas
If you are interested in driver development, or wifi internals, the blog post has lots of interesting details and covers the story of Adrian’s recent adventures in bringing the drivers up
***
Beastie Bits
The Design of the NetBSD I/O Subsystems (2002)
ZFS, BTRFS, XFS, EXT4 and LVM with KVM – a storage performance comparison
Swift added to FreeBSD Ports
misc@openbsd: 'NSA addition to ifconfig'
Papers We Love: Memory by the Slab: The Tale of Bonwick's Slab Allocator
Feedback/Questions
Lars - Poudriere
Warren - .NET
Eddy - Sys Init
Tim - ZFS Resources
Morgan - Ports and Kernel
***
2 hr
May 18, 2016142: Diving for BSD Perls
This week on the show, we have all the latest news and stories! Plus an interview with BSD developer Alfred Perlstein, that you
This episode was brought to you by
Headlines
The May issus of BSDMag is now out
GhostBSD
Reusing OpenBSD's arc4random in multi-threaded user space programs
Securing VPN's with GRE / Strongswan
Installing XFCE 4.12 on NetBSD 7
Interview with Fernando Rodriguez, the co-founder of KeepCoding
***
A rundown of the FPT_WX_EXT.1 security reqiurement for General Purpose Operating Systems by the NSA
NIST/NSA Validation Scheme Report
The SFR or Security Functional Requirement requires that; "The OS shall prevent allocation of any memory region with both write and execute permissions except for [assignment: list of exceptions]."
While nearly all operating systems currently support the use of the NX bit, or the equivalent on processors such as SPARC and ARM, and will correctly mark the stack as non-executable, the fact remains that this in and of itself is deemed insufficient by NIST and NSA.
OpenBSD 5.8, FreeBSD, Solaris, RHEL, and most other Linux distro have failed.
HardenedBSD passes all three tests out of the box.
NetBSD will do so with a single sysctl tweak. Since they are using the PaX model, anything else using PaX, such as a grsecurity-enabled Linux distribution pass these assurance activities as well.
OpenBSD 5.9 does not allow memory mapping due to WX being enforced by the kernel, however the kernel will panic if there are any attempts to create such mappings.
***
DistroWatch reviews new features in FreeBSD 10.3
DistroWatch did a review of FreeBSD 10.3
They ran into a few problems, but hopefully those can be fixed
An issue with beadm setting the canmount property incorrectly causing the ZFS BE menu to not work as expected should be resolved in the next version, thanks to a patch from kmoore
The limitations of the Linux 64 support are what they are, CentOS 6 is still fairly popular with enterprise software, but hopefully some folks are interested in working on bringing the syscall emulation forward
In a third issue, the reviewer seemed to have issues SSHing from inside the jail. This likely has to do with how they got a console in the jail. I remember having problems with this in the past, something about a secure console.
***
BSD Unix: Power to the people, from the code
Salon.com has a very long article, chronicling much of the history behind BSD UNIX.
It starts with detailing the humble origins of BSD, starting with Bill Joy in the mid-70’s, and then goes through details on how it rapidly grew, and the influence that the University of Berkeley had on open-source.
“But too much focus on Joy, a favorite target for business magazine hagiography, obscures the larger picture. Berkeley’s most important contribution was not software; it was the way Berkeley created software. At Berkeley, a small core group — never more than four people at any one time — coordinated the contributions of an ever-growing network of far-flung, mostly volunteer programmers into progressive releases of steadily improving software. In so doing, they codified a template for what is now referred to as the “open-source software development methodology.” Put more simply, the Berkeley hackers set up a system for creating free software.”
The article goes on to talk about some of the back and forth between Linux and BSD, and why Linux has captured more of the market in recent years, but BSD is far from throwing in the towel.
“BSD patriots argue that the battle is far from over, that BSD is technically superior and will therefore win in the end. That’s for the future to determine. What’s indisputable is BSD’s contribution in the past. Even if, by 1975, Berkeley’s Free Speech Movement was a relic belonging to a fast-fading generation, on the fourth floor of Evans Hall, where Joy shared an office, the free-software movement was just beginning.”
An excellent article (If a bit long), but well worth your time to understand the origins of what we consider modern day BSD, and how the University of Berkley helped shape it.
***
iXsystems
#ServerEnvy: It's over 10,000 Terabytes!
***
Interview - Alfred Perlstein - [email protected] / @splbio
Using BSD for projects
***
News Roundup
.NET framework ported to NetBSD
This pull request adds basic support for the .NET framework on NetBSD 7.x amd64
It includes documentation on how to get the .NET framework installed
It uses pkgsrc to bootstrap the required tools
pkgsrc-wip is used to get the actual .NET framework, as porting is still in progress
The .NET Core-CLR is now available for: FreeBSD, Linux, NetBSD, and OS X
***
OpenBSD SROP mitigation – call for testing
A new technique for exploiting flaws in applications and operating systems has been developed, called SROP
“we describe Sigreturn Oriented Programming (SROP), a novel technique for exploits and backdoors in UNIX-like systems. Like return-oriented programming (ROP), sigreturn oriented programming constructs what is known as a ‘weird machine’ that can be programmed by attackers to change the behavior of a process. To program the machine, attackers set up fake signal frames and initiate returns from signals that the kernel never really delivered. This is possible, because UNIX stores signal frames on the process’ stack.”
“Sigreturn oriented programming is interesting for attackers, OS developers and academics. For attackers, the technique is very versatile, with pre-conditions that are different from those of existing exploitation techniques like ROP. Moreover, unlike ROP, sigreturn oriented programming programs are portable. For OS developers, the technique presents a problem that has been present in one of the two main operating system families from its inception, while the fixes (which we also present) are non-trivial. From a more academic viewpoint, it is also interesting because we show that sigreturn oriented programming is Turing complete.”
Paper describing SROP
OpenBSD has developed a mitigation against SROP
“Utilizing a trick from kbind(2), the kernel now only accepts signal returns from the PC address of the sigreturn(2) syscall in the signal trampoline. Since the signal trampoline page is randomized placed per process, it is only known by directly returning from a signal handler.”
“As well, the sigcontext provided to sigreturn(2) now contains a magic cookie constructed from a per-process cookie XOR'd against the address of the signal context.”
This is just a draft of the patch, not yet considered production quality
***
Running Tor in a NetBSD rump unikernel
We’ve talked about “rump” kernels before, and also Tor pretty frequently, but this new github project combines the two!
Specifically, this set of Makefile and scripts will prep a system to run Tor via the Unikernel through Qemu.
The script mainly describes how to do the initial setup on Linux, using iptables, but could easily be adapted to a BSD if somebody wants to do so. (Send them a pull request with the instructions!)
All in all, this is a fascinating way to run a Tor node or relay, in the most minimal operating environment possible.
***
An update on SSH protocol 1 ("we're most of the way towards fully deprecating SSH protocol 1"
Damien Miller has given us an update on the status of the “SSH protocol 1”, and the current plans to deprecate it in an upcoming version of openssh.
“We've had this old protocol in various stages of deprecation for almost 10 years and it has been compile-time disabled for about a year.
Downstream vendors, to their credit, have included this change in recent OS releases by shipping OpenSSH packages that disable protocol 1 by default and/or offering separate, non-default packages to enable it.
This seems to have proceeded far more smoothly than even my most optimistic hopes, so this gives us greater confidence that we can complete the removal of protocol 1 soon. We want to do this partly to hasten the demise of this cryptographic trainwreck, but also because doing so removes a lot of legacy code from OpenSSH that inflates our attack surface. Having it gone will make our jobs quite a bit easier as we maintain and refactor.”
The current time-line looks like removing server-size protocol 1 support this August after OpenSSH 7.4 is released, leaving client-side disabled.
Then a year from now (June 2017) all protocol 1 code will be removed.
Beastie Bits
Last day to get your BSDNow Shirts! Order now, wear at BSDCan!
Move local government (Austin TX) from Microsoft Windows (incl. Office) to Linux and/or PC-BSD
Plan9 boot camp is back... and already at capacity. Another opportunity may come in September
Smaller is better - building an openbsd based router
Baby Unix
Security Update for FreeBSD & Another security update for FreeBSD
Feedback/Questions
Eric - The iX experience
Mike - Building Ports
David - ZFS Backups
James - BSD VPS
Rich - ZFS Followup
***
1 hr 37 min
May 11, 2016141: BSD Likes Ike!
This week on the show, we have all the latest news and stories! Plus we’ll be hearing more about OpnSense from the man himself, Ike!
This episode was brought to you by
Headlines
Regarding Embargoes
Our buddy TedU has a great thought piece today on the idea of “embargoes” for security advisories.
This all stemmed from a recent incident with LibreSSL patches from embargoed OpenSSL vulns, that accidentally got committed too early.
Ted makes a pretty good case on the difficulties of having embargos, and maybe the reason there shouldn’t be. Couple of quotes to give you a taste:
“There are several difficulties maintaining embargoes. Keeping secrets is against human nature. I don’t want to be the one who leaks, but if I see something that looks like the secret is out, it’s a relief to be able to speak freely. There is a bias towards recognizing such signs where they may not really exist. (Exacerbated by broad embargoes where some parts leak but other parts don’t. It’s actually very hard to tell what’s not publicly known when you know everything.)
The most thorough embargo and release timeline reconstruction is the heartbleed timeline. It’s another great case study. Who exactly decided who were the haves and have nots? Was it determined by who needed to know or who you needed to know? Eventually the dam started to crack.”
“When Cloudflare brags that they get advance notice of vulnerabilities, attracting more customers, and therefore requiring even more early access, how are smaller players to compete? What happens if you’re not big enough to prenotify?
Sometimes vulnerabilities are announced unplanned. Zero day cyber missiles are part of our reality, which means end users don’t really have the luxury of only patching on Tuesday. They need to apply patches when they appear. If applying patches at inconvenient times is a problem, make it not a problem. Not really a gripe about embargoes per se, but the scheduled timing of coordinated release at the end of the embargo is catering to a problem that shouldn’t exist.”
I will admit that CloudFlare bragging around Heartbleed was upsetting
The biggest issue here is the difficulty with coordinating so many open source projects, which are often done by volunteers, in different countries and time zones
The other issue is determining when the secret is “out of the bag”
***
MAJOR ABI BREAK: csu, ld.so, libc, libpthread update
OpenBSD warns those following the -current (development) branch to be careful as they upgrade because of a major ABI break that will result in applications not working
“Handling of single-threaded programs is now closer to multi-threaded, with ld.so and libc.a doing thread information base (TIB) allocation. Threaded programs from before the 2016/03/19 csu and ld.so update will no longer run. An updated ld.so must be built and installed before running make build.”
A special note for those on PowerPC: “PowerPC has been updated to offset the TIB from the hardware register. As a result, all threaded programs are broken until they have been rebuilt with the new libc and libpthread. perl must be built after building the libraries and before building the rest of base.”
“The definitions of environ and __progname for dynamically linked programs have been moved from the C startup code to ld.so(1). An updated ld.so must be built and installed before running make build.”
The link provides instructions on how to update your system properly
***
How to install FreeBSD 10.3 on VMWare Workstation 12 Pro
This tutorial starts at the very basics, running through the FreeBSD installer
But then it goes on to configuring the machine specifically for VMWare
After the system has been booted, the tutorial walks through installing the VMWare tools
Then networking is configured in both VMWare and FreeBSD
A small hack is required to make the VMWare tools startup script wait until the network is up
A very nice tutorial for people using VMWare
I am working on a patch to bsdinstall to ensure that the swap partition is put before the main partition, so it can more easily be resized if you later decide you need more space in your VM
the camcontrol reprobe subcommand has been added ,
“This makes it possible to manually force updating capacity data after the disk got resized. Without it it might be necessary to reboot before FreeBSD notices updated disk size under eg VMWare.”
***
BSD Router project releases v1.59
We’ve talked about the BSD Router project a bit in the past, but today we have a brand new release to bring to you.
For those who don’t remember, the BSDrp is a router aimed at replacing more of your big-commercial type systems.
First up in the new hotness, we have it based upon recently released FreeBSD 10.3!
In addition, there is a new package: New package: mlvpn (aggregated network links in order to benefit from the bandwidth of multiple links)
Other packages have gotten a bump with this release as well:
bsnmp-ucd to 0.4.2
dma to 0.11
dmidecode to 3.0
exabgp to 3.4.15
iperf3 to 3.1.2
monit to 5.17
mpd5 to 5.8
openvpn to 2.3.10
python to 2.7.11
quagga to 1.0.20160315
strongswan to 5.4.0
What are you waiting for? Amd64 and i386 images are ready for you to download now.
Interview - Isaac (.Ike) Levy -
See Ike again at SEMIBug in Troy, Michigan on May 17th
***
News Roundup
Tredly - Prebuilt containers on FreeBSD
Discussion regarding its GPLv3 licensing
A new “container” solution called “Trendly” has started making some news around various tech sites.
In particular, this new project uses FreeBSD as its base OS and jail functionality in the backend.
Their solution seems based around the idea of shipping containers as manifests, such as lists of packages to install and configuration knobs.
The project is still rather new, and we’ll be keeping an eye on it for the future.
One notable change already though, it was (for some reason) released under GPLv3. Understandably this caused quite a ruckus with various folks in the community, since it’s built specifically on BSD. Since this, the code has been re-licensed as MIT, which is far more in the spirit of a traditional BSD license.
***
NVMe driver added to NetBSD - ported from OpenBSD
NetBSD has gained support for Non-Volatile Memory Express, the new standard for PCIe attached Flash Memory
The change of interface from SATA to NVMe offers a number of advantages, mostly, it doesn’t require the device to pretend to be a spinning disk
One of the biggest advantages is that it supports completing multiple operations at once, with the Intel hardware I have tested, 63 I/Os can happen concurrently, so a very large queue depth is required to keep the device busy. The 64th I/O channel is reserved for administrative commands, to keep them from being delayed by the large queue depth
The device I tested could read at 3800 MB/s, and write 1700MB/s, something that wouldn’t be possible with a normal SSD
It is interesting that NetBSD took the NVMe support from OpenBSD, whereas the FreeBSD implementation was contributed directly by Intel
This may have to do with that fact that OpenBSD’s device model is closer to that of NetBSD
Commit Log
***
New BSDNow T-Shirts
By popular demand, we have created a more subtle BSDNow shirt
Featuring only the smallish BSDNow logo over the left breast
Available in a number of styles (T-Shirt, Women’s T-Shirt, Long Sleeve, and Hoodie) as well as a number of colours: Black, Blue, Grey, and White
The hope is that enough orders come though so we can get them shipped in and your sweaty little hands in time for BSDCan. (I’ll be wearing mine, will you B...SD?)
If you still want one of our now-famous “The Usual BSD’s” t-shirts, you can also indicate your interest here, and once 10 or more shirts are ordered, a reprint will happen automatically
***
PC-BSD 11-CURRENT with Package Base
Looking for a way to play with the new FreeBSD base package system?
This month’s PC-BSD -CURRENT image now used packages for base system installation, and is asking for testers to help find bugs.
Known issues so far:
setuid binaries (Fix in works)
Missing tzone files
Distrib packages
If all that doesn’t scare you away, then give it a whirl! Upgrades for previous APRIL images are now online also.
***
BeastieBits
HardenedBSD + LibreSSL
Michael Dexter's talk at LFNW 2016 is the 2nd highest youtube views from this years conference
Why OpenBSD is important to me
Study of nginx-1.9.12 performance/latency on DragonFlyBSD-g67a73
Running FreeBSD / OpenBSD / NetBSD as a virtualised guest on Online.net
The interesting story of how IllumOS syscalls work
The BeaST is the FreeBSD based dual-controller reliable storage system concept with aim to implement ZFS and in-memory cache.
Francois Tigeot updates the drm/i915 driver to match what’s in Linux kernel 4.3
FreeBSD is working on the update to Linux Kernel 4.6, we may finally get ahead of Dragonfly!
Feedback/Questions
Oskar - Torrent Jail
Shane - ZFS Delete
Adam - Zimbra Port
Ray - PC-BSD - FrameBuffer
Richard - ZFS Backups
***
1 hr 42 min
May 04, 2016140: Tracing it back to BSD
This week on BSDNow, Allan is back in down from Europe! We’ll get to hear some of his wrap-up and get caught up on the latest BSD
This episode was brought to you by
Headlines
FreeBSD Quarterly Report
This quarterly status report starts with a rather interesting introduction by Warren Block
ASLR
Porting CEPH to FreeBSD
RCTL I/O Rate Limiting
The Graphics Stack on FreeBSD (Haswell is in, work is progressing on the next update)
CAM I/O Scheduler
NFS Server updates, working around the 16 group limit, and implementing pNFS, allowing NFS to scale beyond a single server
Static Analysis of the FreeBSD Kernel with PVS Studio
PCI-express HotPlug
GitLab Port committed!
WITH_FAST_DEPEND and other improvements to the FreeBSD build system
Lots of other interesting stuff
***
A Prog By Any Other Name
Ted Unangst looks at what goes into the name of a program
“Sometimes two similar programs are really the same program with two names. For example, grep and egrep are two commands that perform very similar functions and are therefore implemented as a single program. Running ls -i and observing the inode number of each file will reveal that there is only one file. Calling the program egrep is a shorthand for -E and does the same thing.”
So BSD provides __progname in libc, so a program can tell what its name is
But, what if it has more than one name?
“In fact, every program has three names: its name in the filesystem, the name it has been invoked with, and whatever it believes its own name to be.”
Of course it is not that easy.
“there’s another set of choices for each name, the full path and the basename”
“It’s even possible on some systems for argv[0] to be NULL.”
He then goes on to rename doas (the OpenBSD light replacement for sudo) to banana and discuss what happens
“On that note, another possible bug is to realize that syslog by default uses progname. A user may be able to evade log monitoring by invoking doas with a different name. (Just fixed.)”
Another interesting article from our friend Ted
***
FreeBSD and NetBSD Google Summer of Code projects have been announced
Some FreeBSD highlights:
Add SCSI passthrough to CTL (share an optical drive via iSCSI)
Add USB target mode driver based on CTL (share a USB device via iSCSI)
API to link created /dev entries to sysctl nodes
Implement Ethernet Ring Protection Switching (ERPS)
HD Audio device model in userspace for bhyve
Some NetBSD highlights:
Implement Ext4fs support in ReadOnly mode
NPF and blacklistd web interface
Port U-Boot so it can be compiled on NetBSD
Split debug symbols for pkgsrc builds
***
libressl - more vague priomises
We haven’t had a Ted U article on the show as of late, however this week we get several! In his next entry “LibreSSL, more vague promises”
He then goes into some detail on what has happened with LibreSSL in the past while, as well as future plans going forward.
“With an eye to the future, what new promises can we make? Some time ago I joked that we only promised to make a better TLS implementation, not a better TLS. Remains true, but fortunately there are people working on that, too. TLS 1.3 support is on the short term watchlist. The good news is we may be ahead of the game, having already removed compression. How much more work can there be?”
“LibreSSL integrated the draft chacha20-poly1305 construction from BoringSSL. The IETF has since standardized a slightly different version because if it were the same it wouldn’t be different. Support for standard variant, and the beginning of deprecation for the existing code, should be landing very shortly. Incidentally, some people got bent out of shape because shipping chacha20 meant exposing non IANA approved numbers to Internet. No promises that won’t happen again.”
***
Interview - Samy Al Bahra - @0xF390
Backtrace
***
News Roundup
systrace(1) is removed for OpenBSD 6.0
OpenBSD has removed systrace, an older mechanism for limiting what syscalls an application can make
It is mostly replaced by the pledge() system
OpenBSD was the first implementation, most others have been unmaintained for some time
The last reported Linux version was for kernel 2.6.1
NetBSD removed systrace in 2007
***
pfSense Video Series: Comprehensive Guide To pfSense 2.3
A series of videos (11 so far), about pfSense
Covers Why you would use it, how to pick your hardware, and installation
Then the series covers some networking basics, to make sure you are up to speed before configuring your pfSense
Then a comprehensive tour of the WebUI
Then goes on to cover graphing, backing up and restoring configuration
There are also videos on running DHCP, NTP, and DNS servers
***
DuckDuckGo announces its 2016 FOSS Donations
The theme is “raising the standard of trust online”
Supported projects include:
OpenBSD Foundation announces DuckDuckGo as a Gold Sponsor
the Freedom of the Press Foundation for SecureDrop
the Freenet Project
the CrypTech Project
the Tor Project
Fight for the Future for Save Security
Open Source Technology Improvement Fund for VeraCrypt (based on TrueCrypt)
Riseup Labs for LEAP (LEAP Encryption Access Project)
GPGTools for GPGMail
***
Larry the BSD Guy hangs up his hat at FOSS Force
After 15 years, Larry the BSD Guy has decided to hang it up, and walk into the sunset! (Figuratively of course)
After wrapping up coverage of recent LinuxFest NorthWest (Which he didn’t attend), Larry has decided it’s time for a change and is giving up his column over at FOSS Force, as well as stepping away from all things technical.
His last write-up is a good one, and he has some nice plugs for both Dru Lavigne and Michael Dexter of the BSD community.
He will be missed, but we wish him all the luck with the future! He also puts out the plug that FOSS Force will be needing a new columnist in the near future, so if you are interested please let them know!
***
Beastie Bits
If you sponsored “FreeBSD Mastery: Advanced ZFS”, check your mail box
pkg-1.7.0 is an order of magnitude slower than pkg-1.6.4 -- Caused by a problem not in pkg
LinuxFest Northwest 2016 Recap
Dru Lavigne's 'Doc like an Egyption' talk from LFNW
Michael Dexters' 'Switching to BSD from Linux' talk from LFNW
Michael Dexters' 'Secrets to enduring user groups' talk from LFNW
January issue of Freebsd Journal online for free
Ghost BSD releases 10.3 Alpha1 for testing
EuroBSDcon 2016 - Call for Papers - Dealine: May 8th
KnoxBUG Initial Meeting
Photos, slides, and videos from the Open Source Data Center Conference
***
Feedback/Questions
Mohammad - Replication
John - Rolling new packages
Clint - Unicast
Bill - GhostBSD
Charles - BSD Videos
***
1 hr 32 min
April 27, 2016139: Cheri-picking BSD
This week, Allan is out of town, but since when has that ever stopped us from bringing you a new episode of BSDNow? We have news,
This episode was brought to you by
Headlines
Unix's file durability problem
Another article by Chris Siebenmann from the University of Toronto
This time, the issue was a lost comment on his Python based blog which uses files on disk rather than a database
After an unexpected restart of the system, a recently posted comment no longer existed
The post goes on to investigate what the ‘right way’ to ensure file durability is
The answer, as you might expect, is “it depends…”
Normally, fsync() should work, but it seems with ext4 and some other file systems, you must also fsync() the directory where the file was created, or it might not be possible to find the file after a crash
Do you need to fsync() the parent of that directory too? Then what is fdatasync() for? What about just calling sync()?
“One issue is that unlike many other Unix API issues, it's impossible to test to see if you got it all correct and complete. If your steps are incomplete, you don't get any errors; your data is just silently sometimes at risk. Even with a test setup to create system crashes or abrupt power loss (which VMs make much easier), you need uncommon instrumentation to know things like if your OS actually issued disk flushes or just did normal buffered writes. And straightforward testing can't tell you if what you're doing will work all the time, because what is required varies by Unix, kernel version, and the specific filesystem involved.”
Second post by author: How I'm trying to do durable disk writes
Additional Discussion on Hacker News
The discussion on HN also gets into AIO and other more complicated facilities, but even those seem to be vague about when your data is actually safe
At least ZFS ensures you never get half of your new data, and half of your old data.
***
Build a FreeBSD 10.3-release Openstack Image with bsd-cloudinit
Are you using FreeBSD and OpenStack or would you like to be? We next have a great tutorial which explains the ins-and-outs of doing exactly that.
Remy van Elst brings us a great walkthrough on his site on how to get started, and hint it involves just a few ‘pip’ commands.
After getting the initial Python tools bootstrapped, next he shows us how to save our OpenStack settings in a sourceable shell command, which comes in handy before doing admin on a instance.
Next the ‘glance’ and ‘cinder’ tools are used to upload the target OS ISO file and then create a volume for it to install onto.
Next the VM is started and some specific steps are outlined on getting FreeBSD 10.3 installed into the instance. It includes some helpful hints as how to fix a mountroot error, if you installed to ada0, but need to mount via vtdb0 instead now.
After the installation is finished, the prep for ‘cloudinit’ is done, and the resulting image is compressed and made ready for deployment.
We’ve kinda stepped through some of the more gory steps here, but if OpenStack is something you work with, this tutorial should be at the top of your “must read” list.
***
Undeadly and HTTPS
Undeadly, the OpenBSD journal, is thinking of moving to HTTPS only
In order to do this, they would like some help rewriting part of the site
Currently, when you login to post comments, this is done over HTTPS, but to an stunnel instance running a custom script that gives you a cookie, and sends you back to the non-HTTPS site
They would like to better integrate the authentication system, and otherwise improve the code for the site
There is some pushback as well, questioning whether it makes sense to block users who are unable to use HTTPS for one reason or another
I think it makes sense to have the site default to HTTPS, but, maybe HTTPS only doesn’t make sense. There is nothing private on the site, other than the authentication system which is optional, not required to post a comment.
There is also some discussion about the code for the site, including the fact that when the code was released, the salt for the password database was included
This is not actually a security problem, but the discussion may be interesting to some viewers
***
FreeBSD Journal March/April Edition
The next issue of the FreeBSD Journal is here, and this time it is about Teaching with Operating Systems
In addition to the usual columns, including: svn update, the ports report, a conference report from FOSDEM, a meetup report from PortsCamp Taipei, A book review of "The Algorithm Design Manual", and the Events Calendar; there are a set of feature articles about teaching
Teaching with FreeBSD through Tracing, Analysis, and Experimentation
CHERI: Building a foundation for secure, trusted computing bases
A brief history of Fast Filesystems
There is also an interview with Gleb Smirnoff, a member of the Core team, release engineering, and the deputy security officer, as well as a senior software developer at Netflix
Get the latest issue from your favourite mobile store, or the “Desktop Edition” directly in your browser from the FreeBSD Foundation’s website
***
Interview - Brooks Davis - [email protected] / @brooksdavis
CHERI and Capabilities
***
TrueNAS Three-Peats!!!
News Roundup
UbuntuBSD Is Looking To Become An Official Ubuntu Flavor
You may recall a few weeks back that we were a bit surprised by the UbuntuBSD project and its longevity / goals.
However the project seems to be pushing forward, with news on softpedia.com that they are now seeking to become an ‘official’ Ubuntu Flavor.
They’ve already released a forth beta, so it seems the project currently has some developers pushing it forward:
"I would like to contribute all my work to Ubuntu Community and, if you think it is worthy, make ubuntuBSD an official Ubuntu project like Xubuntu or Edubuntu," said Jon Boden. "If you're interested, please let me know how would you like me to proceed."
It's Just Bits
We have next an interesting blog post talking about the idea that “It’s just all bits!”
The author then takes us down the idea of no matter how old or mysterious the code may be, in the end it is ending up as bits arranged a certain way.
Then the article transitions and takes us through the idea that old bits, and bits that have grown too large should often be good candidates for replacement by “simpler” bits, using OpenBSD as an example.
“The OpenBSD community exemplifies this in many ways by taking existing solutions and simplifying them. Processing man pages is as old as Unix, and even in the 21st century OpenBSD has taken the time to rewrite the existing solution to be simpler and safer. It's just bits that need to be turned into other bits. Similarly, OpenBSD has introduced doas as an alternative to sudo. While not replacing sudo entirely, doas makes the 99.99% case of what people use sudo for easier and safer. They are just bits that need to be authenticated. “
All in all, a good read, and it reinforces the point that nothing is really truly “finished”. As computing advances and new technologies / practices are made available, sometimes it makes a lot of sense to go back and re-write things in order to simplify the complexity that has snuck in over time.
***
Disk IO limiting is coming to FreeBSD
A much requested feature for both Jails and VM’s on FreeBSD has just landed with experimental support in -HEAD, Disk IO limiting!
The Commit message states as follows:
“Add four new RCTL resources - readbps, readiops, writebps and writeiops,
for limiting disk (actually filesystem) IO.
Note that in some cases these limits are not quite precise. It's ok,
as long as it's within some reasonable bounds.
Testing - and review of the code, in particular the VFS and VM parts - is
very welcome.”
Well, what are you waiting for? This is a fantastic new feature which I’m sure will get incorporated into other tools for controlling jails and VM’s down the road.
If you give it a spin, be sure to report back bugs so they can get quashed in time for 11.
***
BeastieBits
PC-BSD 10.3 Is the Last in the Series, PC-BSD 11.0 Arrives Later This Year
ASLR now on by default in NetBSD amd64
Daniel Bilik's fix for hangs on Baytrail
Don’t forget about PGCon 2016
Get your paper in for EuroBSDCon 2016, deadline is May 8th
Feedback/Questions
John - Destroy all Dataset
Thomas - Misc Questions
Ben - ZFS Copy
Bryson - SysV IPC
Drin - IPSEC
***
1 hr 10 min

About BSD Now

From the publisher's feed

Created by three guys who love BSD, we cover the latest news and have an extensive series of tutorials, as well as interviews with various people from all areas of the BSD community. It also serves as…