Sign up to save your podcasts
Or
Share 190: The Moore You Know
Share to email
Share to Facebook
Share to X
Share to email
Share to Facebook
Share to X
Or
190: The Moore You Know
This week, we look forward with the latest OpenBSD release, look back with Dennis Ritchies paper on the evolution of Unix Time Sharing, have an Interview with Kris
This episode was brought to you by
href="http://www.digitalocean.com/" title="DigitalOcean">
href="http://www.tarsnap.com/bsdnow" title="Tarsnap">
OpenBSD 6.1 RELEASED
***
Fuzz Testing OpenSSH
***
Getting OpenBSD running on Raspberry Pi 3
Ian Darwin writes in about his work deploying the arm64 platform and the Raspberry Pi 3
So I have this empty white birdhouse-like thing in the yard, open at the front. It was intended to house the wireless remote temperature sensor from a low-cost weather station, which had previously been mounted on a dark-colored wall of the house [...]. But when I put the sensor into the birdhouse, the signal is too weak for the weather station to receive it (the mounting post was put in place by a previous owner of our property, and is set deeply in concrete). So the next plan was to pop in a tiny OpenBSD computer with a uthum(4) temperature sensor and stream the temperature over WiFi.
The Raspberry Pi computers are interesting in their own way: intending to bring low-cost computing to everybody, they take shortcuts and omit things that you'd expect on a laptop or desktop. They aren't too bright on their own: there's very little smarts in the board compared to the "BIOS" and later firmwares on conventional systems. Some of the "smarts" are only available as binary files. This was part of the reason that our favorite OS never came to the Pi Party for the original rpi, and didn't quite arrive for the rpi2. With the rpi3, though, there is enough availability that our devs were able to make it boot. Some limitations remain, though: if you want to build your own full release, you have to install the dedicated raspberrypi-firmware package from the ports tree. And, the boot disks have to have several extra files on them - this is set up on the install sets, but you should be careful not to mess with these extra files until you know what you're doing!
But wait! Before you read on, please note that, as of April 1, 2017, this platform boots up but is not yet ready for prime time:
But wait - there's more! The "USB disk" can be a USB thumb drive, though they're generally slower than a "real" disk. My first forays used a Kingston DTSE9, the hardy little steel-cased version of the popular DataTraveler line. I was able to do the install, and boot it, once (when I captured the dmesg output shown below). After that, it failed - the boot process hung with the ever-unpopular "scanning usb for storage devices..." message. I tried the whole thing again with a second DTSE9, and with a 32GB plastic-cased DataTraveler. Same results. After considerable wasted time, I found a post on RPI's own site which dates back to the early days of the PI 3, in which they admit that they took shortcuts in developing the firmware, and it just can't be made to work with the Kingston DataTraveler! Not having any of the "approved" devices, and not living around the corner from a computer store, I switched to a Sabrent USB dock with a 320GB Western Digital disk, and it's been rock solid. Too big and energy-hungry for the final project, but enough to show that the rpi3 can be solid with the right (solid-state) disk. And fast enough to build a few simple ports - though a lot will not build yet. I then found and installed OpenBSD onto a PNY brand thumb drive and found it solid - in fact I populated it by dding from one of the DataTraveller drives, so theyre not at fault.
***
Dennis M. Ritchies Paper: The Evolution of the Unix Time Sharing System
This paper presents a brief history of the early development of the Unix operating system. It concentrates on the evolution of the file system, the process-control mechanism, and the idea of pipelined commands. Some attention is paid to social conditions during the development of the system.
During the past few years, the Unix operating system has come into wide use, so wide that its very name has become a trademark of Bell Laboratories. Its important characteristics have become known to many people. It has suffered much rewriting and tinkering since the first publication describing it in 1974 [1], but few fundamental changes. However, Unix was born in 1969 not 1974, and the account of its development makes a little-known and perhaps instructive story. This paper presents a technical and social history of the evolution of the system.
Origins
The PDP-7 Unix file system
Process control
IO Redirection
The advent of the PDP-11
The first PDP-11 system
Pipes
High-level languages
Conclusion
One of the comforting things about old memories is their tendency to take on a rosy glow. The programming environment provided by the early versions of Unix seems, when described here, to be extremely harsh and primitive. I am sure that if forced back to the PDP-7 I would find it intolerably limiting and lacking in conveniences. Nevertheless, it did not seem so at the time; the memory fixes on what was good and what lasted, and on the joy of helping to create the improvements that made life better. In ten years, I hope we can look back with the same mixed impression of progress combined with continuity.
Interview - Kris Moore - [email protected] | @pcbsdkris
News Roundup
Compressed zfs send / receive now in FreeBSDs vendor area
***
Granting privileges using the FreeBSD MAC framework
FreeBSDs kernel provides quite sophisticated privilege model that extends the traditional UNIX user-and-group one. Here Ill show how to leverage it to grant access to specific privileges to group of non-root users.
mac(9) allows creating pluggable modules with policies that can extend existing base system security definitions. struct mac_policy_ops consist of many entry points that we can use to amend the behaviour.
This time, I wanted to grant a privilege to change realtime priority to a selected group. While Linux kernel lets you specify a named group, FreeBSD doesnt have such ability, hence I created this very simple policy.
The privilege check can be extended using two user supplied functions: priv_check and priv_grant. The first one can be used to further restrict existing privileges, i.e. you can disallow some specific priv to be used in jails, etc. The second one is used to explicitly grant extra privileges not available for the target in base configuration.
The core of the mac_rtprio module is dead simple. I defined sysctl tree for two oids: enable (on/off switch for the policy) and gid (the GID target has to be member of), then I specified our custom version of mpo_priv_grant called rtprio_priv_grant. Body of my granting function is even simpler. If the policy is disabled or the privilege that is being checked is not PRIV_SCHED_RTPRIO, we simply skip and return EPERM. If the user is member of the designated group we return 0 thatll allow the action target would change realtime privileges.
***
The Story of the PING Program
Yes, it's true! I'm the author of ping for UNIX. Ping is a little thousand-line hack that I wrote in an evening which practically everyone seems to know about. :-)
I named it after the sound that a sonar makes, inspired by the whole principle of cho-location. In college I'd done a lot of modeling of sonar and radar systems, so the "Cyberspace" analogy seemed very apt. It's exactly the same paradigm applied to a new problem domain: ping uses timed IP/ICMP ECHO_REQUEST and ECHO_REPLY packets to probe the "distance" to the target machine.
My original impetus for writing PING for 4.2a BSD UNIX came from an offhand remark in July 1983 by Dr. Dave Mills while we were attending a DARPA meeting in Norway, in which he described some work that he had done on his "Fuzzball" LSI-11 systems to measure path latency using timed ICMP Echo packets.
In December of 1983 I encountered some odd behavior of the IP network at BRL. Recalling Dr. Mills' comments, I quickly coded up the PING program, which revolved around opening an ICMP style SOCK_RAW AF_INET Berkeley-style socket(). The code compiled just fine, but it didn't work -- there was no kernel support for raw ICMP sockets! Incensed, I coded up the kernel support and had everything working well before sunrise. Not surprisingly, Chuck Kennedy (aka "Kermit") had found and fixed the network hardware before I was able to launch my very first "ping" packet. But I've used it a few times since then. grin If I'd known then that it would be my most famous accomplishment in life, I might have worked on it another day or two and added some more options.
The folks at Berkeley eagerly took back my kernel modifications and the PING source code, and it's been a standard part of Berkeley UNIX ever since. Since it's free, it has been ported to many systems since then, including Microsoft Windows95 and WindowsNT.
In 1993, ten years after I wrote PING, the USENIX association presented me with a handsome scroll, pronouncing me a Joint recipient of The USENIX Association 1993 Lifetime Achievement Award presented to the Computer Systems Research Group, University of California at Berkeley 1979-1993. ``Presented to honor profound intellectual achievement and unparalleled service to our Community. At the behest of CSRG principals we hereby recognize the following individuals and organizations as CSRG participants, contributors and supporters.'' Wow!
The best ping story I've ever heard was told to me at a USENIX conference, where a network administrator with an intermittent Ethernet had linked the ping program to his vocoder program, in essence writing:
ping goodhost | sed -e 's/.*/ping/' | vocoder
He wired the vocoder's output into his office stereo and turned up the volume as loud as he could stand. The computer sat there shouting "Ping, ping, ping..." once a second, and he wandered through the building wiggling Ethernet connectors until the sound stopped. And that's how he found the intermittent failure.
FreeBSD: /usr/local/lib/libpkg.so.3: Undefined symbol "utimensat"
The internet will tell you that, of course, 10.2 is EOL, that packages are being built for 10.3 by now and to better upgrade to the latest version of FreeBSD. While all of this is true and running the latest versions is generally good advise, in most cases it is unfeasible to do an entire OS upgrade just to be able to install a package.
Now, if you have 10.2 installed and 10.3 is the current latest FreeBSD version, this url will point to packages built for 10.3 resulting in the problem that, when running pkg upgrade pkg itll go ahead and install the latest version of pkg build for 10.3 onto your 10.2 system. Yikes! FreeBSD 10.3 and pkgng broke the ABI by introducing new symbols, like utimensat.
Have a look at the actual repo url http://pkg.FreeBSD.org/FreeBSD:10:amd64 theres repos for each release! Instead of going through the tedious process of upgrading FreeBSD you just need to Use a repo url that fits your FreeBSD release:
Update the package cache: pkg update
Downgrade pkgng (in case you accidentally upgraded it already): pkg delete -f pkg
pkg install -y pkg
Install your package
There you go. Dont fret. But upgrade your OS soon ;)
Beastie Bits
***
Feedback/Questions
***
View all episodes
By JT Pennington
4.9
8989 ratings
This week, we look forward with the latest OpenBSD release, look back with Dennis Ritchies paper on the evolution of Unix Time Sharing, have an Interview with Kris
This episode was brought to you by
href="http://www.digitalocean.com/" title="DigitalOcean">
href="http://www.tarsnap.com/bsdnow" title="Tarsnap">
OpenBSD 6.1 RELEASED
***
Fuzz Testing OpenSSH
***
Getting OpenBSD running on Raspberry Pi 3
Ian Darwin writes in about his work deploying the arm64 platform and the Raspberry Pi 3
So I have this empty white birdhouse-like thing in the yard, open at the front. It was intended to house the wireless remote temperature sensor from a low-cost weather station, which had previously been mounted on a dark-colored wall of the house [...]. But when I put the sensor into the birdhouse, the signal is too weak for the weather station to receive it (the mounting post was put in place by a previous owner of our property, and is set deeply in concrete). So the next plan was to pop in a tiny OpenBSD computer with a uthum(4) temperature sensor and stream the temperature over WiFi.
The Raspberry Pi computers are interesting in their own way: intending to bring low-cost computing to everybody, they take shortcuts and omit things that you'd expect on a laptop or desktop. They aren't too bright on their own: there's very little smarts in the board compared to the "BIOS" and later firmwares on conventional systems. Some of the "smarts" are only available as binary files. This was part of the reason that our favorite OS never came to the Pi Party for the original rpi, and didn't quite arrive for the rpi2. With the rpi3, though, there is enough availability that our devs were able to make it boot. Some limitations remain, though: if you want to build your own full release, you have to install the dedicated raspberrypi-firmware package from the ports tree. And, the boot disks have to have several extra files on them - this is set up on the install sets, but you should be careful not to mess with these extra files until you know what you're doing!
But wait! Before you read on, please note that, as of April 1, 2017, this platform boots up but is not yet ready for prime time:
But wait - there's more! The "USB disk" can be a USB thumb drive, though they're generally slower than a "real" disk. My first forays used a Kingston DTSE9, the hardy little steel-cased version of the popular DataTraveler line. I was able to do the install, and boot it, once (when I captured the dmesg output shown below). After that, it failed - the boot process hung with the ever-unpopular "scanning usb for storage devices..." message. I tried the whole thing again with a second DTSE9, and with a 32GB plastic-cased DataTraveler. Same results. After considerable wasted time, I found a post on RPI's own site which dates back to the early days of the PI 3, in which they admit that they took shortcuts in developing the firmware, and it just can't be made to work with the Kingston DataTraveler! Not having any of the "approved" devices, and not living around the corner from a computer store, I switched to a Sabrent USB dock with a 320GB Western Digital disk, and it's been rock solid. Too big and energy-hungry for the final project, but enough to show that the rpi3 can be solid with the right (solid-state) disk. And fast enough to build a few simple ports - though a lot will not build yet. I then found and installed OpenBSD onto a PNY brand thumb drive and found it solid - in fact I populated it by dding from one of the DataTraveller drives, so theyre not at fault.
***
Dennis M. Ritchies Paper: The Evolution of the Unix Time Sharing System
This paper presents a brief history of the early development of the Unix operating system. It concentrates on the evolution of the file system, the process-control mechanism, and the idea of pipelined commands. Some attention is paid to social conditions during the development of the system.
During the past few years, the Unix operating system has come into wide use, so wide that its very name has become a trademark of Bell Laboratories. Its important characteristics have become known to many people. It has suffered much rewriting and tinkering since the first publication describing it in 1974 [1], but few fundamental changes. However, Unix was born in 1969 not 1974, and the account of its development makes a little-known and perhaps instructive story. This paper presents a technical and social history of the evolution of the system.
Origins
The PDP-7 Unix file system
Process control
IO Redirection
The advent of the PDP-11
The first PDP-11 system
Pipes
High-level languages
Conclusion
One of the comforting things about old memories is their tendency to take on a rosy glow. The programming environment provided by the early versions of Unix seems, when described here, to be extremely harsh and primitive. I am sure that if forced back to the PDP-7 I would find it intolerably limiting and lacking in conveniences. Nevertheless, it did not seem so at the time; the memory fixes on what was good and what lasted, and on the joy of helping to create the improvements that made life better. In ten years, I hope we can look back with the same mixed impression of progress combined with continuity.
Interview - Kris Moore - [email protected] | @pcbsdkris
News Roundup
Compressed zfs send / receive now in FreeBSDs vendor area
***
Granting privileges using the FreeBSD MAC framework
FreeBSDs kernel provides quite sophisticated privilege model that extends the traditional UNIX user-and-group one. Here Ill show how to leverage it to grant access to specific privileges to group of non-root users.
mac(9) allows creating pluggable modules with policies that can extend existing base system security definitions. struct mac_policy_ops consist of many entry points that we can use to amend the behaviour.
This time, I wanted to grant a privilege to change realtime priority to a selected group. While Linux kernel lets you specify a named group, FreeBSD doesnt have such ability, hence I created this very simple policy.
The privilege check can be extended using two user supplied functions: priv_check and priv_grant. The first one can be used to further restrict existing privileges, i.e. you can disallow some specific priv to be used in jails, etc. The second one is used to explicitly grant extra privileges not available for the target in base configuration.
The core of the mac_rtprio module is dead simple. I defined sysctl tree for two oids: enable (on/off switch for the policy) and gid (the GID target has to be member of), then I specified our custom version of mpo_priv_grant called rtprio_priv_grant. Body of my granting function is even simpler. If the policy is disabled or the privilege that is being checked is not PRIV_SCHED_RTPRIO, we simply skip and return EPERM. If the user is member of the designated group we return 0 thatll allow the action target would change realtime privileges.
***
The Story of the PING Program
Yes, it's true! I'm the author of ping for UNIX. Ping is a little thousand-line hack that I wrote in an evening which practically everyone seems to know about. :-)
I named it after the sound that a sonar makes, inspired by the whole principle of cho-location. In college I'd done a lot of modeling of sonar and radar systems, so the "Cyberspace" analogy seemed very apt. It's exactly the same paradigm applied to a new problem domain: ping uses timed IP/ICMP ECHO_REQUEST and ECHO_REPLY packets to probe the "distance" to the target machine.
My original impetus for writing PING for 4.2a BSD UNIX came from an offhand remark in July 1983 by Dr. Dave Mills while we were attending a DARPA meeting in Norway, in which he described some work that he had done on his "Fuzzball" LSI-11 systems to measure path latency using timed ICMP Echo packets.
In December of 1983 I encountered some odd behavior of the IP network at BRL. Recalling Dr. Mills' comments, I quickly coded up the PING program, which revolved around opening an ICMP style SOCK_RAW AF_INET Berkeley-style socket(). The code compiled just fine, but it didn't work -- there was no kernel support for raw ICMP sockets! Incensed, I coded up the kernel support and had everything working well before sunrise. Not surprisingly, Chuck Kennedy (aka "Kermit") had found and fixed the network hardware before I was able to launch my very first "ping" packet. But I've used it a few times since then. grin If I'd known then that it would be my most famous accomplishment in life, I might have worked on it another day or two and added some more options.
The folks at Berkeley eagerly took back my kernel modifications and the PING source code, and it's been a standard part of Berkeley UNIX ever since. Since it's free, it has been ported to many systems since then, including Microsoft Windows95 and WindowsNT.
In 1993, ten years after I wrote PING, the USENIX association presented me with a handsome scroll, pronouncing me a Joint recipient of The USENIX Association 1993 Lifetime Achievement Award presented to the Computer Systems Research Group, University of California at Berkeley 1979-1993. ``Presented to honor profound intellectual achievement and unparalleled service to our Community. At the behest of CSRG principals we hereby recognize the following individuals and organizations as CSRG participants, contributors and supporters.'' Wow!
The best ping story I've ever heard was told to me at a USENIX conference, where a network administrator with an intermittent Ethernet had linked the ping program to his vocoder program, in essence writing:
ping goodhost | sed -e 's/.*/ping/' | vocoder
He wired the vocoder's output into his office stereo and turned up the volume as loud as he could stand. The computer sat there shouting "Ping, ping, ping..." once a second, and he wandered through the building wiggling Ethernet connectors until the sound stopped. And that's how he found the intermittent failure.
FreeBSD: /usr/local/lib/libpkg.so.3: Undefined symbol "utimensat"
The internet will tell you that, of course, 10.2 is EOL, that packages are being built for 10.3 by now and to better upgrade to the latest version of FreeBSD. While all of this is true and running the latest versions is generally good advise, in most cases it is unfeasible to do an entire OS upgrade just to be able to install a package.
Now, if you have 10.2 installed and 10.3 is the current latest FreeBSD version, this url will point to packages built for 10.3 resulting in the problem that, when running pkg upgrade pkg itll go ahead and install the latest version of pkg build for 10.3 onto your 10.2 system. Yikes! FreeBSD 10.3 and pkgng broke the ABI by introducing new symbols, like utimensat.
Have a look at the actual repo url http://pkg.FreeBSD.org/FreeBSD:10:amd64 theres repos for each release! Instead of going through the tedious process of upgrading FreeBSD you just need to Use a repo url that fits your FreeBSD release:
Update the package cache: pkg update
Downgrade pkgng (in case you accidentally upgraded it already): pkg delete -f pkg
pkg install -y pkg
Install your package
There you go. Dont fret. But upgrade your OS soon ;)
Beastie Bits
***
Feedback/Questions
***
More shows like BSD Now
View all
Security Now (Audio)
1,971 Listeners
Software Engineering Radio - the podcast for professional software developers
272 Listeners
The Changelog: Software Development, Open Source
283 Listeners
LINUX Unplugged
265 Listeners
Python Bytes
215 Listeners
Late Night Linux
154 Listeners
Home Assistant Podcast
65 Listeners
CoRecursive: Coding Stories
189 Listeners
Kubernetes Podcast from Google
181 Listeners
Late Night Linux Family All Episodes
44 Listeners
Linux Dev Time
21 Listeners
Self-Hosted
135 Listeners
2.5 Admins
92 Listeners
Linux After Dark
29 Listeners
Oxide and Friends
47 Listeners