Synopsis

In this episode, I describe my ongoing attempt to develop a static site generator for the HPR website. As a search through the community mailing list will reveal, the idea for moving the HPR website has been rattling around for a few years. I have been interested in helping out with the website for a while, so when the latest round of discussion occurred I decided to give it a whirl.

While I have read about various static site generators over the years, the only one I had any experience with was Template-Toolkit (TT2) when helping Ken Fallon and Dave Morriss develop the Free Culture Podcasts webpage. It is the tool Dave uses for various projects. Since they are the janitors for HPR, I figured a website generator based on the TT2 would be something relatively painless enough for them to use, and update when needed.

The code for this generator is hosted at https://repo.anhonesthost.net/rho_n/hpr_generator. The generator is written in Perl and generates the pages using TT2 templates.

Anyone interested in helping out is encouraged to create an account on repo.anhonesthost.net and start hacking away.

References

site-generator source code; https://repo.anhonesthost.net/rho_n/hpr_generator

Template Tookit website; http://template-toolkit.org/

Comprehensive Perl Archive Network (CPAN) website; https://www.cpan.org/

HPR repositories at anHonestHost.net; https://repo.anhonesthost.net/explore/repos

Rho`n's public test HPR website; https://hpr.horning.us

Free Culture Podcasts; https://freeculturepodcasts.org/

HPR Mailing list: Static Site Generators - NOT a flat file CMS; message thread; https://hackerpublicradio.org/pipermail/hpr_hackerpublicradio.org/2019-April/014388.html

HPR Mailing list: Source Code for the HPR website.; message thread; https://hackerpublicradio.org/pipermail/hpr_hackerpublicradio.org/2022-June/015296.html

MySQL to SQLite conversion script; https://github.com/dumblob/mysql2sqlite

In this episode our two ageing heroes explore the inner workings of a podcast (or podcast hosting platform depending on your perspective) called Hacker Public Radio. Yes, the platform that the Inlaws have been using since the very inception of this rapidly growing FLOSS podcast content. Wondering what the heck this episode is all about, why exactly Martin and Chris are talking about this now and the importance of statistics, lies and damned lies? Then just listen to this episode. You may also find out the difference between mere caching and content syndication. Never mind HPR's inner workings.

Links:

hpr3648: https://hackerpublicradio.org/eps.php?id=3648

The Internet Archive: https://archive.org

Extraterrestrial sheep: https://aliens.fandom.com/wiki/Sheep_(Kaeloo)

QuickAxe: https://fireemblem.fandom.com/wiki/Quick_Axe

Linux Episode S01E35 (The Free Software Foundation Europe): https://hackerpublicradio.org/eps.php?id=3388

Ford v Ferrari: https://www.imdb.com/title/tt1950186/?ref_=nv_sr_srsg_0

Book of Monsters: https://www.imdb.com/title/tt7260818/?ref_=fn_al_tt_1

Creating Backups.

This script was trimmed to serve as an example.

The three options shown (email, jop, dots) demonstrates, how to list items with case statements:

Single item/directory (jop).

Multiple items in single directory (dots).

Multiple items in multiple directories (email).

The text files created after the archive serves as an item list

with current permissions.

tar --directory= /path/to/directory/ --create --file INSERT_ARCHIVE_NAME.tar /path/to/file;

#!/bin/bash

#License: GPL v3

# This program is free software: you can redistribute it and/or modify

# it under the terms of the GNU General Public License as published by

# the Free Software Foundation, either version 3 of the License, or

# (at your option) any later version.

#

# This program is distributed in the hope that it will be useful,

# but WITHOUT ANY WARRANTY; without even the implied warranty of

# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the

# GNU General Public License for more details.

#

# You should have received a copy of the GNU General Public License

# along with this program. If not, see <https://www.gnu.org/licenses/>.

#Name: getoverhere.sh

#Purpose:

#Version: beta 0.07

#Author: SGOTI (Some Guy On The Internet)

#Date: Sat 29 Jan 2022 02:19:29 AM EST

#variables:

VAR_TBALL=

VAR_TARGET=

VAR_JUMP=

VAR_VALUE=

#start:

cat << "EOT01"

Options:

email |"${HOME}/.thunderbird/"

jop |"${HOME}/Documents/joplin"

dots |"${HOME}/.bashrc .vimrc .bash_aliases"

EOT01

echo -e "What do you want to backup? : c."

read VAR_VALUE

case ${VAR_VALUE} in

"email" )

VAR_TBALL="INSERT_EMAIL_NAME$(date +%m-%d-%Y).tar.gz"

VAR_TARGET="msgFilterRules.dat"

VAR_JUMP="${HOME}/.thunderbird/*.default-release/ImapMail/imap.mail.yahoo.com/"

echo -e "Grabbing INSERT_EMAIL_NAME...n"

tar -C ${VAR_JUMP} --create --file ${VAR_TBALL} --gzip ${VAR_TARGET}

sleep 1

VAR_TBALL="INSERT_EMAIL_NAME$(date +%m-%d-%Y).tar.gz"

VAR_TARGET="msgFilterRules.dat"

VAR_JUMP="${HOME}/.thunderbird/*.default-release/ImapMail/imap.gmail.com/"

echo -e "Grabbing INSERT_EMAIL_NAME...n"

tar -C ${VAR_JUMP} --create --file ${VAR_TBALL} --gzip ${VAR_TARGET}

sleep 1

VAR_TBALL="EMAIL_ARCHIVES$(date +%m-%d-%Y).tar.gz"

VAR_TARGET="Mail/"

VAR_JUMP="${HOME}/.thunderbird/*.default-release/"

echo -e "Grabbing email EMAIL_ARCHIVES...n"

tar -C ${VAR_JUMP} --create --file ${VAR_TBALL} --gzip ${VAR_TARGET}

echo -e "Creating List for ${VAR_TBALL}...n"

ls -lhAR --group-directories-first ${VAR_JUMP}${VAR_TARGET} > EMAIL_ARCHIVES$(date +%m-%d-%Y).txt

sleep 1

VAR_TBALL="THUNDERBIRD_CALENDER$(date +%m-%d-%Y).tar.gz"

VAR_TARGET="calenders/"

VAR_JUMP="${HOME}/Documents/"

echo -e "Grabbing email THUNDERBIRD_CALENDER...n"

tar -C ${VAR_JUMP} --create --file ${VAR_TBALL} --gzip ${VAR_TARGET}

echo -e "Creating List for ${VAR_TBALL}...n"

ls -lhAR --group-directories-first ${VAR_JUMP}${VAR_TARGET} > THUNDERBIRD_CALENDER$(date +%m-%d-%Y).txt

sleep 1

VAR_TBALL="THUNDERBIRD_ADDRESS_BOOK$(date +%m-%d-%Y).tar.gz"

VAR_TARGET="address-book/"

VAR_JUMP="${HOME}/Documents/"

echo -e "Grabbing ${VAR_TARGET}...n"

tar -C ${VAR_JUMP} --create --file ${VAR_TBALL} --gzip ${VAR_TARGET}

echo -e "Creating List for ${VAR_TBALL}...n"

ls -lhAR --group-directories-first ${VAR_JUMP}${VAR_TARGET} > THUNDERBIRD_ADDRESS_BOOK$(date +%m-%d-%Y).txt

sleep 1

VAR_TBALL="THUNDERBIRD_ALL$(date +%m-%d-%Y).tar.gz"

VAR_TARGET=".thunderbird/"

VAR_JUMP="${HOME}/"

echo -e "Grabbing ${VAR_TARG

Importance of Small toy projects

Inspired by

https://talkpython.fm/episodes/transcript/327/little-automation-tools-in-python

Automating little things

hpr3558 :: How I'm learning Haskell

The Host talks about learning Haskell and says something like

Finishing a small project is better than Starting a big project and not completing it.

And You can not learn to code by reading, you need practice

My recommendation for learning a new language.

Think about a small problem you want to solve with code.

Start banging away at it.

Redo over and over, its OK

Same project but better

Learn new tech and practice

An example of one of my small projects

Original intent of my podfaded project

Mostly Tech/Linux podcasts

Find every podcast I could

Track release cadence

Rate how podfaded the feed is

Use search API to find RSS feeds

First attempt plan

Scrape podcast networks

used beautiful soup - something I wanted to learn anyway

HTML

Jinja templating - something I wanted to learn anyway

Bootstrap - something I wanted to learn anyway

Different colors based on how podfaded

First attempt problems

individual script per network

4-5 different scripts

took forever to run

Broke when networks changed their page layout

Redo

Test Driven Development and pytest

Red Green Refactor

Confidence to change code

Simplify - one scrapper script

Embed audio player for latest episode

How it works - Scraping pages for feeds

scrape_for_feeds.py

Provide a list of websites that have lists of podcasts

Search the page for any links

Check if the feed is valid by trying to parse it with feedparser

Add feed to database

Podcasts pages I scrape

https://wiki.ubuntu.com/Podcasts

http://www.thelinuxlink.net

http://freeculturepodcasts.org

How it works - Checking the feeds

feed_info.py

loop through feeds

use feedparser to find latest episode

title and enclosure (audio file)

HTML Output

https://podfaded.norrist.xyz/

Currently hosted on a Free Tier VPS

No Promises it will be there forever

Feedback Welcome

https://gitlab.com/norrist/podfaded2

Bugs

Sometimes the title is missing, so some of the CSS coloring doesn't work

Excluded podcasts still occasionally show up on the list

Looking for more curated lists of podcast pages

not search sites

Maybe scrape HPR podcast recommendation episodes

Example

UNIX, Linux, and BSD

Linux was created by PC users attempting to use mainframe UNIX. BSD was created by mainframe UNIX users attempting to use a PC.

BSD is what I like to call a “Pedigree UNIX”, meaning that it is a pure blooded descendant of AT&T UNIX. Although all of the original AT&T code has been re-written so a permissive license, the heritage persists.

In contrast to Linux (which shares no original Bell Labs code), BSD was originally all Labs code.

BSD in the wild

BSD style licensing is quite simple to understand compared to the tome that is the GPL. Interpreting it usually goes something like follows:

Do whatever the hell you want with this code, just don’t blame me when it breaks something and don’t claim you wrote it

Permissive licensing means that various companies can put lipstick on the UNIX pig and falsely assert that it’s anything other than lipstick on the UNIX pig. Not that UNIX is a pig, but you cannot disguise a pig with lipstick. Those burdened with the gift of sight and knowledge can spot a UNIX system quite easily.

Apple software is basically stolen BSD

Windows TCP/IP stack (and ftp/rcp/rsh/ssh/scp and other various non-trash networking protocols) is basically stolen BSD

Sony PlayStation is basically stolen BSD

Nintendo switch is basically stolen BSD

a million others that I can’t be bothered to list because they’re either abandonware or are embedded in your e-toaster and internet enabled dishwasher so no one cares

Interacting with BSD guys

Most Linux enthusiasts are missionaries. They are generally helpful and seek to guide the computing neophyte into the inner circle of FSF initiates.

The BSD guys tend to be like hermits. They don’t care if you use their code, they only care that the code works for them. When you ask for help, a typical response will be “did you even read the error logs?” or “did you even RTFM? What about supplemental documentation? We didn’t write TFM just so you could go online and ask something clearly documented in TFM.”

Not all BSD guys are bitter, but you really should consult available resources before asking questions

Forking vs distros

In Linux land, all the distros are basically the same with varying coats of paint. We call these distributions because all “implementations” of linux are nearly identical code bases built with varying compile time options.

In BSD land, distros don’t exist. Free/Net/Open are entirely independent and don’t share a common upstream. They are forks of primordial BSD that run separate kernels, separate userlands, etc. Although code is shared amongst each other, a statically linked binary can’t simply be dumped from one to another and still run as it would in Linux land.

Meta-distos of FreeBSD do exist but they are short lived unless they have corporate backers.

Idiot’s guide to picking a BSD

I want basically Linux desktop out of the box but with a BSD kernel so I can look cool when I post a neofetch screenshot to the /g/ desktop thread!!

Selecting a FreeBSD fork that comes with a desktop is your goto. The currently maintained desktop distros are HelloSystems, GhostBSD, NomadBSD, and MidnightBSD.

I want a viable desktop operating system

FreeBSD with a non-GNOME DE is fairly reliable. I’ve had success with KDE, XFCE, and various tiling window managers. GNOME is too reliant on systemd so the port is janky.

I want something to learn by example with

OpenBSD is a great learning platform. The source code for userland utils is simple, short, and generally free from OS specific function calls. RTFM goes by the wayside when you easily

Make a website using the HPR RSS Feed

Welcome to part II of my experiment in how much can be done with the data ion the HPR feed

In my previous episode https://hackerpublicradio.org/eps.php?id=3637 I discussed taking the data from the HPR feed and stuffing the useful bits into a sqlite database.

Today, Ill discuss some of my adventures generating a static site using the feed data

Lessons learned

Maybe skip DB step

I tend to over use Markdown - sometimes HTML is Better

So much content on HPR

Tech

My original intent was to create markdown files that could be feed into an existing static site generator, something like Hugo or Jekyll.

I started with markdown but had to add too much html.

I'm am a bit fan of using Markdown, but it did not fit this particular use case

Peewee - ORM - https://docs.peewee-orm.com

Jinja - Templates - https://jinja.palletsprojects.com

Feed as source of data

I added some HTML for header footer

Header

bootstrap CSS columns

hpr logo

footer

Links

Copyright

Pages

Main page listing the most recent shows

A page listing all shows

Individual pages for each show

A page listing all the hosts

individual page for each host

One thing I would like to see on the HPR page is a count of how many shows there are for each host. I have enough data, so I added show counts.

Missing data

host profiles - web page, avatars

tags

series

Summary - The short 100 character summary of what your show is about

License

Other pages on the HPR site.

What you need to know

How to help out

Requested topics

...

How does it work

Peewee to read from sqlite file

Python aggregates the appropriate data

Jinja Templates for

index - passed data from most recent shows

All shows

host page

corespondents - all host

TODO

Incorporate Comment feed

Generate static copy of RSS feed.

Copy content from HPR pages that are not in the RSS feed.

Generate Tags from Keywords in the show notes.

Only use the Full Feed on the first run.

Links

Project - https://gitlab.com/norrist/hprfeed2db - Suggestions/Pull Requests are welcome

Example site - https://hpr.norrist.xyz/

Long time listener, first time caller...

https://en.wikipedia.org/wiki/Registered_memory

https://www.futureplus.com/ddr4-3ds-dimms-the-next-big-thing-in-the-data-center/

table td.shrink {

white-space:nowrap

}

New hosts

Welcome to our new hosts:

binrc,

Celeste.

Last Month's Shows

Id

Day

Date

Title

Host

3630

Fri

2022-07-01

Planning an RV Trip

Ahuka

3631

Mon

2022-07-04

HPR Community News for June 2022

HPR Volunteers

3632

Tue

2022-07-05

Intro to web scraping with Python

Klaatu

3633

Wed

2022-07-06

The collective history of RAID controller brands

JWP

3634

Thu

2022-07-07

Everything You Always Wanted to Know About PEX Part 05 - PEX and the Single Installer

Trey

3635

Fri

2022-07-08

A short podcast on a nice tool called system-monitoring-center

Jeroen Baten

3636

Mon

2022-07-11

The Importance of Data Reduction

Lurking Prion

3637

Tue

2022-07-12

HPR feed to Sqlite

norrist

3638

Wed

2022-07-13

Ken drops a bear on his android phone

We continue our look at the planning process by looking at how to build a plan around major destinations, and add memberships that give discounted stays like Escapees and Passport America.

Links:

https://www.escapees.com/benefits/rv-parking/discount-park-directory/maps/

https://passportamerica.com/

https://www.palain.com/travel/major-destinations/