Hacker Public Radio by Hacker Public Radio

June 19, 2025 HPR4404: Kevie nerd snipes Ken by grepping xml

This show has been flagged as Explicit by the host.

More Command line fun: downloading a podcast

hpr4398 :: Command line fun: downloading a podcast

Kevie walked us through a command to download a podcast.

He used some techniques here that I hadn't used before, and it's always great to see how other people approach the problem.

Let's have a look at the script and walk through what it does, then we'll have a look at some "traps for young players" as the

EEVBlog

is fond of saying.

Analysis of the Script

wget `curl https://tuxjam.otherside.network/feed/podcast/ | grep -o 'https*://[^"]*ogg' | head -1`

It chains four different commands together to "Save the latest file from a feed".

Let's break it down so we can have checkpoints between each step.

I often do this when writing a complex one liner - first do it as steps, and then combine it.

The curl command gets

https://tuxjam.otherside.network/feed/podcast/

To do this ourselves we will call

curl https://tuxjam.otherside.network/feed/podcast/ --output tuxjam.xml

, as the default file name is index.html.

This gives us a xml file, and we can confirm it's valid xml with the

xmllint

command.

$ xmllint --format tuxjam.xml >/dev/null

$ echo $?

Here the output of the command is ignored by redirecting it to

/dev/null

Then we check the error code the last command had. As it's

it completed sucessfully.

Kevie then passes the output to the

grep

search command with the option

-o

and then looks for any string starting with https followed by anything then followed by two forward slashes, then

-o, --only-matching

Print only the matched (non-empty) parts of a matching line, with each such part on a separate output line

We can do the same with. I was not aware that grep defaulted to regex, as I tend to add the

--perl-regexp

to explicitly add it.

grep --only-matching 'https*://[^"]*ogg' tuxjam.xml

http matches the characters http literally (case sensitive)

s* matches the character s literally (case sensitive)

Quantifier: * Between zero and unlimited times, as many times as possible, giving back as needed [greedy]

: matches the character : literally

/ matches the character / literally

[^"]* match a single character not present in the list below

Quantifier: * Between zero and unlimited times, as many times as possible, giving back as needed [greedy]

" a single character in the list " literally (case sensitive)

ogg matches the characters ogg literally (case sensitive)

When we run this ourselves we get the following

$ grep --only-matching 'https*://[^"]*ogg' tuxjam.xml

https://archive.org/download/tuxjam-121/tuxjam_121.ogg

https://archive.org/download/tuxjam-120/TuxJam_120.ogg

https://archive.org/download/tux-jam-119/TuxJam_119.ogg

https://archive.org/download/tuxjam_118/tuxjam_118.ogg

https://archive.org/download/tux-jam-117-uncut/TuxJam_117.ogg

https://tuxjam.otherside.network/tuxjam-115-ogg

https://archive.org/download/tuxjam_116/tuxjam_116.ogg

https://tuxjam.otherside.network/tuxjam-115-ogg

https://ogg

http://tuxjam.otherside.network/wp-content/uploads/sites/5/2024/10/tuxjam_115_OggCamp2024.ogg

https://ogg

https://archive.org/download/tuxjam_114/tuxjam_114.ogg

https://archive.org/download/tuxjam_113/tuxjam_113.ogg

https://archive.org/download/tuxjam_112/tuxjam_112.ogg

The last command returns the first line, so therefore

https://archive.org/download/tuxjam-121/tuxjam_121.ogg

Finally that line is used as the input to the

wget

command.

Problems with the approach

Relying on grep with structured data like xml or json can lead to problems.

When we looked at the output of the command in step 2, some of the results gave

https://ogg

When run the same command without the

--only-matching

argument we see what was matched.

$ grep 'https*://[^"]*ogg' tuxjam.xml

This episode may not be live as in TuxJam 115 from Oggcamp but your friendly foursome of Al, Dave (thelovebug), Kevie and Andrew (mcnalu) are very much alive to treats of Free and Open Source Software and Creative Commons tunes.

https://tuxjam.otherside.network/tuxjam-115-oggcamp-2024/

https://tuxjam.otherside.network/tuxjam-115-oggcamp-2024/#respond

https://tuxjam.otherside.network/tuxjam-115-oggcamp-2024/feed/

With the group meeting up together for the first time in person, it was decided that a live recording would be an appropriate venture. With the quartet squashed around a table and a group of adoring fans crowded into a room at the Pendulum Hotel in Manchester, the discussion turns to TuxJam reviews that become regularly used applications, what we enjoyed about OggCamp 2024 and for the third section the gang put their reputation on the line and allow open questions from the sea of dedicated fans.

OggCamp 2024 on Saturday 12 and Sunday 13 October 2024, Manchester UK.

Two of the hits are not enclosures at all, they are references in the text to OggCamp

what we enjoyed about OggCamp 2024

Normally running

grep

will only get one entry per line, and if the xml is minimised it can miss entries on a file that comes across as one big line.

I did this myself using

xmllint --noblanks tuxjam.xml > tuxjam-min.xml

I then edited it and replaced the new lines with spaces. I have to say that the

--only-matching

argument is doing a great job at pulling out the matches.

That said the results were not perfect either.