Hacker Public Radio

HPR4201: Today I learnt (2024-08-23)


Listen Later

TIL 1:
Incrementing times with the date command
  • I have been working on an HPR project which is restoring external
  • files to shows which lost them when we migrated to the current static
    site.
  • As I make changes I want to be able to check that they are correct,
  • but to make this check I need to wait for the next update of the static
    site.
  • Ken was away recently, and set up a cron job to refresh
  • the site every three hours. Each show page shows the refresh time in UTC
    form in the header.
  • Being a bit numerically challenged I wanted a way of computing the
  • next refresh time in my timezone from the previous refresh time.
  • The GNU date command accepts a date and time expression
  • after the -d option (or using the
    alternative--date=STRING option). The contents of the
    STRING here are very flexible but quite complex since you
    can include time zone data, offsets, day and month names, etc. See the
    links below for links to the GNU manual.
  • My first attempt used the date command like this and
  • got the wrong answer (using the output format +%T which
    writes the time in a default form):
    $ date -d '16:27:16 + 3 hours' +%T
    15:27:16
    • It is not clear why this fails, but the GNU function which parses
    • these date parameters is obviously confused. The second try included the
      time zone after the time, and worked better, but is a little
      confusing:
      $ date -d '16:27:16 UTC + 3 hours' +%T
      20:27:16
      • The time returned is local time for me. The date
      • command has added three hours to the UTC date to get 19:27:16, but since
        I am in the UK, which is in DST (called BST - British Summer Time - UTC
        plus 1 hour), an hour is added.
      • The final try used the -u option which writes UTC
      • time:
        $ date -u -d '16:27:16 UTC + 3 hours' +%T
        19:27:16
        • I actually ended up using and re-using these commands (though a
        • script would have been better):
          $ current='06:27:55'
          $ next=$(date -u -d "${current}UTC + 3 hours 3 minutes" +%T); echo "$next UTC / $(date -d "${next} UTC" +'%T %Z')"
          09:30:55 UTC / 10:30:55 BST
          $ current=$next
          $ next=$(date -u -d "${current}UTC + 3 hours 3 minutes" +%T); echo "$next UTC / $(date -d "${next} UTC" +'%T %Z')"
          12:33:55 UTC / 13:33:55 BST
          $ current=$next
          $ next=$(date -u -d "${current}UTC + 3 hours 3 minutes" +%T); echo "$next UTC / $(date -d "${next} UTC" +'%T %Z')"
          15:36:55 UTC / 16:36:55 BST
          TIL 1: Links
          • GNU
          • Coreutils manual
            • System
            • context
              • date:
              • Print or set system date and time
                • Date
                • input formats (The quotation at the start of this section of the
                  manual is quite interesting!)
                • Date
                • examples
                  TIL 2: Merging lines of
                  files with paste
                  • While processing and "repairing" shows I came across the need to
                  • generate a list of show numbers separated by commas. In the past I have
                    loaded these into a Bash array and turned them into a comma-delimited
                    string, using the parameter substitution capabilities of Bash which can
                    add a comma to each element. The trouble with this is that it leaves a
                    trailing comma which has to be removed. I stumbled upon
                    paste as an alternative way of doing this.
                  • The GNU paste command is another from the GNU
                  • Coreutils group. This one merges lines of files. Its synopsis
                    is:
                    paste [OPTION]... [FILE]...
                    • It merges lines consisting of the corresponding lines from each file
                    • provided as an argument, by default separated by TABs, and writes them
                      to standard output. This means it produces lines consisting of the first
                      line from each of the files, separated by tabs, then the second lines,
                      and so on.
                    • Any of the files can be linked to standard in by using a
                    • - (hyphen) as the file name.
                    • The delimiters can be changed with the -d LIST or
                    • --delimiters=LIST option. The use of a list of delimiters
                      causes the characters in the list to be used sequentially for each
                      delimiter.
                    • The merged lines can be visualised as rows in a matrix, where each
                    • file provides a column.
                    • The "matrix" is rotated by using the -s or
                    • --serial option. Here the lines from one file at a time are
                      merged - the files are processed serially rather than in
                      parallel.
                    • The paste command can be used to generate the
                    • comma-delimited list I wanted by using the options -s and
                      -d ',':
                      $ printf '%s\n' {1..10} | paste -s -d, -
                      1,2,3,4,5,6,7,8,9,10
                      • Note that you can't get the same result with
                      • echo {1..10} because all the numbers will be written to one
                        line rather than being the separate lines that paste
                        requires.
                      • The file arguments to paste may also be Bash
                      • process substitution expressions:
                        $ paste -s -d, <(printf '%s\n' {1..10})
                        1,2,3,4,5,6,7,8,9,10
                        • This means you can generate more complex output by using multiple
                        • process substitution expressions where each is seen as a
                          file:
                          $ paste -d'|' <(printf '%d\n' {1..7}) <(printf '%s\n' {A..G}) <(printf '%d\n' {100..106})
                          1|A|100
                          2|B|101
                          3|C|102
                          4|D|103
                          5|E|104
                          6|F|105
                          7|G|106
                          • Note how using the -s option "rotates" this:
                          • $ paste -s -d'|' <(printf '%d\n' {1..7}) <(printf '%s\n' {A..G}) <(printf '%d\n' {100..106})
                            1|2|3|4|5|6|7
                            A|B|C|D|E|F|G
                            100|101|102|103|104|105|106
                            • This is what I was actually trying to do, so I could feed show
                            • numbers to another script which will accept a CSV list as an
                              option:
                              $ echo "select episode_id from repairs where repair_date is null order by episode_id desc limit 5" |\
                              sqlite3 -list ~/HPR/InternetArchive/ia.db | paste -s -d',' -
                              1959,1952,1951,1946,1941
                              TIL 2: Links
                              • GNU
                              • Coreutils manual
                                • Operating
                                • on fields
                                  • paste:
                                  • Merge lines of files
                                    ...more
                                    View all episodesView all episodes
                                    Download on the App Store

                                    Hacker Public RadioBy Hacker Public Radio

                                    • 4.2
                                    • 4.2
                                    • 4.2
                                    • 4.2
                                    • 4.2

                                    4.2

                                    34 ratings


                                    More shows like Hacker Public Radio

                                    View all
                                    The Infinite Monkey Cage by BBC Radio 4

                                    The Infinite Monkey Cage

                                    1,952 Listeners

                                    Click Here by Recorded Future News

                                    Click Here

                                    418 Listeners

                                    Hacker And The Fed by Chris Tarbell & Hector Monsegur

                                    Hacker And The Fed

                                    168 Listeners