Little Pi's Tech Music Tips

Is AI composing just a word chain game? Little Pi introduces autoregressive generation: like a phone keyboard guessing your next word, the model looks at all previous audio tokens, guesses the next one, adds it, and guesses again. The Transformer doing the guessing uses attention to look back at earlier rhythms and themes; OpenAI's Jukebox (2020) took this path. Meta's MusicGen, open-sourced in 2023, uses a 32 kHz EnCodec with four codebooks at 50 frames per second, and its delay pattern offsets the codebooks by one step so they can be predicted together, so each second of music takes only 50 steps. It reads text with T5, the melody version can follow a tune you hum, and it was trained on 20,000 hours of licensed music in 300M, 1.5B and 3.3B parameter sizes.

References:

Simple and Controllable Music Generation (MusicGen) - arXiv: https://arxiv.org/abs/2306.05284

AudioCraft MusicGen docs - GitHub: https://github.com/facebookresearch/audiocraft/blob/main/docs/MUSICGEN.md

Jukebox (OpenAI): https://openai.com/index/jukebox/

Autoregressive model - Wikipedia: https://en.wikipedia.org/wiki/Autoregressive_model

Transformer (deep learning architecture) - Wikipedia: https://en.wikipedia.org/wiki/Transformer_(deep_learning_architecture)

Before AI can write a song, it has to learn to read sound! Little Pi introduces neural audio codecs: one second of audio has 24,000 numbers, far too many and too fine, so an encoder squeezes sound into codes and a decoder turns them back into sound. Examples include Google's SoundStream (2021) and Meta's EnCodec (2022, as low as 1.5 kbps). Where do the codes come from? RVQ (residual vector quantization) stacks several codebooks: the first captures the outline, the next fills in the error, layer by layer. X-Codec (2024) adds HuBERT semantic features before the codebooks so the tokens are easier for AI to understand, and the open-source song model YuE uses a compact version of it. These audio tokens are the foundation of music generation models like MusicGen and YuE.

References:

EnCodec (Meta AI) - GitHub: https://github.com/facebookresearch/encodec

High Fidelity Neural Audio Compression (EnCodec) - arXiv: https://arxiv.org/abs/2210.13438

SoundStream: An End-to-End Neural Audio Codec - arXiv: https://arxiv.org/abs/2107.03312

Codec Does Matter (X-Codec) - arXiv: https://arxiv.org/abs/2408.17175

X-Codec - GitHub: https://github.com/zhenye234/xcodec

YuE - GitHub: https://github.com/multimodal-art-projection/YuE

Can a finished mix be taken apart? Little Pi explains source separation, starting from the cocktail party problem described by Colin Cherry in 1953. The most common approach turns sound into a spectrogram, lets AI decide which source each time-frequency square belongs to, and applies a mask to keep only the vocals. The AI learns from datasets like MUSDB18 (150 songs split into vocals, drums, bass and other). Two famous tools: Spleeter, open-sourced by Deezer in 2019, and Meta's Demucs, whose fourth version in 2022 added a Transformer. The most touching use: the Beatles' 2023 song Now and Then, where AI separated John Lennon's voice from a late-1970s cassette demo.

References:

Music Source Separation - Wikipedia: https://en.wikipedia.org/wiki/Music_Source_Separation

Cocktail party effect - Wikipedia: https://en.wikipedia.org/wiki/Cocktail_party_effect

Demucs (Meta AI) - GitHub: https://github.com/facebookresearch/demucs

Spleeter (Deezer) - GitHub: https://github.com/deezer/spleeter

MUSDB18 dataset - SigSep: https://sigsep.github.io/datasets/musdb.html

Now and Then (Beatles song) - Wikipedia: https://en.wikipedia.org/wiki/Now_and_Then_(Beatles_song)

Whether an interval sounds smooth can actually be calculated! Little Pi introduces the sensory dissonance curve: two close frequencies create beats, and faster beats turn into roughness, which Helmholtz proposed in 1863 as the main source of dissonance. In 1965, Plomp and Levelt's listening experiments found that two pure tones sound roughest at about a quarter of a critical band. Add up the roughness between all the overtones of two notes, and the curve's valleys land on the octave, fifth, fourth and major third. William Sethares (1993 paper; 1998 book Tuning, Timbre, Spectrum, Scale) showed that for sounds with uneven overtones, like xylophones, bells and gamelan, the valleys move: timbre and scale are partners, so you can design the timbre first and then the scale.

References:

William Sethares - Relating Tuning and Timbre: https://sethares.engr.wisc.edu/consemi.html

Consonance and dissonance - Wikipedia: https://en.wikipedia.org/wiki/Consonance_and_dissonance

Beat (acoustics) - Wikipedia: https://en.wikipedia.org/wiki/Beat_(acoustics)

Critical band - Wikipedia: https://en.wikipedia.org/wiki/Critical_band

Hermann von Helmholtz - Wikipedia: https://en.wikipedia.org/wiki/Hermann_von_Helmholtz

Can trains and pots and pans become music? Little Pi introduces Pierre Schaeffer, the father of musique concrète: born in Nancy, France in 1910 and trained as an engineer, he founded the Studio d'Essai at French radio in 1942. On October 5, 1948, the radio broadcast a "concert of noises" premiering his Cinq études de bruits, including the Railway Study. He cut closed grooves into shellac records to loop sounds forever, the ancestor of today's loop. In 1950 he and Pierre Henry created Symphonie pour un homme seul. He founded a musique concrète research group in 1951, reorganized as the GRM in 1958, and introduced the ideas of the sound object and acousmatic listening (hearing a sound without seeing its source). His 1966 Traité des objets musicaux summed up twenty years of research. His students included Éliane Radigue and Jean-Michel Jarre.

References:

Pierre Schaeffer - Wikipedia: https://en.wikipedia.org/wiki/Pierre_Schaeffer

Musique concrète - Wikipedia: https://en.wikipedia.org/wiki/Musique_concr%C3%A8te

Pierre Henry - Wikipedia: https://en.wikipedia.org/wiki/Pierre_Henry

Groupe de Recherches Musicales (GRM) - Wikipedia: https://en.wikipedia.org/wiki/Groupe_de_Recherches_Musicales

Acousmatic sound - Wikipedia: https://en.wikipedia.org/wiki/Acousmatic_sound

That robot singing in electronic songs is really a human voice controlling a synthesizer! Little Pi introduces the vocoder: Homer Dudley at Bell Labs patented it in 1938 to compress speech and save telephone bandwidth. It splits the voice into frequency bands and uses each band's loudness to control a synthesizer. In World War Two it became the heart of SIGSALY, the Allies' encrypted phone system (in use from 1943, weighing over 50 tonnes). In 1970 Wendy Carlos and Robert Moog built a ten-band vocoder for the 1971 film A Clockwork Orange, and Kraftwerk's Autobahn (1974) carried it into pop, dance music and hip-hop. Plus: how a vocoder differs from a talk box and Auto-Tune. Try it yourself with EVOC 20 PolySynth in Logic Pro.

References:

Vocoder - Wikipedia: https://en.wikipedia.org/wiki/Vocoder

Homer Dudley - Wikipedia: https://en.wikipedia.org/wiki/Homer_Dudley

SIGSALY - Wikipedia: https://en.wikipedia.org/wiki/SIGSALY

Wendy Carlos - Wikipedia: https://en.wikipedia.org/wiki/Wendy_Carlos

Robotic voice effects - Wikipedia: https://en.wikipedia.org/wiki/Robotic_voice_effects

In 1968, an album of Bach played on a synthesizer won three Grammy Awards! Little Pi introduces Wendy Carlos, who studied physics and music at Brown University and earned a master's in composition from Columbia in 1965. She met Robert Moog in 1964 and convinced him to add a touch-sensitive keyboard. Because the synthesizer could only play one note at a time, Switched-On Bach was built up layer by layer on an eight-track recorder over about 1,100 hours, and it topped the classical albums chart for three years. She went on to score A Clockwork Orange, The Shining and Tron, divided the octave into 35 notes on Beauty in the Beast in 1986, and received the SEAMUS Lifetime Achievement Award in 2005.

References:

Wendy Carlos - Wikipedia: https://en.wikipedia.org/wiki/Wendy_Carlos

Switched-On Bach - Wikipedia: https://en.wikipedia.org/wiki/Switched-On_Bach

Moog synthesizer - Wikipedia: https://en.wikipedia.org/wiki/Moog_synthesizer

Beauty in the Beast (album) - Wikipedia: https://en.wikipedia.org/wiki/Beauty_in_the_Beast_(album)

There's a magical box in the recording studio that pulls the volume down whenever a sound gets too loud! In this audio tip, Little Pi introduces the compressor: it squeezes dynamic range, which is completely different from MP3 file compression. Learn about threshold and ratio (like 4:1), attack and release, and makeup gain, and how a ratio of 10:1 or more turns it into a limiter. Find out how sidechain and ducking make background music drop the moment a host starts talking, and meet some legendary machines: the 1959 Fairchild 660 tube compressor designed by Rein Narma, twelve of which were bought by Abbey Road Studios and shaped the drum sound of the Beatles' Revolver and Sgt. Pepper, plus the 1176 and LA-2A from Bill Putnam's company.

References:

Dynamic range compression - Wikipedia: https://en.wikipedia.org/wiki/Dynamic_range_compression

Fairchild 660 - Wikipedia: https://en.wikipedia.org/wiki/Fairchild_660

Universal Audio (company) - Wikipedia: https://en.wikipedia.org/wiki/Universal_Audio_(company)

Loudness war - Wikipedia: https://en.wikipedia.org/wiki/Loudness_war

At the 1958 Brussels World's Fair, there was a building with hundreds of loudspeakers hidden in its walls, and it sang when you walked in! In this episode, Little Pi introduces the Philips Pavilion: commissioned by the electronics company Philips from architect Le Corbusier, who wanted not a showroom but an "electronic poem"; designed by Iannis Xenakis, also a composer, from nine hyperbolic paraboloids; and filled with Edgard Varèse's Poème électronique. Varèse, who championed "organized sound" and wrote the all-percussion Ionisation in 1931, created the roughly eight-minute piece from September 1957 to April 1958. Played from three tape tracks, it was routed with telephone dials through three to four hundred wall-mounted speakers, moving around the audience: one of the earliest immersive sound systems. The pavilion was demolished on January 30, 1959, and was later rebuilt virtually in an EU-funded project.

References:

Philips Pavilion - Wikipedia: https://en.wikipedia.org/wiki/Philips_Pavilion

Poème électronique - Wikipedia: https://en.wikipedia.org/wiki/Po%C3%A8me_%C3%A9lectronique

Edgard Varèse - Wikipedia: https://en.wikipedia.org/wiki/Edgard_Var%C3%A8se

Iannis Xenakis - Wikipedia: https://en.wikipedia.org/wiki/Iannis_Xenakis

Expo 58 - Wikipedia: https://en.wikipedia.org/wiki/Expo_58

You can add a filter to a photo, and you can retouch sound, too! In this audio tip, Little Pi introduces the equalizer (EQ): the roughly 20 Hz to 20 kHz range of human hearing and what the lows, mids and highs each do; how the name "equalize" comes from restoring the high frequencies lost on telephone lines; RCA engineer John Volkman using equalization in a movie theater sound system in the 1920s; high-pass and low-pass filters, shelf and bell filters, and graphic versus parametric EQ, with Daniel Flickinger introducing the first parametric equalizer in 1971 and George Massenburg naming parametric equalization at the AES convention in 1972; plus Pultec, founded in 1953, and the famous "Pultec trick" on its classic 1961 EQP-1A tube equalizer. And a mixing tip: cut before you boost!

References:

Equalization (audio) - Wikipedia: https://en.wikipedia.org/wiki/Equalization_(audio)

Pultec - Wikipedia: https://en.wikipedia.org/wiki/Pultec

George Massenburg - Wikipedia: https://en.wikipedia.org/wiki/George_Massenburg

Audio frequency - Wikipedia: https://en.wikipedia.org/wiki/Audio_frequency