𝕭𝖆𝖘𝖘𝖕𝖎𝖘𝖙𝖔𝖑 🎬

The billion row challenge: splitting lines using SIMD in Rust


Listen Later

I was scared to do it, but it was actually fine! This time we look at the quickest way to split up massive text files into separate lines, first by implementing our own zero-copy version of BufReader::lines and later using SIMD, which allows us to process 64 bytes simultaneously, using a single CPU instruction! I'm pretty sure this is the first time we've ventured into nightly Rust on this channel...

You can find the code at https://codeberg.org/andybalaam/changelog-builder

Read my blog at https://artificialworlds.net/blog

Follow me on mastodon: @[email protected]

...more
View all episodesView all episodes
Download on the App Store

𝕭𝖆𝖘𝖘𝖕𝖎𝖘𝖙𝖔𝖑 🎬By