Developer Voices

Ask any software developer about their favourite stack, and they'll probably have strong opinions. (They might even have a snappy acronym for it, like LAMP or JAM or...) But if you ask any business about their stack, things aren’t so clear. They'll probably tell you the key components, and then add a laundry list of extra parts that got glued on later. The reality is, assembling large systems gets complicated. And with more and more options coming out every year, that potential complexity is only growing. These days, a good software developer must also be part researcher, and part explorer. 

So this week we're going to survey the landscape, and head out on the hunt for workable architectures. And joining us to do it is Ben Gamble. He's worked in fields as diverse as video games, enterprise software and AR headsets, and these days works for an Everything As A Service company helping people find and assemble the missing pieces their system needs. Who better to guide us through the software landscape? 

If you've ever worried that your system is too complicated, or is missing important parts, or you're just yearning for an architecture that feels more cohesive and less like a box of parts, come exploring with us.

--

Kris on Twitter: https://twitter.com/krisajenkins
Kris on LinkedIn: https://www.linkedin.com/in/krisjenkins/
Ben on Twitter: https://twitter.com/BenGamble7
The Schemaverse: https://schemaverse.com/
Nerdsniping: https://xkcd.com/356/
Tremor: https://www.tremor.rs/
Anna McDonald’s talk on Completion Criteria: https://www.confluent.io/events/kafka-summit-london-2023/pragmatic-patterns-and-pitfalls-for-event-streaming-in-brownfield/
Temporal.IO: https://temporal.io/

In modern systems, the amount of data keeps getting larger, and the time available keeps getting shorter. So it's almost inevitable that we're augmenting our general-purpose databases with dedicated analytics databases.

This week we dive into the world of OLAP with a thorough look at Clickhouse, a high-performance, columnar database designed to "query billions of rows in microseconds."

Alasdair Brown joins us to discuss what Clickhouse is, how it performs queries so quickly, and where it fits into a wider system. We talk about its origins as a Google Analytics-like, and how it's grown into one of the most popular OLAP databases around.

There's a lot of ground to cover, and a lot of questions to ask, all in the service of faster answers...

--

Alasdair's Blog: alasdairb.com
Alasdair on Threads: https://www.threads.net/@sdairsab
Alasdair on LinkedIn https://www.linkedin.com/in/alasdair-brown
Kris on Twitter: https://twitter.com/krisajenkins
Kris on LinkedIn: https://www.linkedin.com/in/krisjenkins/
Clickhouse: https://clickhouse.com/
Tinybird: https://www.tinybird.co/
Birdhouse in your Soul: https://youtu.be/vn_or9gEB6g

Neil Buesing is a seasoned Apache Kafka® user, and a respected voice from the Kafka community who specialises in helping companies make the best use of Kafka. And that makes him the ideal person to ask the $64,000 question: What problems can Kafka actually solve for me? Because Kafka's definitely interesting, and it can be fun, but to earn a place in the toolbox it has to make life easier. 

In answering that question, Neil covers a tonne of ground, from queuing and quasi-databases, transitioning from batch to real-time, and solving general software integration headaches. 

If you have data problems, big or small, join us to figure out if Kafka is the answer. 

-- 

Kinetic Edge: https://www.kineticedge.io
Neil on Twitter: https://twitter.com/nbuesing
Neil on LinkedIn: https://www.linkedin.com/in/nbuesing/
Kris on Twitter: https://twitter.com/krisajenkins
Kris on LinkedIn: https://www.linkedin.com/in/krisjenkins/

Building an Enterprise Eventing Framework (Conference Talk): https://www.confluent.io/en-gb/kafka-summit-san-francisco-2019/building-an-enterprise-eventing-framework/
Synchronous Kafka (Conference Talk): https://www.confluent.io/resources/kafka-summit-2020/synchronous-commands-over-apache-kafka/
Dev Local: https://github.com/kineticedge/dev-local
KIP-714 - Client metrics and observability: https://cwiki.apache.org/confluence/display/KAFKA/KIP-714%3A+Client+metrics+and+observability

How do distributed systems work? If you’ve got a database spread over three servers, how do they elect a leader? How does that change when we spread those machines out across data centers, situated around the globe? Do we even need to understand how it works, or can we relegate those problems to an off the shelf tool like Zookeeper?

Joining me this week is Distributed Systems Doctor—Benjamin Bengfort—for a deep dive into consensus algorithms. We start off by discussing how much of “the clustering problem” is your problem, and how much can be handled by a library. We go through many of the constraints and tradeoffs that you need to understand either way. And we eventually reach Benjamin’s surprising message - maybe the time is ripe to roll your own. Should we be writing our own bespoke Raft implementations? And if so, how hard would that be? What guidance can he offer us? 

Somewhere in the recording of this episode, I decided I want to sit down and try to implement a leader election protocol. Maybe you will too. And if not, you’ll at least have a better appreciation for what it takes. Distributed systems used to be rocket science, but they’re becoming deployment as usual. This episode should help us all to keep up!

--

KubeCon talk on the FCD bug: https://kccncna2022.sched.com/event/182N9/lessons-learned-from-etcd-the-data-inconsistency-issues-marek-siarkowicz-google-benjamin-wang-vmware
The Raft paper by Diego Ongaro and John Ousterhout: https://raft.github.io/raft.pdf
The EPaxos Algorithm: https://www.cs.cmu.edu/~dga/papers/epaxos-sosp2013.pdf
LevelDB: https://github.com/google/leveldb

Benjamin on Twitter: https://twitter.com/bbengfort
Benjamin on LinkedIn: https://www.linkedin.com/in/bbengfort
Benjamin on GitHub: https://github.com/bbengfort
Rotational Labs: https://rotational.io (check out the blog!)
Kris on Twitter: https://twitter.com/krisajenkins
Kris on LinkedIn: https://www.linkedin.com/in/krisjenkins/

How do you get started as a programmer? And how do experienced programmers help them as they start their journey?

This week's guest is a developer-turned-teacher, James Q. Quick. A former coder and developer advocate, he's s been been working on tutorials, courses and bootcamps to teach aspiring developers how to get started with JavaScript. We talk about why people get into programming as a new career, what they need to succeed, and what James thinks is the best indicator of success.

We also discuss what's new and exciting in JavaScript, James' favourite up & coming libraries, and dive into the JavaScript vs. TypeScript debate.

James’s YouTube channel: https://www.youtube.com/c/jamesqquick
James’s website: https://www.jamesqquick.com/
Astro website builder: https://astro.build/
James’ Astro course: https://astrocourse.dev/
Svelte framework: https://svelte.dev/
Vite build tool: https://vitejs.dev/

James on Twitter: https://twitter.com/jamesqquick
Kris on Twitter: https://twitter.com/krisajenkins
Kris on LinkedIn: https://www.linkedin.com/in/krisjenkins/

Sooner or later, every programmer will have to cross the gap between their programming language and their database. It feels like it should be easy, but in practice it’s always a much wider chasm than it seems, and every tool that bridges that gaps comes with its own strengths, weaknesses and opinions.

This week we take a look at a relatively new library for database access—jOOQ—by chatting with its author, Lukas Eder. This episode takes in the simple questions like syntax, the thornier ones like supporting multiple databases, and the deeply philosophical ones like how we even think about data and data-processing.

If you’re a Java (or JVM) programmer, there’s a new tool to learn here, and even if you're not there’s food for thought and ideas to borrow for the next time you need to SELECT…

jOOQ: https://www.jooq.org/
YesQL: https://github.com/krisajenkins/yesql
Datomic: https://www.datomic.com/
XTDB: https://www.xtdb.com/
The Elm Architecture: https://guide.elm-lang.org/architecture/

Kris on Twitter: https://twitter.com/krisajenkins
Kris on LinkedIn: https://www.linkedin.com/in/krisjenkins/

Whether you’re trying to ace the coding interview, sharpen your programming skills or just have some fun learning new things, the world of competitive coding has something to offer you. Some people join with dreams of hitting the podium, and plenty of others are just competing to be their better selves.

Either way, Mathis Hammel is a veteran of the competitive coding scene and he’s going to give us a view into that world, tell a few war stories and share some tips how you can play better, faster and stronger…

ICPC: https://icpc.global/
Advent of Code: https://adventofcode.com/
Advent of Code, Day 18, 2022: https://adventofcode.com/2022/day/18
Clash of Code: https://www.codingame.com/multiplayer/clashofcode
CodeForces: https://codeforces.com/
CodeWars: https://www.codewars.com/
HackerRank: https://www.hackerrank.com/

Mathis on Twitter: https://twitter.com/mathishammel
Kris on Twitter: https://twitter.com/krisajenkins
Kris on LinkedIn: https://www.linkedin.com/in/krisjenkins/

“Software development has not caught up with the internet age.” So says this week’s guest, Rúnar Bjarnason. But what does that mean? What would a programming language for the internet age look like?

Rúnar’s answer is Unison. A language that completely rethinks the way distributing computing can work, from the source code up. Borrowing some key ideas from git, it challenges the way we think about code-sharing, compilation, versioning and more. 

--

Kris on Twitter: https://twitter.com/krisajenkins
Kris on LinkedIn: https://www.linkedin.com/in/krisjenkins/
Rúnar on Twitter: https://twitter.com/runarorama
Rúnar’s book, Function Programming in Scala: https://amzn.to/46I9jew
Unison website: https://unison-lang.org

Complete and Easy Bidirectional Typechecking for Higher-Rank Polymorphism (pdf): https://www.cl.cam.ac.uk/~nk480/bidir.pdf
Do Be Do Be Do (pdf): https://arxiv.org/pdf/1611.09259.pdf
Rúnar’s Øredev conference talk: https://youtu.be/EgIVzOobD48

Cloud icons created by Freepik - Flaticon: https://www.flaticon.com/free-icons/cloud
Computer icons created by xnimrodx - Flaticon: https://www.flaticon.com/free-icons/computer

Have you ever been overwhelmed by the number of databases on offer? This week we welcome database expert Ben Stopford as a guide to help us map the database landscape and make sense of it all!

Join us as we embark on a journey through the history of databases, tracing the path from Edgar Codd to the multitude cloud-era of options available today. Discover the strengths of various database styles and explore the tradeoffs between general-purpose databases like #PostgreSQL and highly customised ones like #Cassandra or #Snowflake.

We delve into the realm of the cloud and the opportunities it brings, both for users and the database vendors themselves. And then we examine the challenges that arise when you're forced to connect multiple databases across an organisation. Should you look at Event Sourcing? Or Event Streaming, and how exactly do they differ?

Finally, we look towards the future, discussing Ben's vision of an ideal database and which programming language he would choose to build it in.

Kris on Twitter: https://twitter.com/krisajenkins
Kris on LinkedIn: https://www.linkedin.com/in/krisjenkins/
Kris on Mastodon: https://mastodon.social/@krisajenkins

LLMs like ChatGPT are not just fascinating, they're becoming increasing useful in our working lives. They've graduated from novelty to valuable tool. But building those tools is still in the hands of huge companies. Or is it?

In this week's episode of Developer Voices, we're learning how you can run LLMs on your own laptop, and how you can customize the system to make a tailored research assistant, a better documentation-searcher, and much more. All you need is a guide on which pieces you need, and how they fit together, and that's exactly what this week's guest—Tobi Fankhänel—is here to take us through.

A leaked memo from Google recently outlined how the Big Company Advantage has almost completely eroded, and how the next wave of LLM development is going to come from the open source community. So hackers rise up - the open source AI revolution begins now!

--

Kris on Twitter: https://twitter.com/krisajenkins
Kris on LinkedIn: https://www.linkedin.com/in/krisjenkins/
Tobias on LinkedIn: https://www.linkedin.com/in/tobias-fankh%C3%A4nel-749712180/
Tobias’ blog: https://blog.exxample.eu
LangChain: https://python.langchain.com/docs/get_started/introduction.html
Embeddings: https://weaviate.io/blog/vector-embeddings-explained
Vector Databases: https://en.wikipedia.org/wiki/Vector_database
"We have no moat" – Google Employee on Open-source LLMs: https://www.semianalysis.com/p/google-we-have-no-moat-and-neither
“Attention is all you need” - https://proceedings.neurips.cc/paper_files/paper/2017/file/3f5ee243547dee91fbd053c1c4a845aa-Paper.pdf
Timeline since Meta open-sourced their first-gen models: https://www.semianalysis.com/i/119223672/the-timeline
Run LLMs on CPU only or, since May, mix CPU and GPU usage: https://github.com/abetlen/llama-cpp-python
Samantha: https://erichartford.com/meet-samantha
Embedding model leaderboards: https://huggingface.co/spaces/mteb/leaderboard
Open-source LLMs: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard
LLaMA: https://ai.facebook.com/blog/large-language-model-llama-meta-ai/
Blog post: Design-pattern ‘In-context learning’ https://a16z.com/2023/06/20/emerging-architectures-for-llm-applications/#section--2
Tobi's GitHub branch ‘In-context learning with LangChain’ https://github.com/aviav/turmbauten/blob/spaghetti-code/CHANGELOG.md
Prompt Syntax Cheat Sheet: https://github.com/oobabooga/text-generation-webui/tree/main/characters/instruction-following
Google Workspace Labs Sign-Up: https://workspace.google.com/labs-sign-up/
GMail Workspace Labs Demo Video, click ‘See it in action’: https://workspace.google.com/solutions/ai/#m10
Prediction trading on open-source LLMs vs GPT-4: https://manifold.markets/PeterWildeford/will-i-peter-wildeford-think-that-t-c95ff3c1b385