July 24, 2025

Explorer: Data Frames in Elixir with Chris Grainger

42 minutes

In this episode of Elixir Wizards, Charles Suggs sits down with Chris Grainger, co-founder and CTO of Amplified and creator of the Explorer library. Chris explains how Explorer brings the familiar data-frame workflows of R’s dplyr and Python’s pandas into the Elixir world. We explore (pun intended!) how Explorer integrates with Ecto, Nx, and LiveView to build end-to-end data pipelines without leaving the BEAM, and how features like lazy evaluation and distributed frames let you tackle large datasets.

Whether you’re generating reports or driving interactive charts in LiveView, Explorer makes tabular data accessible to every Elixir developer. We wrap up by looking ahead to SQL-style backends, ADBC connectivity, and other features on the Explorer roadmap.

Key topics discussed in this episode:

dplyr- and pandas-inspired data manipulation in Elixir

Polars integration via Rust NIFs for blazing performance

Immutable data frames and BEAM-friendly concurrency

Lazy evaluation to work with arbitrarily large tables

Distributed data-frame support for multi-node processing

Seamless integration with Ecto schemas and queries

Zero-copy interoperability between Explorer and Nx tensors

Apache Arrow and ADBC protocols for cross-language I/O

Exploring SQL-style backends for remote query execution

Building interactive dashboards and charts in LiveView

Consolidating ETL workflows into a single Elixir API

Streaming data pipelines for memory-efficient processing

Tidy data principles and behavior-based API design

Real-world use cases: report generation, patent analysis, and more

Future roadmap: new backends, query optimizations, and community plugins

Links mentioned:

https://hexdocs.pm/explorer/Explorer.html

https://www.amplified.ai/

https://www.r-project.org/

https://vita.had.co.nz/papers/tidy-data.pdf

https://www.tidyverse.org/

https://www.python.org/

https://dplyr.tidyverse.org/

https://go.dev/

https://hexdocs.pm/nx/Nx.html

https://github.com/pola-rs/polars

https://github.com/rusterlium/rustler

https://www.rust-lang.org/

https://www.postgresql.org/

https://hexdocs.pm/ecto/Ecto.html

https://www.elastic.co/elasticsearch

https://arrow.apache.org/

Chris Grainger & Chris McCord Keynote ElixirConf 2024: https://youtu.be/4qoHPh0obv0

https://dbplyr.tidyverse.org/

https://spark.posit.co/

https://hexdocs.pm/pythonx/Pythonx.html

https://hexdocs.pm/vega_lite/VegaLite.html

10 Minutes to Explorer: https://hexdocs.pm/explorer/exploring_explorer.html

https://github.com/elixir-nx/scholar

https://scikit-learn.org/stable/

https://github.com/cigrainger

https://erlef.org/slack-invite/erlef

https://bsky.app/profile/cigrainger.bsky.social

https://github.com/cigrainger

...more

View all episodes

By SmartLogic LLC

4.9

2222 ratings

July 24, 2025

Explorer: Data Frames in Elixir with Chris Grainger

42 minutes

Key topics discussed in this episode:

dplyr- and pandas-inspired data manipulation in Elixir

Polars integration via Rust NIFs for blazing performance

Immutable data frames and BEAM-friendly concurrency

Lazy evaluation to work with arbitrarily large tables

Distributed data-frame support for multi-node processing

Seamless integration with Ecto schemas and queries

Zero-copy interoperability between Explorer and Nx tensors

Apache Arrow and ADBC protocols for cross-language I/O

Exploring SQL-style backends for remote query execution

Building interactive dashboards and charts in LiveView

Consolidating ETL workflows into a single Elixir API

Streaming data pipelines for memory-efficient processing

Tidy data principles and behavior-based API design

Real-world use cases: report generation, patent analysis, and more

Future roadmap: new backends, query optimizations, and community plugins

Links mentioned:

https://hexdocs.pm/explorer/Explorer.html

https://www.amplified.ai/

https://www.r-project.org/

https://vita.had.co.nz/papers/tidy-data.pdf

https://www.tidyverse.org/

https://www.python.org/

https://dplyr.tidyverse.org/

https://go.dev/

https://hexdocs.pm/nx/Nx.html

https://github.com/pola-rs/polars

https://github.com/rusterlium/rustler

https://www.rust-lang.org/

https://www.postgresql.org/

https://hexdocs.pm/ecto/Ecto.html

https://www.elastic.co/elasticsearch

https://arrow.apache.org/

Chris Grainger & Chris McCord Keynote ElixirConf 2024: https://youtu.be/4qoHPh0obv0

https://dbplyr.tidyverse.org/

https://spark.posit.co/

https://hexdocs.pm/pythonx/Pythonx.html

https://hexdocs.pm/vega_lite/VegaLite.html

10 Minutes to Explorer: https://hexdocs.pm/explorer/exploring_explorer.html

https://github.com/elixir-nx/scholar

https://scikit-learn.org/stable/

https://github.com/cigrainger

https://erlef.org/slack-invite/erlef

https://bsky.app/profile/cigrainger.bsky.social

https://github.com/cigrainger

...more

More shows like Elixir Wizards

View all

This American Life

91,109 Listeners

Software Engineering Radio

272 Listeners

Software Engineering Daily

625 Listeners

Practical AI

213 Listeners

Thinking Elixir Podcast

32 Listeners

Huberman Lab

29,124 Listeners

Beam Radio

11 Listeners

Software Unscripted

27 Listeners

Share Explorer: Data Frames in Elixir with Chris Grainger

Sign up to save your podcasts

Explorer: Data Frames in Elixir with Chris Grainger

Explorer: Data Frames in Elixir with Chris Grainger

More shows like Elixir Wizards

This American Life

Software Engineering Radio

Software Engineering Daily

Practical AI

Thinking Elixir Podcast

Huberman Lab

Beam Radio

Software Unscripted