Postgres FM

Slow count


Listen Later

Nikolay and Michael discuss why counting can be slow in Postgres, and what the options are for counting things quickly at scale.
 
Here are some links to things they mentioned:

  • Aggregate functions (docs) https://www.postgresql.org/docs/current/functions-aggregate.html
  • PostgREST https://github.com/PostgREST/postgrest 
  • Get rid of count by default in PostgREST https://github.com/PostgREST/postgrest/issues/273 
  • Faster PostgreSQL Counting (by Joe Nelson on the Citus blog) https://www.citusdata.com/blog/2016/10/12/count-performance 
  • Our episode on Index-Only Scans https://postgres.fm/episodes/index-only-scans
  • Postgres HyperLogLog https://github.com/citusdata/postgresql-hll
  • Our episode on Row estimates https://postgres.fm/episodes/row-estimates 
  • Our episode about dangers of NULLs https://postgres.fm/episodes/nulls-the-good-the-bad-the-ugly-and-the-unknown 
  • Aggregate expressions, including FILTER https://www.postgresql.org/docs/current/sql-expressions.html#SYNTAX-AGGREGATES
  • Spread writes for counter cache (tip from Tobias Petry) https://x.com/tobias_petry/status/1475870220422107137
  • pg_ivm extension (Incremental View Maintenance) https://github.com/sraoss/pg_ivm 
  • pg_duckdb announcement https://motherduck.com/blog/pg_duckdb-postgresql-extension-for-duckdb-motherduck
  • Our episode on Queues in Postgres https://postgres.fm/episodes/queues-in-postgres
  • Our episode on Real-time analytics https://postgres.fm/episodes/real-time-analytics
  • ClickHouse acquired PeerDB https://clickhouse.com/blog/clickhouse-acquires-peerdb-to-boost-real-time-analytics-with-postgres-cdc-integration
  • Timescale Continuous Aggregates https://www.timescale.com/blog/materialized-views-the-timescale-way
  • Timescale editions https://docs.timescale.com/about/latest/timescaledb-editions
  • Loose indexscan https://wiki.postgresql.org/wiki/Loose_indexscan


~~~

What did you like or not like? What should we discuss next time? Let us know via a YouTube comment, on social media, or by commenting on our Google doc!


~~~

Postgres FM is produced by:

  • Michael Christofides, founder of pgMustard
  • Nikolay Samokhvalov, founder of Postgres.ai


With special thanks to:

  • Jessie Draws for the elephant artwork 
...more
View all episodesView all episodes
Download on the App Store

Postgres FMBy Nikolay Samokhvalov and Michael Christofides

  • 4.7
  • 4.7
  • 4.7
  • 4.7
  • 4.7

4.7

20 ratings


More shows like Postgres FM

View all
Hanselminutes with Scott Hanselman by Scott Hanselman

Hanselminutes with Scott Hanselman

377 Listeners

Software Engineering Radio - the podcast for professional software developers by se-radio@computer.org

Software Engineering Radio - the podcast for professional software developers

273 Listeners

The Changelog: Software Development, Open Source by Changelog Media

The Changelog: Software Development, Open Source

285 Listeners

Talk Python To Me by Michael Kennedy

Talk Python To Me

586 Listeners

Software Engineering Daily by Software Engineering Daily

Software Engineering Daily

631 Listeners

Python Bytes by Michael Kennedy and Brian Okken

Python Bytes

214 Listeners

Data Engineering Podcast by Tobias Macey

Data Engineering Podcast

138 Listeners

Syntax - Tasty Web Development Treats by Wes Bos & Scott Tolinski - Full Stack JavaScript Web Developers

Syntax - Tasty Web Development Treats

989 Listeners

CoRecursive: Coding Stories by Adam Gordon Bell - Software Developer

CoRecursive: Coding Stories

185 Listeners

Scaling Postgres by Creston Jamison

Scaling Postgres

12 Listeners

Kubernetes Podcast from Google by Abdel Sghiouar, Kaslin Fields

Kubernetes Podcast from Google

180 Listeners

The Real Python Podcast by Real Python

The Real Python Podcast

138 Listeners

Hard Fork by The New York Times

Hard Fork

5,363 Listeners

Oxide and Friends by Oxide Computer Company

Oxide and Friends

47 Listeners

The Pragmatic Engineer by Gergely Orosz

The Pragmatic Engineer

53 Listeners