Nikolay and Michael discuss why counting can be slow in Postgres, and what the options are for counting things quickly at scale.
Here are some links to things they mentioned:
- Aggregate functions (docs) https://www.postgresql.org/docs/current/functions-aggregate.html
- PostgREST https://github.com/PostgREST/postgrest
- Get rid of count by default in PostgREST https://github.com/PostgREST/postgrest/issues/273
- Faster PostgreSQL Counting (by Joe Nelson on the Citus blog) https://www.citusdata.com/blog/2016/10/12/count-performance
- Our episode on Index-Only Scans https://postgres.fm/episodes/index-only-scans
- Postgres HyperLogLog https://github.com/citusdata/postgresql-hll
- Our episode on Row estimates https://postgres.fm/episodes/row-estimates
- Our episode about dangers of NULLs https://postgres.fm/episodes/nulls-the-good-the-bad-the-ugly-and-the-unknown
- Aggregate expressions, including FILTER https://www.postgresql.org/docs/current/sql-expressions.html#SYNTAX-AGGREGATES
- Spread writes for counter cache (tip from Tobias Petry) https://x.com/tobias_petry/status/1475870220422107137
- pg_ivm extension (Incremental View Maintenance) https://github.com/sraoss/pg_ivm
- pg_duckdb announcement https://motherduck.com/blog/pg_duckdb-postgresql-extension-for-duckdb-motherduck
- Our episode on Queues in Postgres https://postgres.fm/episodes/queues-in-postgres
- Our episode on Real-time analytics https://postgres.fm/episodes/real-time-analytics
- ClickHouse acquired PeerDB https://clickhouse.com/blog/clickhouse-acquires-peerdb-to-boost-real-time-analytics-with-postgres-cdc-integration
- Timescale Continuous Aggregates https://www.timescale.com/blog/materialized-views-the-timescale-way
- Timescale editions https://docs.timescale.com/about/latest/timescaledb-editions
- Loose indexscan https://wiki.postgresql.org/wiki/Loose_indexscan
~~~
What did you like or not like? What should we discuss next time? Let us know via a YouTube comment, on social media, or by commenting on our Google doc!
~~~
Postgres FM is produced by:
- Michael Christofides, founder of pgMustard
- Nikolay Samokhvalov, founder of Postgres.ai
With special thanks to:
- Jessie Draws for the elephant artwork