Sign up to save your podcasts
Or
Share Your SSD lies but that's ok | Postgres fsync
Share to email
Share to Facebook
Share to X
Share to email
Share to Facebook
Share to X
Or
Your SSD lies but that's ok | Postgres fsync
fsync is a linux system call that flushes all pages and metadata for a given file to the disk. It is indeed an expensive operation but required for durability especially for database systems. Regular writes that make it to the disk controller are often placed in the SSD local cache to accumulate more writes before getting flushed to the NAND cells.
However when the disk controller receives this flush command it is required to immediately persist all of the data to the NAND cells.
Some SSDs however don't do that because they don't trust the host and no-op the fsync. In this video I explain this in details and go through details on how postgres provide so many options to fine tune fsync
0:00 Intro
1:00 A Write doesn’t write
2:00 File System Page Cache
6:00 Fsync
7:30 SSD Cache
9:20 SSD ignores the flush
9:30 15 Year old Firefox fsync bug
12:30 What happens if SSD loses power
15:00 What options does Postgres exposes?
15:30 open_sync (O_SYNC)
16:15 open_datasync (O_DSYNC)
17:10 O_DIRECT
19:00 fsync
20:50 fdatasync
21:13 fsync = off
23:30 Don’t make your API simple
26:00 Database on metal?
View all episodes
By Hussein Nasser
4.9
4040 ratings
fsync is a linux system call that flushes all pages and metadata for a given file to the disk. It is indeed an expensive operation but required for durability especially for database systems. Regular writes that make it to the disk controller are often placed in the SSD local cache to accumulate more writes before getting flushed to the NAND cells.
However when the disk controller receives this flush command it is required to immediately persist all of the data to the NAND cells.
Some SSDs however don't do that because they don't trust the host and no-op the fsync. In this video I explain this in details and go through details on how postgres provide so many options to fine tune fsync
0:00 Intro
1:00 A Write doesn’t write
2:00 File System Page Cache
6:00 Fsync
7:30 SSD Cache
9:20 SSD ignores the flush
9:30 15 Year old Firefox fsync bug
12:30 What happens if SSD loses power
15:00 What options does Postgres exposes?
15:30 open_sync (O_SYNC)
16:15 open_datasync (O_DSYNC)
17:10 O_DIRECT
19:00 fsync
20:50 fdatasync
21:13 fsync = off
23:30 Don’t make your API simple
26:00 Database on metal?
More shows like The Backend Engineering Show with Hussein Nasser
View all
Freakonomics Radio
32,245 Listeners
Software Engineering Radio - the podcast for professional software developers
273 Listeners
Risky Business
374 Listeners
Science Vs
12,172 Listeners
Syntax - Tasty Web Development Treats
987 Listeners
Darknet Diaries
8,107 Listeners
Practical AI
210 Listeners
Within Reason
1,658 Listeners
All-In with Chamath, Jason, Sacks & Friedberg
10,228 Listeners
Dwarkesh Podcast
541 Listeners
Big Technology Podcast
513 Listeners
Hard Fork
5,548 Listeners
The AI Daily Brief: Artificial Intelligence News and Analysis
655 Listeners
Prof G Markets
1,474 Listeners
The Pragmatic Engineer
74 Listeners