The Backend Engineering Show with Hussein Nasser

Fundamentals of Operating Systems Course

This video is an overview of how the operating system kernel does socket management and the different data structures it utilizes to achieve that.

timestamps

0:00 Intro

1:38 Socket vs Connections

7:50 SYN and Accept Queue

18:56 Socket Sharding

23:14 Receive and Send buffers

27:00 Summary

Polling is the ability to interrogate a backend to see if a piece of information is ready. It can introduce a chatty system and as a result long polling was born. In this video I explain the beauty of this design pattern and how we can push it to its limit.

You get better as a software engineer when you go through these stages.

0:00 Intro 

1:15 Understand a technology

7:07 Articulate how it works

15:30 Understand its’ limitations

19:48 Try to build something better

27:45 Realize what you built also has limitations

32:48 Appreciate the original tech as is

 We use technologies all the time without knowing how it works. And it is ok not knowing how things work if interests isn’t there. But when there is interest to understand how something works, pursue it. It feels good when you understand how something works because you work better with it, you swim with the tide instead of against it. 

When I learned how TCP/IP work..  you would appreciate every connection request, how you read requests. You will ask questions,

 what is my code doing here? 

When exactly I’m creating connections?

When am I reading from the connection? 

Is it safe to share connections?

This one is not easy, you might think you understand something until you try to explain how it works. If you find yourself using jargon you probably don’t understand and you just try to impress others. Have you seen people who want to talk about something to show they understand it? It’s the opposite. Try to truly articlate how it works, you will really understand it , back to 1.

I thought I understand how backend reads requests until I tried to speak to it. 

Once 1,2 are done you will truly understand the tech, now you are confidant, you are excited about the tech and you will truly see when you can use the tech to its full potential and also know the weak points of the tech where it breaks, this happens a lot with TCP/IP. We know tcps limitations. 

This one is optional and can be skipped, but attempting to design or building something better then the tech because you know the limitations will truly reveal how you became better. But the challenge here is the ego, you might understand the limitations but you problem is thinking that what you will build is flawless. This step must be proceed with caution. 

Dust settles.. this step hurts, and you may take a while to realize it, but whatever you build will have flaws… and when you realize this it is when you get better as an engineer. 

This is when you are back full circle you are back to the first stage, look at the technology and understand it but don’t judge it.. just know the limitations and its strength and flow with it. Stop fighting and instead build around a tech, does that mean you shouldn’t build anything new, of course not. Go build, but don’t stress around making something better to defeat existing tech. But actually build it for building it.

Fundamentals of Operating Systems Course

Cloudflare built a global cache purge system that runs under 150 ms.

This is how they did it.

Using RockDB to maintain local CDN cache, and a peer-to-peer data center distributed system and clever engineering, they went from 1.5 second purge, down to 150 ms.

However, this isn’t full picture, because that 150 ms is just actually the P50. In this video I explore Clouldflare CDN work, how the old core-based centralized quicksilver, lazy purge work compared to the new coreless, decentralized active purge. In it I explore the pros and cons of both systems and give you my thoughts of this system.

0:00 Intro

4:25　From Core Base Lazy Purge to Coreless Active

12:50 CDN Basics

16:00 TTL Freshness

17:50 Purge

20:00 Core-Based Purge

24:00 Flexible Purges

26:36 Lazy Purge

30:00 Old Purge System Limitations

36:00 Coreless / Active Purge

39:00 LSM vs BTree

45:30 LSM Performance issues

48:00 How Active Purge Works

50:30 My thoughts about the new system

58:30 Summary

Cloudflare blog

https://blog.cloudflare.com/instant-purge/

Mentioned Videos

Cloudflare blog

https://blog.cloudflare.com/instant-purge/

Percentile Tail Latency Explained (95%, 99%) Monitor Backend performance with this metric

https://www.youtube.com/watch?v=3JdQOExKtUY

How Discord Stores Trillions of Messages | Deep Dive

https://www.youtube.com/watch?v=xynXjChKkJc

Fundamentals of Operating Systems Course

https://os.husseinnasser.com

Backend Troubleshooting Course

https://performance.husseinnasser.com

Fundamentals of Database Engineering udemy course https://databases.win

Fundamentals of Operating Systems Course

Fundamentals of Operating Systems Course

I am fascinated by how timeouts affect backend and frontend programming.

When a party is waiting on something you can place a timeout to break the wait. This is useful for freeing resources to more critical processes, detecting slow operations and even avoiding DOS attacks.

Contrary to common beliefs, timeouts are not exclusive to request processing, they can be applied to other parts of the frontend-backend communications. Let us explore this briefly.

0:00 Intro

2:30 Connection Timeout

5:00 Request Read timeout

10:00 Wait Timeout 

12:00 Usage Timeout

14:00 Response Timeout

16:00 Canceling a request

19:50 Proxies and timeouts

Learn more about database and OS internals, check out my courses 

Fundamentals of database engineering https://databases.win 

Fundamentals of operating systems https://oscourse.win

This new PostgreSQL 17 feature is game changer.

You see, postgres like most databases work with fixed size pages. Pretty much everything is in this format, indexes, table data, etc. Those pages are 8K in size, each page will have the rows, or index tuples and a fixed header. The pages are just bytes in files and they are read and cached in the buffer pool.

To read page 0, for example, you would call read on offset 0 for 8192 bytes, To read page 1 that is another read system call from offset 8193 for 8192, page 7 is offset 57,345 for 8192 and so on. 

If table is 100 pages stored a file, to do a full table scan, we would be making 100 system calls, each system call had an overhead (I talk about all of that in my OS course). 

The enhancement in Postgres 17 is to combine I/Os you can specify how much IO to combine, so technically while possible you can scan that entire table in one system call doesn’t mean its always a good idea of course and Ill talk about that. 

This also seems to included a vectorized I/O, with preadv system call which takes an array of offsets and lengths for random reads. 

The challenge will become how to not read too much, say I’m doing a seq scan to find something, I read page 0 and found it and quit I don’t need to read any more pages. With this feature I might read 10 pages in one I/O and pull all its content, put in shared buffers only to find my result in the first page (essentially wasting disk bandwidth, memory etc) 

It is going to be interesting to balance this out.