In this Machine Learning System Design Review, Shaji Chennan Kunnummel walks us through the system design for Pinterest’s near-real-time architecture for detecting similar images. We discuss their usage of Kafka, Flink, rocksdb, and much more. Starting with the high-level requirements for the system, we discussed Pinterest’s focus on debuggability and an easy transition from their batch processing system to stream processing. We then touch on the different system interfaces and components involved such as Manas—Pinterest’s custom search engine—and how it all ends up in their custom graph database, downstream Kafka streams, and to Pinterest’s feature store—Galaxy. With Shaji’s expert knowledge of the system, we were able to do a deep dive into the system’s architecture and some of its components.