
Sign up to save your podcasts
Or


This paper details the evolution of Google's Spanner, a globally-distributed database system, from a key-value store to a fully-fledged SQL system. Key improvements discussed include distributed query execution, handling of transient failures via query restarts, efficient range extraction for data retrieval, and the adoption of a common SQL dialect. The authors also explain the transition from a Bigtable-like storage format to a more efficient blockwise-columnar store (Ressi). Finally, the paper highlights lessons learned during Spanner's large-scale deployment and outlines remaining challenges.
The article explores Change Data Capture (CDC), a method for tracking database changes, highlighting its advantages over traditional daily snapshots. It details three CDC implementation approaches: using database triggers (e.g., in PostgreSQL), capturing API requests and using a message broker (e.g., Kafka), and leveraging change streams within a data warehouse (e.g., Snowflake). The article compares these methods, weighing their pros and cons in terms of performance, scalability, and ease of implementation. A subsequent discussion critiques the presented methods, suggesting alternative, more robust solutions based on logical replication tools like Debezium.
This research paper introduces DeepSeek-R1, a large language model enhanced for reasoning capabilities using reinforcement learning (RL). Two versions are presented: DeepSeek-R1-Zero, trained purely via RL without supervised fine-tuning, and DeepSeek-R1, which incorporates additional multi-stage training and cold-start data for improved readability and performance. DeepSeek-R1 achieves results comparable to OpenAI's o1-1217 on various reasoning benchmarks. The study also explores distilling DeepSeek-R1's reasoning capabilities into smaller, more efficient models, achieving state-of-the-art results. Finally, the paper discusses unsuccessful attempts using process reward models and Monte Carlo Tree Search, providing valuable insights for future research.
https://github.com/deepseek-ai/DeepSeek-R1/blob/main/DeepSeek_R1.pdf
This Atlassian blog post details the migration of Jira Cloud's Issue Service from JSON to Protocol Buffers (Protobuf) to enhance performance. The switch involved a phased approach to minimise downtime, creating new endpoints and logic to handle both formats concurrently before a complete transition. The results showcased significant improvements: 75% less Memcached CPU usage, 80% smaller data size, and a substantially faster response time. Challenges encountered included Protobuf's handling of null values and incompatibility with Spring's default error controller, which required workarounds. Ultimately, the migration yielded substantial performance gains and reduced infrastructure needs.
https://www.atlassian.com/blog/atlassian-engineering/using-protobuf-to-make-jira-cloud-faster
This research paper introduces Hyaline, a novel family of memory reclamation schemes for lock-free data structures in unmanaged C/C++ code. Hyaline leverages reference counting, but only during reclamation, minimising overhead during object access and balancing workload across threads. The paper details Hyaline's design, including a scalable multi-list version and robust extensions to handle stalled threads. Extensive testing across multiple architectures demonstrates Hyaline's superior performance and memory efficiency compared to existing schemes like epoch-based reclamation and hazard pointers, particularly in read-dominated and oversubscribed scenarios. The paper concludes by proving Hyaline's correctness and lock-freedom properties.
This Atlassian blog post details Trello's migration from RabbitMQ to Kafka for its websocket architecture. RabbitMQ's unreliability during network partitions and high costs associated with queue creation and deletion prompted the switch. The article compares various queuing systems, highlighting Kafka's superior failover capabilities and in-order message delivery. Trello implemented a master-client architecture with Kafka, resulting in improved performance, reduced costs, and fewer outages. Key performance improvements included a 33% decrease in memory usage and a substantial cost reduction.
This podcast explores the field of reliability engineering, tracing its origins at Google with the development of Site Reliability Engineering (SRE). It differentiates reliability engineering from SRE, highlighting its broader applicability across various organisational structures. The podcast outlines four key promises of a successful reliability team: defining service levels (SLA/SLO/SLI), managing the service infrastructure, participating in technical design, and providing tactical support during incidents. Finally, it discusses the evolving landscape of reliability engineering, emphasising pragmatic approaches to balancing cost and reliability needs, and advocating for a more nuanced understanding of when to build versus buy solutions.
This podcast profiles Antithesis, a company developing a "multiverse debugger" for large, distributed systems. It traces the history of debugging tools, highlighting Antithesis's innovative approach using deterministic simulation testing (DST) to allow time travel debugging. The podcast includes a Q&A with Antithesis's co-founder, detailing the challenges of debugging large systems and how Antithesis addresses them. Furthermore, it discusses Antithesis's tech stack, engineering culture, and the trade-offs of using their complex, but potentially game-changing, technology. Finally, it considers the implications of widespread adoption of Antithesis's technology for the future of software development.
This podcast details the creation of Shopify's interactive Black Friday/Cyber Monday live dashboard, nicknamed "Live Globe". The 2024 version, built by a six-person team in two months, features a spaceship-themed interface showcasing real-time sales data and boasts impressive technical specifications, including peak loads of nearly 30 million database reads per second. The design process involved extensive prototyping and the use of AI-generated imagery for inspiration. The podcast also highlights the technology stack (React Three Fiber, Go, Rails, Kafka, and Flink), the inclusion of numerous Easter eggs, and the challenges of performance optimisation and real-time data streaming. Finally, it explores the project's unique approach to ROI, prioritising fun and innovation.
This podcast examines the contrasting "wartime" and "peacetime" operating modes in tech companies, drawing on the author's experiences at Uber and observations across the industry. It defines these modes in terms of leadership styles, employee behaviours, and organisational priorities, highlighting the differences in approaches to project management, performance reviews, and tech debt. The text explores the transitions between these modes, identifying common triggers and observable signs, and offers advice for employees and managers on thriving in each environment. Finally, it discusses the counterintuitive relationship between extended "wartime" periods and tech debt accumulation.