November 18, 2016

Preslav Le on How Dropbox Moved off AWS and What They Have Been Able to Do Since

26 minutes

As InfoQ previously reported in March 2016, Dropbox announced that they had migrated away from Amazon Web Services (AWS).

In this week's podcast Robert Bluman talks to Preslav Le. Preslav has been a software engineer at Dropbox for the past three years, contributing to various aspects of Dropbox’s infrastructure including traffic, performance and storage. He was part of the core oncall and storage oncall rotations, dealing with high emergency real world issues, from bad code pushes to complete datacenter outages.

Why listen to this podcast:

- Dropbox migrated away from Amazon S3 to their own data centres to allow them to optimise for their specific use case.

- They are experimenting with Shingled Magnetic Recording (SMR) drives for primary storage to increase storage density. All writes go to an SSD cache and then get pushed asynchronously to the SMR disk.

- Their average block size is 1.6MB with a maximum block size of 4MB. Knowing this allows the team to tune their storage system.

- Three languages are used for the backend infrastructure. Python is used mainly for business logic, Go is the primary language used for heavy infrastructure services, and in some cases, for example where more direct control over memory is needed, Rust is also used.

- Dropbox invest very heavily in verification and automation. A verifier scans every byte on disk and checks that it matches the checksum in the index.

- Verification is also used to check that each box has the block keys it should have.

Notes and links can be found on http://bit.ly/preslav-le

Dropbox’s motivation for moving off the cloud

2:40 - Dropbox used Amazon S3 and other services where it made sense, but they stored all the metadata in their own data centres.

3:30 - Initially this was done because Amazon had poor support for persistent storage at the time. This has since improved but it didn’t make sense for dropbox to move the metadata back.

4:01 - By that time the dropbox team was ready to tackle the storage problem and built their own in-house replacement for S3, called Magic Pocket. Magic Pocket allowed Dropbox to move away from Amazon altogether.

4:30 - The move saved money, but also allowed DropBox to optimise for their specific use case and be faster.