No, this isn't an episode about how Facebook's algorithm and feed works. The data science function is popping up in companies small and large given the amount of data swimming around. No other company understand the power and influence that data science can have on the customer experience than Facebook (Meta, to be exact). Akos Lada is Facebook's Director of Data Science for Feed Ranking and Recommendations. Akos has always been interested in the intersection of social science and data, so this role at Facebook seems fitting. In this episode, Akos discusses what the analytics team does at Facebook, an analytics framework his team developed and open-sourced, A/B testing, and more.

What does the data science team do at Facebook?

I know the company is called Meta, but I grew up calling it Facebook, so I'm just going to stick with Facebook for now. The data science team actually consists of two teams: Analytics and Core Applied Data Science.

The Analytics team partners with product managers and engineers and their focus is on delivering long-term value for users (you'll hear a lot about this during this episode). There is also another data science team Akos used to work on, called Central Applied Science (formerly known as Core Data Science), which is a smaller team that focuses on scientific problems and research that every product team at Facebook might be able to benefit from. One of the frameworks the Central Applied science team created and open-sourced is called Ax. This framework helps optimize any kind of experiment including machine learning experiments, A/B tests, and simulations.

Making better decisions with the GTMF model

Akos' team published a blog post on four analytics best practices at Facebook which is worth a read. The impetus for this blog post was one question: how does Facebook drive more long-term value for users?

There are many different lenses you can put on to answer this question. Of course, Akos' team treats this question as a data science question. The Ground Truth Maturity Framework (GTMF) improves ground truth data--the data that powers Facebook's machine learning models. In a sense, the GTMF model ensures your data is clean. One place where GTMF is used is News Feed ranking. The team's ultimate goal with News Feed is trying to figure out if a post is something you would want to click on. You can read more about how machine learning is used in the News Feed algorithm here.

Running A/B experiments to figure out the right number of notifications to send to Facebook users

Akos discusses at length his team's experimentation frameworks. One interesting insight is that the longer his team kept experiments running (say one year) then the outcome of the experiment would change. One of the more surprising results from a long-term experiment his team ran was that if you send less notifications to users, it led to better long-term value for users (e.g. clicking on more posts). In the short-term, sending less notifications would naturally lead to less people engaging with posts.

At the end of the day, this is a behavioral science challenge. Given the amount of data Akos' team can analyze, they suggested that the product team drastically red...