At Affirm, we lend billions of dollars to millions of customers using data-driven machine learning models that are trained on years of lending history. Feature extraction and validation in a risk management context presents unique challenges, so we have built a pipeline to eliminate training/serving skew in both development and production systems, and to run extensive regression testing on proposed PRs. Running Spark on Amazon EMR clusters provides an economic means to speed up this process. In this talk, we review our signal extraction and decisioning systems architecture and discuss how we use AWS to scale our compute.