The growing demand for cloud services is driving the need to deliver an always-on and safe user experience in accessing their data and applications. Examples include web search, social networking, email, ecommerce, video streaming, data analytics and even mission-critical services such as power grid control. Such environments are required to be highly available and secure. This is often satisfied by having experts monitoring the system 24x7 to ensure that problems, if any, are resolved within a reasonable time. The need to solve a problem within the minimum time gives rise to a "whatever-it-takes-to-fix-the-problem" attitude amongst experts and produces a constant flow of informal text documenting the debugging steps taken to resolve problems. Understanding the content within this informal text at scale is the key to uncovering big problem trends that will enable us learn from mistakes and improve system design. In this talk, I will present NetSieve, a system that we built that aims to do automated problem inference from trouble tickets. Specifically, I will show you how statistical natural language processing (NLP) can be combined with knowledge representation, ontology modeling and human-guided learning to automatically analyze natural language text in trouble tickets to infer the problem symptoms, troubleshooting activities and resolution actions. I will further discuss fundamental challenges which arise when extracting meaning from such massive open-domain text corpora. Finally, I will then discuss how we applied NetSieve in a massive data center setting to automatically analyze 10K+ network trouble tickets and how we used these results to improve several key network operations.